Loading Raw Video Data

Raw video data recorded from body, left and right cameras

Relevant Alf objects

• bodyCamera

• leftCamera

• rightCamera

Loading

Option 1: Stream single frame

from one.api import ONE
import ibllib.io.video as vidio

one = ONE()
eid = '4ecb5d24-f5cc-402c-be28-9d0f7cb14b3a'
label = 'body' # 'left', 'right' or 'body'

# Find url of video data to stream
url = vidio.url_from_eid(eid, one=one)[label]

# Load video timestamps
ts = one.load_dataset(eid, f'*{label}Camera.times*', collection='alf')

# Find the frame closest to 1000s into data
import numpy as np
frame_n = np.searchsorted(ts, 1000)

# Stream the data
frame = vidio.get_video_frame(url, frame_n)

/home/runner/Downloads/ONE/openalyx.internationalbrainlab.org/hoferlab/Subjects/SWC_043/2020-09-21/001/alf/_ibl_bodyCamera.times.npy: 100%|██████████| 963k/963k [00:00<00:00, 5.84MB/s]

Option 2: Stream multiple frames (see also Example 2)

# Load the first 10 video frames
frames = vidio.get_video_frames_preload(url, range(10))

loading frame 9/10

Option 3: Downloading full video data

Warning.

The raw video data is very large and downloading will take a long period of time

video_body = one.load_dataset(eid, f'*{label}Camera.raw*', collection='raw_video_data')

/home/runner/Downloads/ONE/openalyx.internationalbrainlab.org/hoferlab/Subjects/SWC_043/2020-09-21/001/raw_video_data/_iblrig_bodyCamera.raw.mp4: 100%|██████████| 1.21G/1.21G [00:06<00:00, 182MB/s]

More details

• Description of raw video datasets

• Description of video qc metrics

• IBL video white paper

Useful modules

• ibllib.io.video

• ibllib.qc.camera

Exploring raw video data

Example 1: Obtaining video meta data

meta = vidio.get_video_meta(url, one=one)
for k, v in meta.items():
    print(f'The video {k} = {v}')

The video length = 120355
The video fps = 30
The video width = 640
The video height = 512
The video duration = 1:06:51.833333
The video size = 1214614838

Example 2: Efficiently loading multiple frames

# The preload function will by default pre-allocate the memory before loading the frames,
# and will return the frames as a numpy array of the shape (l, h, w, 3), where l = the number of
# frame indices given.  The indices must be an iterable of positive integers.  Because the videos
# are in black and white the values of each color channel are identical.   Therefore to save on
# memory you can provide a slice that returns only one of the three channels for each frame.  The
# resulting shape will be (l, h, w).  NB: Any slice or boolean array may be provided which is
# useful for cropping to an ROI.
#
# If you don't need to apply operations over all the fetched frames you can use the `as_list`
# kwarg to return the frames as a list.  This is slightly faster than fetching as an ndarray.
#
# A warning is printed if fetching a frame fails.  The affected frames will be returned as zeros
# or None if `as_list` is True.

frames = vidio.get_video_frames_preload(url, range(10), mask=np.s_[:, :, 0])

loading frame 9/10

Example 3: Computing Video QC for camera

from ibllib.qc.camera import CameraQC
qc = CameraQC(one.eid2path(eid), 'body', download_data=True)
outcome, extended = qc.run()
print(f'video QC = {outcome}')
extended

Information about individual qc checks can be found by looking at the docstring (replace _videoBody with check), e.g.

help(CameraQC.check_dropped_frames)

Other relevant examples

• COMING SOON