Loading Raw Video Data

Raw video data recorded from body, left and right cameras

Relevant Alf objects

  • bodyCamera

  • leftCamera

  • rightCamera

Loading

Option 1: Stream single frame

[2]:
from one.api import ONE
import ibllib.io.video as vidio

one = ONE()
eid = '4ecb5d24-f5cc-402c-be28-9d0f7cb14b3a'
label = 'body' # 'left', 'right' or 'body'

# Find url of video data to stream
url = vidio.url_from_eid(eid, one=one)[label]

# Load video timestamps
ts = one.load_dataset(eid, f'*{label}Camera.times*', collection='alf')

# Find the frame closest to 1000s into data
import numpy as np
frame_n = np.searchsorted(ts, 1000)

# Stream the data
frame = vidio.get_video_frame(url, frame_n)

Option 2: Stream multiple frames (see also Example 2)

[3]:
# Load the first 10 video frames
frames = vidio.get_video_frames_preload(url, range(10))
loading frame 9/10

Option 3: Downloading full video data

Warning.

The raw video data is very large and downloading will take a long period of time

[4]:
video_body = one.load_dataset(eid, f'*{label}Camera.raw*', collection='raw_video_data')
100%|██████████████████████████████████████████████████████████████████████████████████| 5/5.0 [00:02<00:00,  1.92it/s]

More details

Useful modules

Exploring raw video data

Example 1: Obtaining video meta data

[5]:
meta = vidio.get_video_meta(url, one=one)
for k, v in meta.items():
    print(f'The video {k} = {v}')
The video length = 120355
The video fps = 30
The video width = 640
The video height = 512
The video duration = 1:06:51.833333
The video size = 1214614838.0

Example 2: Efficiently loading multiple frames

[6]:
# The preload function will by default pre-allocate the memory before loading the frames,
# and will return the frames as a numpy array of the shape (l, h, w, 3), where l = the number of
# frame indices given.  The indices must be an iterable of positive integers.  Because the videos
# are in black and white the values of each color channel are identical.   Therefore to save on
# memory you can provide a slice that returns only one of the three channels for each frame.  The
# resulting shape will be (l, h, w).  NB: Any slice or boolean array may be provided which is
# useful for cropping to an ROI.
#
# If you don't need to apply operations over all the fetched frames you can use the `as_list`
# kwarg to return the frames as a list.  This is slightly faster than fetching as an ndarray.
#
# A warning is printed if fetching a frame fails.  The affected frames will be returned as zeros
# or None if `as_list` is True.

frames = vidio.get_video_frames_preload(url, range(10), mask=np.s_[:, :, 0])
loading frame 9/10

Example 3: Computing Video QC for camera

[7]:
from ibllib.qc.camera import CameraQC
qc = CameraQC(one.eid2path(eid), 'body', download_data=True)
outcome, extended = qc.run()
print(f'video QC = {outcome}')
extended
2022-06-06 12:02:37.596 INFO     [camera.py:345] Computing QC outcome for body camera, session 4ecb5d24-f5cc-402c-be28-9d0f7cb14b3a
Downloading: C:\Users\Mayo\Downloads\FlatIron\hoferlab\Subjects\SWC_043\2020-09-21\001\raw_video_data\_iblrig_bodyCamera.frame_counter.56cd0ba8-d1c0-4936-9a22-58721dfdef25.bin Bytes: 962840
100%|██████████████████████████████████████████████████| 0.9182357788085938/0.9182357788085938 [00:02<00:00,  2.57s/it]
Downloading: C:\Users\Mayo\Downloads\FlatIron\hoferlab\Subjects\SWC_043\2020-09-21\001\raw_video_data\_iblrig_bodyCamera.GPIO.64507d74-1af4-41e1-b785-680d801e325a.bin Bytes: 962840
100%|██████████████████████████████████████████████████| 0.9182357788085938/0.9182357788085938 [00:02<00:00,  2.56s/it]
Downloading: C:\Users\Mayo\Downloads\FlatIron\hoferlab\Subjects\SWC_043\2020-09-21\001\raw_video_data\_iblrig_bodyCamera.timestamps.cb336885-2bba-43d0-9e25-e6ed316ab22d.ssv Bytes: 5625912
100%|████████████████████████████████████████████████████| 5.365287780761719/5.365287780761719 [00:06<00:00,  1.28s/it]
Downloading: C:\Users\Mayo\Downloads\FlatIron\hoferlab\Subjects\SWC_043\2020-09-21\001\raw_behavior_data\_iblrig_taskData.raw.7bcaf2fc-6cb0-45a4-9708-193904026721.jsonable Bytes: 15272842
100%|██████████████████████████████████████████████████| 14.565317153930664/14.565317153930664 [00:10<00:00,  1.37it/s]
Downloading: C:\Users\Mayo\Downloads\FlatIron\hoferlab\Subjects\SWC_043\2020-09-21\001\raw_behavior_data\_iblrig_taskSettings.raw.5c426c45-3e1f-4f90-b3f7-bca76c6e0725.json Bytes: 141300
100%|████████████████████████████████████████████████| 0.13475418090820312/0.13475418090820312 [00:00<00:00,  4.08s/it]
Downloading: C:\Users\Mayo\Downloads\FlatIron\hoferlab\Subjects\SWC_043\2020-09-21\001\raw_video_data\_iblrig_bodyCamera.raw.face92e8-c8e9-4d10-8a89-525acc3381e2.mp4 Bytes: 1214614838
100%|████████████████████████████████████████████████████| 1158.346975326538/1158.346975326538 [03:29<00:00,  5.52it/s]
Downloading: C:\Users\Mayo\Downloads\FlatIron\hoferlab\Subjects\SWC_043\2020-09-21\001\alf\_ibl_bodyCamera.times.e2723912-e6fd-45b7-b203-d2a18c46071a.npy Bytes: 962968
100%|██████████████████████████████████████████████████| 0.9183578491210938/0.9183578491210938 [00:02<00:00,  2.58s/it]
Downloading: C:\Users\Mayo\Downloads\FlatIron\hoferlab\Subjects\SWC_043\2020-09-21\001\alf\_ibl_wheel.position.67b3a4fb-29fc-40d2-a4fd-477689ba0aa2.npy Bytes: 8081024
100%|████████████████████████████████████████████████████████| 7.7066650390625/7.7066650390625 [00:08<00:00,  1.04s/it]
Downloading: C:\Users\Mayo\Downloads\FlatIron\hoferlab\Subjects\SWC_043\2020-09-21\001\alf\_ibl_wheel.timestamps.53090473-9474-4a23-8cea-c5e062c6120e.npy Bytes: 8081024
100%|████████████████████████████████████████████████████████| 7.7066650390625/7.7066650390625 [00:08<00:00,  1.04s/it]
Downloading: C:\Users\Mayo\Downloads\FlatIron\hoferlab\Subjects\SWC_043\2020-09-21\001\raw_ephys_data\_spikeglx_sync.channels.495dfc0f-3450-4f75-8629-055e16e053a3.npy Bytes: 23694848
100%|██████████████████████████████████████████████████████████| 22.59716796875/22.59716796875 [00:12<00:00,  1.86it/s]
Downloading: C:\Users\Mayo\Downloads\FlatIron\hoferlab\Subjects\SWC_043\2020-09-21\001\raw_ephys_data\_spikeglx_sync.polarities.2e0f57b0-8fc9-4d08-b16d-6b9e4009deb5.npy Bytes: 23694848
100%|██████████████████████████████████████████████████████████| 22.59716796875/22.59716796875 [00:17<00:00,  1.32it/s]
Downloading: C:\Users\Mayo\Downloads\FlatIron\hoferlab\Subjects\SWC_043\2020-09-21\001\raw_ephys_data\_spikeglx_sync.times.eb04d3e9-5e67-4833-8321-fbccb3e9699b.npy Bytes: 23694848
100%|██████████████████████████████████████████████████████████| 22.59716796875/22.59716796875 [00:16<00:00,  1.41it/s]
Downloading: C:\Users\Mayo\Downloads\FlatIron\hoferlab\Subjects\SWC_043\2020-09-21\001\raw_ephys_data\_spikeglx_ephysData_g0_t0.nidq.64924ee8-395e-41f8-ac07-c74f853a8671.meta Bytes: 1016
100%|████████████████████████████████████████████████| 0.00096893310546875/0.00096893310546875 [00:00<00:00,  3.09s/it]
2022-06-06 12:07:53.608 INFO     [camera.py:186] Gathering data for QC
2022-06-06 12:07:53.696 WARNING  [ephys_fpga.py:93] Keys missing from provided channel map, setting missing keys from default channel map
2022-06-06 12:08:07.276 INFO     [camera.py:245] Inspecting video file...
video QC = PASS9/100
Out[7]:
{'_videoBody_brightness': 'PASS',
 '_videoBody_camera_times': ('PASS', 0),
 '_videoBody_dropped_frames': 'NOT_SET',
 '_videoBody_file_headers': 'PASS',
 '_videoBody_focus': 'PASS',
 '_videoBody_framerate': ('PASS', 29.943),
 '_videoBody_pin_state': 'NOT_SET',
 '_videoBody_position': 'PASS',
 '_videoBody_resolution': 'PASS',
 '_videoBody_timestamps': 'PASS',
 '_videoBody_wheel_alignment': 'NOT_SET'}

Information about individual qc checks can be found by looking at the docstring (replace _videoBody with check), e.g.

[8]:
help(CameraQC.check_dropped_frames)
Help on function check_dropped_frames in module ibllib.qc.camera:

check_dropped_frames(self, threshold=0.1)
    Check how many frames were reported missing

    :param threshold: The maximum allowable percentage of dropped frames

Other relevant examples

  • COMING SOON