Get video frames and metadata

Video frames and meta data can be loaded using the ibllib.io.video module, which contains functions for loading individual or groups of frames efficiently. The video may be streamed remotely or loaded from a local file. In these examples a remote URL is used.

import numpy as np
from one.api import ONE

import ibllib.io.video as vidio

one = ONE(base_url='https://openalyx.internationalbrainlab.org', silent=True)
eid = 'c7bd79c9-c47e-4ea5-aea3-74dda991b48e'  # 2020-09-19_1_CSH_ZAD_029

# Example 1: get the remote video URL from eid
urls = vidio.url_from_eid(eid, one=one)
# Without the `label` kwarg, returns a dictionary of camera URLs
url = urls['left']  # URL for the left camera

# Example 2: get the video label from a video file path or URL
label = vidio.label_from_path(url)
print(f'Using URL for the {label} camera')

# Example 3: loading a single frame
frame_n = 1000  # Frame number to fetch.  Indexing starts from 0.
frame = vidio.get_video_frame(url, frame_n)
assert frame is not None, 'failed to load frame'

# Example 4: loading multiple frames
"""
The preload function will by default pre-allocate the memory before loading the frames,
and will return the frames as a numpy array of the shape (l, h, w, 3), where l = the number of
frame indices given.  The indices must be an iterable of positive integers.  Because the videos
are in black and white the values of each color channel are identical.   Therefore to save on
memory you can provide a slice that returns only one of the three channels for each frame.  The
resulting shape will be (l, h, w).  NB: Any slice or boolean array may be provided which is
useful for cropping to an ROI.

If you don't need to apply operations over all the fetched frames you can use the `as_list`
kwarg to return the frames as a list.  This is slightly faster than fetching as an ndarray.

A warning is printed if fetching a frame fails.  The affected frames will be returned as zeros
or None if `as_list` is True.
"""
frames = vidio.get_video_frames_preload(url, range(10), mask=np.s_[:, :, 0])

# Example 5: load video meta-data
"""
You can load all the information for a given video.  In order to load the video size from a URL
an instance of ONE must be provided, otherwise this entry will be blank. An Bunch is returned
with a number of fields.
"""
meta = vidio.get_video_meta(url, one=one)
for k, v in meta.items():
    print(f'The video {k} = {v}')

# Example 6: load video timestamps
ts = one.load_dataset(eid, f'_ibl_{label}Camera.times.npy', collection='alf')

# Example 7: load dlc
cam = one.load_object(eid, f'{label}Camera', collection='alf')
print(cam.keys())
print(cam.dlc.columns)

Using URL for the left camera
The video length = 364739
The video fps = 60
The video width = 1280
The video height = 1024
The video duration = 1:41:18.983333
The video size = 4468082511.0
dict_keys(['dlc', 'times'])
Index(['nose_tip_x', 'nose_tip_y', 'nose_tip_likelihood', 'pupil_top_r_x',
       'pupil_top_r_y', 'pupil_top_r_likelihood', 'pupil_right_r_x',
       'pupil_right_r_y', 'pupil_right_r_likelihood', 'pupil_bottom_r_x',
       'pupil_bottom_r_y', 'pupil_bottom_r_likelihood', 'pupil_left_r_x',
       'pupil_left_r_y', 'pupil_left_r_likelihood', 'paw_l_x', 'paw_l_y',
       'paw_l_likelihood', 'paw_r_x', 'paw_r_y', 'paw_r_likelihood',
       'tube_top_x', 'tube_top_y', 'tube_top_likelihood', 'tube_bottom_x',
       'tube_bottom_y', 'tube_bottom_likelihood', 'tongue_end_l_x',
       'tongue_end_l_y', 'tongue_end_l_likelihood', 'tongue_end_r_x',
       'tongue_end_r_y', 'tongue_end_r_likelihood'],
      dtype='object')