ibllib.io.session_params

A module for handling experiment description files.

Each device computer adds its piece of information and consolidates into the final acquisition description.

The purpose is 3-fold:

• provide modularity in the extraction: the acquisition description allows to dynamically build

pipelines. - assist the copying of the experimental data from device computers to the server computers, in a way that each device is independent from another. - assist the copying of the experimental data from device computers to the server computers, in a way that intermediate states (failed copies) are easily recoverable from and completion criteria (ie. session ready to extract) is objective and simple (all device files copied).

INGRESS

• each device computer needs to know the session path on the server.

• create a device file locally in a queue directory. This will serve as a copy flag.

• copy the device file to the local server.

EGRESS

• go through the queue and for each item:

  • if the device file is not on the server create it.

  • once copy is complete aggregate the qc from file.

Functions

aggregate_device	Add the contents of a device file to the main acquisition description file.
get_cameras
get_collections	Find all collections associated with the session.
get_remote_stub_name	Get or create device specific file path for the remote experiment.description stub.
get_sync
get_sync_collection
get_sync_extension
get_sync_label
get_sync_namespace
get_sync_values
get_task_collection	Fetch the task collection from an experiment description dict.
get_task_protocol	Fetch the task protocol from an experiment description dict.
get_task_protocol_number	Fetch the task protocol number from an experiment description dict.
get_video_compressed
merge_params	Given two experiment descriptions, update first with fields in second.
read_params	Load an experiment description file.
write_params	Write acquisition description data to the session path.
write_yaml	Write a device file.

write_yaml(file_path, data)

Write a device file. This is basically just a yaml dump that ensures the folder tree exists.

Parameters:

• file_path (pathlib.Path) – The full path to the description yaml file to write to.

• data (dict) – The data to write to the yaml file.

write_params(session_path, data) → Path

Write acquisition description data to the session path.

Parameters:

• session_path (str, pathlib.Path) – A session path containing an _ibl_experiment.description.yaml file.

• data (dict) – The acquisition description data to save

Returns:

The full path to the saved acquisition description.

Return type:

pathlib.Path

read_params(path) → dict

Load an experiment description file.

In addition to reading the yaml data, this functions ensures that the specification is the most recent one. If the file is missing None is returned. If the file cannot be parsed an empty dict is returned.

Parameters:

path (pathlib.Path, str) – The path to the description yaml file (or it’s containing folder) to load.

Returns:

The parsed yaml data, or None if the file was not found.

Return type:

dict, None

Examples

# Load a session’s _ibl_experiment.description.yaml file

>>> data = read_params('/home/data/subject/2020-01-01/001')

# Load a specific device’s description file

>>> data = read_params('/home/data/subject/2020-01-01/001/_devices/behaviour.yaml')

merge_params(a, b, copy=False)

Given two experiment descriptions, update first with fields in second.

Parameters:

• a (dict) – An experiment description dictionary to be updated with fields from b.

• b (dict) – An experiment description dictionary to update a with

• copy (bool) – If true, return a deep copy of a instead of updating directly.

Returns:

A merged dictionary consisting of fields from a and b.

Return type:

dict

aggregate_device(file_device, file_acquisition_description, unlink=False)

Add the contents of a device file to the main acquisition description file.

Parameters:

• file_device (pathlib.Path) – The full path to the device yaml file to add to the main description file.

• file_acquisition_description (pathlib.Path) – The full path to the main acquisition description yaml file to add the device file to.

• unlink (bool) – If True, the device file is removed after successfully aggregation.

Returns:

The aggregated experiment description data.

Return type:

dict

Raises:

AssertionError – Device file contains a main ‘sync’ key that is already present in the main description file. For an experiment only one main sync device is allowed.

get_cameras(sess_params)

get_sync_label(sess_params)

get_sync(sess_params)

get_sync_values(sess_params)

get_sync_collection(sess_params)

get_sync_extension(sess_params)

get_sync_namespace(sess_params)

get_task_protocol(sess_params, task_collection=None)

Fetch the task protocol from an experiment description dict.

Parameters:

• sess_params (dict) – The loaded experiment.description file.

• task_collection (str, optional) – Return the protocol that corresponds to this collection (returns the first matching protocol in the list). If None, all protocols are returned.

Returns:

If task_collection is None, returns the set of task protocols, otherwise returns the first protocol that corresponds to the collection, or None if collection not present.

Return type:

str, set, None

get_task_collection(sess_params, task_protocol=None)

Fetch the task collection from an experiment description dict.

Parameters:

• sess_params (dict) – The loaded experiment.description file.

• task_protocol (str, optional) – Return the collection that corresponds to this protocol (returns the first matching protocol in the list). If None, all collections are returned.

Returns:

If task_protocol is None, returns the set of collections, otherwise returns the first collection that corresponds to the protocol, or None if protocol not present.

Return type:

str, set, None

Notes

• The order of the set may not be the same as the descriptions tasks order when iterating.

get_task_protocol_number(sess_params, task_protocol=None)

Fetch the task protocol number from an experiment description dict.

Parameters:

• sess_params (dict) – The loaded experiment.description file.

• task_protocol (str, optional) – Return the number that corresponds to this protocol (returns the first matching protocol in the list). If None, all numbers are returned.

Returns:

If task_protocol is None, returns list of all numbers, otherwise returns the first number that corresponds to the protocol, or None if protocol not present.

Return type:

str, list, None

get_collections(sess_params, flat=False)

Find all collections associated with the session.

Parameters:

• sess_params (dict) – The loaded experiment description map.

• flat (bool (False)) – If True, return a flat set of collections, otherwise return a map of device/sync/task

Returns:

• dict[str, str] – A map of device/sync/task and the corresponding collection name.

• set[str] – A set of unique collection names.

Notes

• Assumes only the following data types contained: list, dict, None, str.

get_video_compressed(sess_params)

get_remote_stub_name(session_path, device_id=None)

Get or create device specific file path for the remote experiment.description stub.

Parameters:

• session_path (pathlib.Path) – A remote session path.

• device_id (str, optional) – A device name, if None the TRANSFER_LABEL parameter is used (defaults to this device’s hostname with a unique numeric ID)

Returns:

The full file path to the remote experiment description stub.

Return type:

pathlib.Path

Example

>>> get_remote_stub_name(Path.home().joinpath('subject', '2020-01-01', '001'), 'host-123')
Path.home() / 'subject/2020-01-01/001/_devices/2020-01-01_1_subject@host-123.yaml'