ibllib.pipes.base_tasks

Abstract base classes for dynamic pipeline tasks.

Classes

AudioTask
BehaviourTask
DynamicTask
EphysTask
ExperimentDescriptionRegisterRaw	dict of list: custom sign off keys corresponding to specific devices
MesoscopeTask
RegisterRawDataTask	Base register raw task. To rename files 1. input and output must have the same length 2. output files must have full filename.
VideoTask
WidefieldTask

class DynamicTask(session_path, **kwargs)

Bases: Task

get_sync_collection(sync_collection=None)

get_sync(sync=None)

get_sync_extension(sync_ext=None)

get_sync_namespace(sync_namespace=None)

get_protocol(protocol=None, task_collection=None)

get_task_collection(collection=None)

get_device_collection(device, device_collection=None)

read_params_file()

class BehaviourTask(session_path, **kwargs)

Bases: DynamicTask

extractor = None

A trials extractor object.

Type:

ibllib.io.extractors.base.BaseBpodExtractor

get_protocol(protocol=None, task_collection=None)

Return the task protocol name.

This returns the task protocol based on the task collection. If protocol is not None, this acts as an identity function. If both task_collection and protocol are None, returns the protocol defined in the experiment description file only if a single protocol was run. If the task_collection is not None, the associated protocol name is returned.

Parameters:

• protocol (str) – A task protocol name. If not None, the same value is returned.

• task_collection (str) – The task collection whose protocol name to return. May be None if only one protocol run.

Returns:

The task protocol name, or None, if no protocol found.

Return type:

str, None

Raises:

ValueError – For session with multiple task protocols, a task collection must be passed.

get_task_collection(collection=None)

Return the task collection.

If collection is not None, this acts as an identity function. Otherwise loads it from the experiment description if only one protocol was run.

Parameters:

collection (str) – A task collection. If not None, the same value is returned.

Returns:

The task collection, or None if no task protocols were run.

Return type:

str, None

Raises:

AssertionError – Raised if multiple protocols were run and collection is None, or if experiment description file is improperly formatted.

get_protocol_number(number=None, task_protocol=None)

Return the task protocol number.

Numbering starts from 0. If the ‘protocol_number’ field is missing from the experiment description, None is returned. If task_protocol is None, the first protocol number if n protocols == 1, otherwise raises an AssertionError.

NB: ibllib.pipes.dynamic_pipeline.make_pipeline() will determine the protocol number from the order of the tasks in the experiment description if the task collection follows the pattern ‘raw_task_data_XX’. If the task protocol does not follow this pattern, the experiment description file should explicitly define the number with the ‘protocol_number’ field.

Parameters:

• number (int) – The protocol number. If not None, the same value is returned.

• task_protocol (str) – The task protocol name.

Returns:

The task protocol number, if defined.

Return type:

int, None

extract_behaviour(save=True)

Extract trials data.

This is an abstract method called by _run and run_qc methods. Subclasses should return the extracted trials data and a list of output files. This method should also save the trials extractor object to the :prop:`extractor` property for use by run_qc.

Parameters:

save (bool) – Whether to save the extracted data as ALF datasets.

Returns:

• dict – A dictionary of trials data.

• list of pathlib.Path – A list of output file paths if save == true.

run_qc(trials_data=None, update=True)

Run task QC.

Subclass method should return the QC object. This just validates the trials_data is not None.

Parameters:

• trials_data (dict) – A dictionary of extracted trials data. The output of extract_behaviour().

• update (bool) – If true, update Alyx with the QC outcome.

Returns:

A TaskQC object replete with task data and computed metrics.

Return type:

ibllib.qc.task_metrics.TaskQC

class VideoTask(session_path, cameras, **kwargs)

Bases: DynamicTask

extract_camera(save=True)

Extract trials data.

This is an abstract method called by _run and run_qc methods. Subclasses should return the extracted trials data and a list of output files. This method should also save the trials extractor object to the :prop:`extractor` property for use by run_qc.

Parameters:

save (bool) – Whether to save the extracted data as ALF datasets.

Returns:

• dict – A dictionary of trials data.

• list of pathlib.Path – A list of output file paths if save == true.

run_qc(camera_data=None, update=True)

Run camera QC.

Subclass method should return the QC object. This just validates the trials_data is not None.

Parameters:

• camera_data (dict) – A dictionary of extracted trials data. The output of extract_behaviour().

• update (bool) – If true, update Alyx with the QC outcome.

Returns:

A TaskQC object replete with task data and computed metrics.

Return type:

ibllib.qc.task_metrics.TaskQC

class AudioTask(session_path, **kwargs)

Bases: DynamicTask

class EphysTask(session_path, **kwargs)

Bases: DynamicTask

get_pname(pname)

get_nshanks(nshanks=None)

class WidefieldTask(session_path, **kwargs)

Bases: DynamicTask

class MesoscopeTask(session_path, **kwargs)

Bases: DynamicTask

get_signatures(**kwargs)

From the template signature of the task, create the exact list of inputs and outputs to expect based on the available device collection folders

Necessary because we don’t know in advance how many device collection folders (“imaging bouts”) to expect

load_sync()

Load the sync and channel map.

This method may be expanded to support other raw DAQ data formats.

Returns:

• one.alf.io.AlfBunch – A dictionary with keys (‘times’, ‘polarities’, ‘channels’), containing the sync pulses and the corresponding channel numbers.

• dict – A map of channel names and their corresponding indices.

class RegisterRawDataTask(session_path, **kwargs)

Bases: DynamicTask

Base register raw task. To rename files

1. input and output must have the same length

2. output files must have full filename

priority = 100

job_size = 'small'

rename_files(symlink_old=False)

register_snapshots(unlink=False, collection=None)

Register any photos in the snapshots folder to the session. Typically imaging users will take numerous photos for reference. Supported extensions: .jpg, .jpeg, .png, .tif, .tiff

If a .txt file with the same name exists in the same location, the contents will be added to the note text.

Parameters:

• unlink (bool) – If true, files are deleted after upload.

• collection (str, list, optional) – Location of ‘snapshots’ folder relative to the session path. If None, uses ‘device_collection’ attribute (if exists) or root session path.

Returns:

The newly registered Alyx notes.

Return type:

list of dict

Notes

• Animated GIF files are not resized and therefore may take up significant space on the database.

• TIFF files are converted to PNG format before upload. The original file is not replaced.

• JPEG and PNG files are resized by Alyx.

class ExperimentDescriptionRegisterRaw(session_path, **kwargs)

Bases: RegisterRawDataTask

dict of list: custom sign off keys corresponding to specific devices

sign_off_categories = {'neuropixel': ['raw', 'spike_sorting', 'alignment']}

property signature

The signature of the task specifies inputs and outputs for the given task. For some tasks it is dynamic and calculated. The legacy code specifies those as tuples. The preferred way is to use the ExpectedDataset input and output constructors.

I = ExpectedDataset.input O = ExpectedDataset.output signature = {

‘input_files’: [

I(name=’extract.me.npy’, collection=’raw_data’, required=True, register=False, unique=False),

], ‘output_files’: [

O(name=’look.atme.npy’, collection=’shiny_data’, required=True, register=True, unique=False)

]} is equivalent to: signature = {

‘input_files’: [(‘extract.me.npy’, ‘raw_data’, True, True)], ‘output_files’: [(‘look.atme.npy’, ‘shiny_data’, True)], }

Returns: