ibllib.pipes

IBL preprocessing pipeline.

This module concerns the data extraction and preprocessing for IBL data. The lab servers routinely call local_server.job_creator to search for new sessions to extract. The job creator registers the new session to Alyx (i.e. creates a new session record on the database), if required, then deduces a set of tasks (a.k.a. the pipeline [*]) from the ‘experiment.description’ file at the root of the session (see dynamic_pipeline.make_pipeline). If no file exists one is created, inferring the acquisition hardware from the task protocol. The new session’s pipeline tasks are then registered for another process (or server) to query.

Another process calls local_server.task_queue to get a list of queued tasks from Alyx, then local_server.tasks_runner to loop through tasks. Each task is run by called tasks.run_alyx_task with a dictionary of task information, including the Task class and its parameters.

Notes

All new tasks are subclasses of the base_tasks.DynamicTask class. All others are defunct and shall be removed in the future.

Functions

assign_task

param task_deck:

assign_task(task_deck, session_path, task, **kwargs)[source]
Parameters:
  • task_deck

  • session_path

  • task

  • kwargs

ibllib.pipes.audio_tasks

ibllib.pipes.base_tasks

Abstract base classes for dynamic pipeline tasks.

ibllib.pipes.behavior_tasks

Standard task protocol extractor dynamic pipeline tasks.

ibllib.pipes.dynamic_pipeline

Task pipeline creation from an acquisition description.

ibllib.pipes.ephys_alignment

ibllib.pipes.ephys_preprocessing

(Deprecated) Electrophysiology data preprocessing tasks.

ibllib.pipes.ephys_tasks

ibllib.pipes.histology

ibllib.pipes.local_server

Lab server pipeline construction and task runner.

ibllib.pipes.mesoscope_tasks

The mesoscope data extraction pipeline.

ibllib.pipes.misc

Miscellaneous pipeline utility functions.

ibllib.pipes.photometry_tasks

Extraction tasks for fibrephotometry

ibllib.pipes.purge_rig_data

Purge data from acquisition PC.

ibllib.pipes.remote_server

ibllib.pipes.scan_fix_passive_files

ibllib.pipes.sync_tasks

ibllib.pipes.tasks

The abstract Pipeline and Task superclasses and concrete task runner.

ibllib.pipes.training_preprocessing

(Deprecated) Training data preprocessing tasks.

ibllib.pipes.training_status

ibllib.pipes.transfer_rig_data

ibllib.pipes.video_tasks

ibllib.pipes.widefield_tasks

The widefield data extraction pipeline.