ibllib.pipes.tasks

Functions

run_alyx_task

Runs a single Alyx job and registers output datasets :param tdict: :param session_path: :param one: :param job_deck: optional list of job dictionaries belonging to the session.

Classes

Pipeline

Pipeline class: collection of related and potentially interdependent tasks

Task

class Task(session_path, parents=None, taskid=None, one=None, machine=None, clobber=False)[source]

Bases: abc.ABC

cpu = 1
gpu = 0
io_charge = 5
priority = 30
ram = 4
level = 0
outputs = None
time_elapsed_secs = None
time_out_secs = None
version = '1.10.2'
log = ''
one = None
property name
run(**kwargs)[source]

— do not overload, see _run() below— wraps the _run() method with - error management - logging to variable

register_datasets(one=None, **kwargs)[source]

Register output datasets form the task to Alyx :param one: :param jobid: :param kwargs: directly passed to the register_dataset function :return:

rerun()[source]
setUp()[source]

Function to optionally overload to check inputs. :return:

tearDown()[source]

Function to optionally overload to check results :return:

class Pipeline(session_path=None, one=None, eid=None)[source]

Bases: abc.ABC

Pipeline class: collection of related and potentially interdependent tasks

tasks = {}
one = None
make_graph(out_dir=None, show=True)[source]
create_alyx_tasks(rerun__status__in=None)[source]

Instantiate the pipeline and create the tasks in Alyx, then create the jobs for the session If the jobs already exist, they are left untouched. The re-run parameter will re-init the job by emptying the log and set the status to Waiting :param rerun__status__in: by default no re-run. To re-run tasks if they already exist, specify a list of statuses string that will be re-run, those are the possible choices: [‘Waiting’, ‘Started’, ‘Errored’, ‘Empty’, ‘Complete’] to always patch, the string ‘__all__’ can also be provided :return: list of alyx tasks dictionaries (existing and or created)

run(status__in=['Waiting'], machine=None, clobber=False, **kwargs)[source]

Get all the session related jobs from alyx and run them :param status__in: lists of status strings to run in [‘Waiting’, ‘Started’, ‘Errored’, ‘Empty’, ‘Complete’] :param kwargs: arguments passed downstream to run_alyx_task :return: jalyx: list of REST dictionaries of the job endpoints :return: job_deck: list of REST dictionaries of the jobs endpoints :return: all_datasets: list of REST dictionaries of the dataset endpoints

rerun_failed(**kwargs)[source]
rerun(**kwargs)[source]
property name
run_alyx_task(tdict=None, session_path=None, one=None, job_deck=None, max_md5_size=None, machine=None, clobber=False)[source]

Runs a single Alyx job and registers output datasets :param tdict: :param session_path: :param one: :param job_deck: optional list of job dictionaries belonging to the session. Needed to check dependency status if the jdict has a parent field. If jdict has a parent and job_deck is not entered, will query the database :param max_md5_size: in bytes, if specified, will not compute the md5 checksum above a given filesize to save time :param machine: string identifying the machine the task is run on, optional :return: