ibllib.pipes.ephys_tasks

Classes

CellQCMixin	This mixin class is used to compute the cell QC metrics and update the json field of the probe insertion The compute_cell_qc method is static and can be used independently.
EphysCompressNP1
EphysCompressNP21
EphysCompressNP24	Compresses NP2.4 data by splitting into N binary files, corresponding to N shanks
EphysPulses	Extract Pulses from raw electrophysiology data into numpy arrays Perform the probes synchronisation with nidq (3B) or main probe (3A) First the job extract the sync pulses from the synchronisation task in all probes, and then perform the synchronisation with the nidq
EphysRegisterRaw	Creates the probe insertions and uploads the probe descriptions file, also compresses the nidq files and uploads
EphysSyncPulses
EphysSyncRegisterRaw	Task to rename, compress and register raw daq data with .bin format collected using NIDAQ
RawEphysQC
SpikeSorting	Pykilosort 2.5 pipeline

class EphysRegisterRaw(session_path, **kwargs)

Bases: DynamicTask

Creates the probe insertions and uploads the probe descriptions file, also compresses the nidq files and uploads

priority = 100

job_size = 'small'

property signature

The signature of the task specifies inputs and outputs for the given task. For some tasks it is dynamic and calculated. The legacy code specifies those as tuples. The preferred way is to use the ExpectedDataset input and output constructors.

I = ExpectedDataset.input O = ExpectedDataset.output signature = {

‘input_files’: [

I(name=’extract.me.npy’, collection=’raw_data’, required=True, register=False, unique=False),

], ‘output_files’: [

O(name=’look.atme.npy’, collection=’shiny_data’, required=True, register=True, unique=False)

]} is equivalent to: signature = {

‘input_files’: [(‘extract.me.npy’, ‘raw_data’, True, True)], ‘output_files’: [(‘look.atme.npy’, ‘shiny_data’, True)], }

Returns:

class EphysSyncRegisterRaw(session_path, **kwargs)

Bases: DynamicTask

Task to rename, compress and register raw daq data with .bin format collected using NIDAQ

priority = 90

cpu = 2

io_charge = 30

job_size = 'small'

property signature

The signature of the task specifies inputs and outputs for the given task. For some tasks it is dynamic and calculated. The legacy code specifies those as tuples. The preferred way is to use the ExpectedDataset input and output constructors.

I = ExpectedDataset.input O = ExpectedDataset.output signature = {

‘input_files’: [

I(name=’extract.me.npy’, collection=’raw_data’, required=True, register=False, unique=False),

], ‘output_files’: [

O(name=’look.atme.npy’, collection=’shiny_data’, required=True, register=True, unique=False)

]} is equivalent to: signature = {

‘input_files’: [(‘extract.me.npy’, ‘raw_data’, True, True)], ‘output_files’: [(‘look.atme.npy’, ‘shiny_data’, True)], }

Returns:

class EphysCompressNP1(session_path, **kwargs)

Bases: EphysTask

priority = 90

cpu = 2

io_charge = 100

job_size = 'small'

property signature

The signature of the task specifies inputs and outputs for the given task. For some tasks it is dynamic and calculated. The legacy code specifies those as tuples. The preferred way is to use the ExpectedDataset input and output constructors.

I = ExpectedDataset.input O = ExpectedDataset.output signature = {

‘input_files’: [

I(name=’extract.me.npy’, collection=’raw_data’, required=True, register=False, unique=False),

], ‘output_files’: [

O(name=’look.atme.npy’, collection=’shiny_data’, required=True, register=True, unique=False)

]} is equivalent to: signature = {

‘input_files’: [(‘extract.me.npy’, ‘raw_data’, True, True)], ‘output_files’: [(‘look.atme.npy’, ‘shiny_data’, True)], }

Returns:

class EphysCompressNP21(session_path, **kwargs)

Bases: EphysTask

priority = 90

cpu = 2

io_charge = 100

job_size = 'large'

property signature

The signature of the task specifies inputs and outputs for the given task. For some tasks it is dynamic and calculated. The legacy code specifies those as tuples. The preferred way is to use the ExpectedDataset input and output constructors.

I = ExpectedDataset.input O = ExpectedDataset.output signature = {

‘input_files’: [

I(name=’extract.me.npy’, collection=’raw_data’, required=True, register=False, unique=False),

], ‘output_files’: [

O(name=’look.atme.npy’, collection=’shiny_data’, required=True, register=True, unique=False)

]} is equivalent to: signature = {

‘input_files’: [(‘extract.me.npy’, ‘raw_data’, True, True)], ‘output_files’: [(‘look.atme.npy’, ‘shiny_data’, True)], }

Returns:

class EphysCompressNP24(session_path, *args, pname=None, device_collection='raw_ephys_data', nshanks=None, **kwargs)

Bases: EphysTask

Compresses NP2.4 data by splitting into N binary files, corresponding to N shanks

Parameters:

• pname – a probe name string

• device_collection – the collection containing the probes (usually ‘raw_ephys_data’)

• nshanks – number of shanks used (usually 4 but it may be less depending on electrode map), optional

priority = 90

cpu = 2

io_charge = 100

job_size = 'large'

property signature

The signature of the task specifies inputs and outputs for the given task. For some tasks it is dynamic and calculated. The legacy code specifies those as tuples. The preferred way is to use the ExpectedDataset input and output constructors.

I = ExpectedDataset.input O = ExpectedDataset.output signature = {

‘input_files’: [

I(name=’extract.me.npy’, collection=’raw_data’, required=True, register=False, unique=False),

], ‘output_files’: [

O(name=’look.atme.npy’, collection=’shiny_data’, required=True, register=True, unique=False)

]} is equivalent to: signature = {

‘input_files’: [(‘extract.me.npy’, ‘raw_data’, True, True)], ‘output_files’: [(‘look.atme.npy’, ‘shiny_data’, True)], }

Returns:

class EphysSyncPulses(session_path, **kwargs)

Bases: SyncPulses

priority = 90

cpu = 2

io_charge = 30

job_size = 'small'

property signature

The signature of the task specifies inputs and outputs for the given task. For some tasks it is dynamic and calculated. The legacy code specifies those as tuples. The preferred way is to use the ExpectedDataset input and output constructors.

I = ExpectedDataset.input O = ExpectedDataset.output signature = {

‘input_files’: [

I(name=’extract.me.npy’, collection=’raw_data’, required=True, register=False, unique=False),

], ‘output_files’: [

O(name=’look.atme.npy’, collection=’shiny_data’, required=True, register=True, unique=False)

]} is equivalent to: signature = {

‘input_files’: [(‘extract.me.npy’, ‘raw_data’, True, True)], ‘output_files’: [(‘look.atme.npy’, ‘shiny_data’, True)], }

Returns:

class EphysPulses(*args, **kwargs)

Bases: EphysTask

Extract Pulses from raw electrophysiology data into numpy arrays Perform the probes synchronisation with nidq (3B) or main probe (3A) First the job extract the sync pulses from the synchronisation task in all probes, and then perform the

synchronisation with the nidq

Parameters:

• pname – a list of probes names or a single probe name string

• device_collection – the collection containing the probes (usually ‘raw_ephys_data’)

• sync_collection – the collection containing the synchronisation device - nidq (usually ‘raw_ephys_data’)

priority = 90

cpu = 2

io_charge = 30

job_size = 'small'

property signature

The signature of the task specifies inputs and outputs for the given task. For some tasks it is dynamic and calculated. The legacy code specifies those as tuples. The preferred way is to use the ExpectedDataset input and output constructors.

I = ExpectedDataset.input O = ExpectedDataset.output signature = {

‘input_files’: [

I(name=’extract.me.npy’, collection=’raw_data’, required=True, register=False, unique=False),

], ‘output_files’: [

O(name=’look.atme.npy’, collection=’shiny_data’, required=True, register=True, unique=False)

]} is equivalent to: signature = {

‘input_files’: [(‘extract.me.npy’, ‘raw_data’, True, True)], ‘output_files’: [(‘look.atme.npy’, ‘shiny_data’, True)], }

Returns:

class RawEphysQC(session_path, **kwargs)

Bases: EphysTask

cpu = 2

io_charge = 30

priority = 10

job_size = 'small'

property signature

The signature of the task specifies inputs and outputs for the given task. For some tasks it is dynamic and calculated. The legacy code specifies those as tuples. The preferred way is to use the ExpectedDataset input and output constructors.

I = ExpectedDataset.input O = ExpectedDataset.output signature = {

‘input_files’: [

I(name=’extract.me.npy’, collection=’raw_data’, required=True, register=False, unique=False),

], ‘output_files’: [

O(name=’look.atme.npy’, collection=’shiny_data’, required=True, register=True, unique=False)

]} is equivalent to: signature = {

‘input_files’: [(‘extract.me.npy’, ‘raw_data’, True, True)], ‘output_files’: [(‘look.atme.npy’, ‘shiny_data’, True)], }

Returns:

class CellQCMixin

Bases: object

This mixin class is used to compute the cell QC metrics and update the json field of the probe insertion The compute_cell_qc method is static and can be used independently.

static compute_cell_qc(folder_alf_probe)

Computes the cell QC given an extracted probe alf path

Parameters:

folder_alf_probe – folder

Returns:

class SpikeSorting(session_path, **kwargs)

Bases: EphysTask, CellQCMixin

Pykilosort 2.5 pipeline

gpu = 1

io_charge = 100

priority = 60

job_size = 'large'

force = True

env = 'iblsorter'

SHELL_SCRIPT = PosixPath('/home/runner/Documents/PYTHON/iblscripts/deploy/serverpc/iblsorter/sort_recording.sh')

SPIKE_SORTER_NAME = 'iblsorter'

SORTER_REPOSITORY = PosixPath('/home/runner/Documents/PYTHON/SPIKE_SORTING/ibl-sorter')

property signature

The signature of the task specifies inputs and outputs for the given task. For some tasks it is dynamic and calculated. The legacy code specifies those as tuples. The preferred way is to use the ExpectedDataset input and output constructors.

I = ExpectedDataset.input O = ExpectedDataset.output signature = {

‘input_files’: [

I(name=’extract.me.npy’, collection=’raw_data’, required=True, register=False, unique=False),

], ‘output_files’: [

O(name=’look.atme.npy’, collection=’shiny_data’, required=True, register=True, unique=False)

]} is equivalent to: signature = {

‘input_files’: [(‘extract.me.npy’, ‘raw_data’, True, True)], ‘output_files’: [(‘look.atme.npy’, ‘shiny_data’, True)], }

Returns:

property scratch_folder_run

Constructs a path to a temporary folder for the spike sorting output and scratch files This is usually on a high performance drive, and we should factor around 2.5 times the uncompressed raw recording size For a scratch drive at /mnt/h0 we would have the following temp dir: /mnt/h0/iblsorter_1.8.0_CSHL071_2020-10-04_001_probe01/