one.converters

A module for inter-converting experiment identifiers.

There are multiple ways to uniquely identify an experiment:

  • eid (str) : An experiment UUID as a string

  • np (int64) : An experiment UUID encoded as 2 int64s

  • path (Path) : A pathlib ALF path of the form <lab>/Subjects/<subject>/<date>/<number>

  • ref (str) : An experiment reference string of the form yyyy-mm-dd_n_subject

  • url (str) : A remote http session path of the form <lab>/Subjects/<subject>/<date>/<number>

Functions

datasets2records

Extract datasets DataFrame from one or more Alyx dataset records.

one_path_from_dataset

Returns local one file path from a dset record or a list of dsets records from REST.

parse_values

Convert str values in reference dict to appropriate type.

path_from_dataset

Returns the local file path from a dset record from a REST query.

path_from_filerecord

Returns a data file Path constructed from an Alyx file record.

recurse

Decorator to call decorated function recursively if first arg is non-string iterable.

ses2records

Extract session cache record and datasets cache from a remote session data record.

session_record2path

Convert a session record into a path.

Classes

ConversionMixin

A mixin providing methods to interconvert experiment identifiers.
recurse(func)[source]

Decorator to call decorated function recursively if first arg is non-string iterable.

Allows decorated methods to accept both single values, and lists/tuples of values. When given the latter, a list is returned. This decorator is intended to work on class methods, therefore the first arg is assumed to be the object. Maps and pandas objects are not iterated over.

Parameters:

func (function) – A method to decorate.

Returns:

The decorated method.

Return type:

function

parse_values(func)[source]

Convert str values in reference dict to appropriate type.

Example

>>> parse_values(lambda x: x)({'date': '2020-01-01', 'sequence': '001'}, parse=True)
{'date': datetime.date(2020, 1, 1), 'sequence': 1}
class ConversionMixin[source]

Bases: object

A mixin providing methods to interconvert experiment identifiers.

to_eid(id: str | Path | UUID | dict | Sequence[str | Path | UUID | dict] = None, cache_dir: str | Path | None = None) → str | Sequence[str][source]

Given any kind of experiment identifier, return a corresponding eid string.

NB: Currently does not support integer IDs.

Parameters:

  • id (str, pathlib.Path, UUID, dict, tuple, list) – An experiment identifier

  • cache_dir (pathlib.Path, str) – An optional cache directory path for intermittent conversion to path

Returns:

An experiment ID string or None if session not in cache

Return type:

str, None

Raises:

ValueError – Input ID invalid

eid2path(eid: str) → ALFPath | Sequence[ALFPath] | None[source]

From an experiment id or a list of experiment ids, gets the local cache path.

Parameters:

eid (str, uuid.UUID) – Experiment ID (UUID) or list of UUIDs.

Returns:

A session path.

Return type:

one.alf.path.ALFPath

path2eid(path_obj)[source]

From a local path, gets the experiment id.

Parameters:

path_obj (pathlib.Path, str) – Local path or list of local paths.

Returns:

Experiment ID (eid) string or list of eids.

Return type:

eid, list

path2record(path) → Series[source]

Convert a file or session path to a dataset or session cache record.

NB: Assumes <lab>/Subjects/<subject>/<date>/<number> pattern.

Parameters:

path (str, pathlib.Path) – Local path or HTTP URL.

Returns:

A cache file record.

Return type:

pandas.Series

path2url(filepath)[source]

Given a local file path, constructs the URL of the remote file.

Parameters:

filepath (str, pathlib.Path) – A local file path

Returns:

A remote URL string

Return type:

str

record2url(record)[source]

Convert a session or dataset record to a remote URL.

NB: Requires online instance

Parameters:

record (pd.Series, pd.DataFrame) – A datasets or sessions cache record. If DataFrame, iterate over and returns list.

Returns:

A dataset URL or list if input is DataFrame

Return type:

str, list

record2path(dataset) → ALFPath | None[source]

Given a set of dataset records, returns the corresponding paths.

Parameters:

dataset (pd.DataFrame, pd.Series) – A datasets dataframe slice.

Returns:

File path for the record.

Return type:

one.alf.path.ALFPath

eid2ref(eid: str | Iterable, as_dict=True, parse=True) → str | Mapping | List[source]

Get human-readable session ref from path.

Parameters:

  • eid (str, uuid.UUID) – The experiment uuid to find reference for.

  • as_dict (bool) – If false a string is returned in the form ‘subject_sequence_yyyy-mm-dd’.

  • parse (bool) – If true, the reference date and sequence are parsed from strings to their respective data types.

Returns:

One or more objects with keys (‘subject’, ‘date’, ‘sequence’), or strings with the form yyyy-mm-dd_n_subject.

Return type:

dict, str, list

Examples

>>> eid = '4e0b3320-47b7-416e-b842-c34dc9004cf8'
>>> one.eid2ref(eid)
{'subject': 'flowers', 'date': datetime.date(2018, 7, 13), 'sequence': 1}
>>> one.eid2ref(eid, parse=False)
{'subject': 'flowers', 'date': '2018-07-13', 'sequence': '001'}
>>> one.eid2ref(eid, as_dict=False)
'2018-07-13_1_flowers'
>>> one.eid2ref(eid, as_dict=False, parse=False)
'2018-07-13_001_flowers'
>>> one.eid2ref([eid, '7dc3c44b-225f-4083-be3d-07b8562885f4'])
[{'subject': 'flowers', 'date': datetime.date(2018, 7, 13), 'sequence': 1},
 {'subject': 'KS005', 'date': datetime.date(2019, 4, 11), 'sequence': 1}]
ref2eid(ref: Mapping | str | Iterable) → str | List[source]

Returns experiment uuid, given one or more experiment references.

Parameters:

ref (str, dict, list) – One or more objects with keys (‘subject’, ‘date’, ‘sequence’), or strings with the form yyyy-mm-dd_n_subject.

Returns:

One or more experiment uuid strings.

Return type:

str, list

Examples

>>> base = 'https://test.alyx.internationalbrainlab.org'
>>> one = ONE(username='test_user', password='TapetesBloc18', base_url=base)
Connected to...
>>> ref = {'date': datetime(2018, 7, 13).date(), 'sequence': 1, 'subject': 'flowers'}
>>> one.ref2eid(ref)
'4e0b3320-47b7-416e-b842-c34dc9004cf8'
>>> one.ref2eid(['2018-07-13_1_flowers', '2019-04-11_1_KS005'])
['4e0b3320-47b7-416e-b842-c34dc9004cf8',
 '7dc3c44b-225f-4083-be3d-07b8562885f4']
ref2path(ref)[source]

Convert one or more experiment references to session path(s).

Parameters:

ref (str, dict, list) – One or more objects with keys (‘subject’, ‘date’, ‘sequence’), or strings with the form yyyy-mm-dd_n_subject.

Returns:

Path object(s) for the experiment session(s).

Return type:

one.alf.path.ALFPath

Examples

>>> base = 'https://test.alyx.internationalbrainlab.org'
>>> one = ONE(username='test_user', password='TapetesBloc18', base_url=base)
Connected to...
>>> ref = {'subject': 'flowers', 'date': datetime(2018, 7, 13).date(), 'sequence': 1}
>>> one.ref2path(ref)
WindowsPath('E:/FlatIron/zadorlab/Subjects/flowers/2018-07-13/001')
>>> one.ref2path(['2018-07-13_1_flowers', '2019-04-11_1_KS005'])
[WindowsPath('E:/FlatIron/zadorlab/Subjects/flowers/2018-07-13/001'),
 WindowsPath('E:/FlatIron/cortexlab/Subjects/KS005/2019-04-11/001')]
static path2ref(path_str: str | Path | Iterable, as_dict=True) → Bunch | List[source]

Returns a human-readable experiment reference, given a session path.

The path need not exist.

Parameters:

  • path_str (str) – A path to a given session.

  • as_dict (bool) – If True a Bunch is returned, otherwise a string.

Returns:

One or more objects with keys (‘subject’, ‘date’, ‘sequence’).

Return type:

dict, str, list

Examples

>>> path_str = Path('E:/FlatIron/Subjects/zadorlab/flowers/2018-07-13/001')
>>> path2ref(path_str)
{'subject': 'flowers', 'date': datetime.date(2018, 7, 13), 'sequence': 1}
>>> path2ref(path_str, parse=False)
{'subject': 'flowers', 'date': '2018-07-13', 'sequence': '001'}
>>> path_str2 = Path('E:/FlatIron/Subjects/churchlandlab/CSHL046/2020-06-20/002')
>>> path2ref([path_str, path_str2])
[{'subject': 'flowers', 'date': datetime.date(2018, 7, 13), 'sequence': 1},
 {'subject': 'CSHL046', 'date': datetime.date(2020, 6, 20), 'sequence': 2}]
static is_exp_ref(ref: str | Mapping | Iterable) → bool | List[bool][source]

Returns True is ref is a valid experiment reference.

Parameters:

ref (str, dict, list) – One or more objects with keys (‘subject’, ‘date’, ‘sequence’), or strings with the form yyyy-mm-dd_n_subject.

Returns:

True if ref is valid.

Return type:

bool, list of bool

Examples

>>> ref = {'date': datetime(2018, 7, 13).date(), 'sequence': 1, 'subject': 'flowers'}
>>> is_exp_ref(ref)
True
>>> is_exp_ref('2018-07-13_001_flowers')
True
>>> is_exp_ref('invalid_ref')
False
static ref2dict(ref: str | Mapping | Iterable) → Bunch | List[source]

Returns a Bunch (dict-like) from a reference string (or list thereof).

Parameters:

ref (str, list) – One or more experiment reference strings.

Returns:

A Bunch in with keys (‘subject’, ‘sequence’, ‘date’).

Return type:

iblutil.util.Bunch

Examples

>>> ref2dict('2018-07-13_1_flowers')
{'date': datetime.date(2018, 7, 13), 'sequence': 1, 'subject': 'flowers'}
>>> ref2dict('2018-07-13_001_flowers', parse=False)
{'date': '2018-07-13', 'sequence': '001', 'subject': 'flowers'}
>>> ref2dict(['2018-07-13_1_flowers', '2020-01-23_002_ibl_witten_01'])
[{'date': datetime.date(2018, 7, 13), 'sequence': 1, 'subject': 'flowers'},
 {'date': datetime.date(2020, 1, 23), 'sequence': 2, 'subject': 'ibl_witten_01'}]
static dict2ref(ref_dict) → str | List[source]

Convert an experiment reference dict to a string in the format yyyy-mm-dd_n_subject.

Parameters:

ref_dict (dict, Bunch, list, tuple) – A map with the keys (‘subject’, ‘date’, ‘sequence’).

Returns:

An experiment reference string, or list thereof.

Return type:

str, list

one_path_from_dataset(dset, one_cache)[source]

Returns local one file path from a dset record or a list of dsets records from REST. Unlike to_eid, this function does not require ONE, and the dataset may not exist.

Parameters:

  • dset (dict, list) – Dataset dictionary or list of dictionaries from Alyx rest endpoint.

  • one_cache (str, pathlib.Path, pathlib.PurePath) – The local ONE data cache directory.

Returns:

The local path for a given dataset.

Return type:

one.alf.path.ALFPath

path_from_dataset(dset, root_path=PurePosixALFPath('/'), repository=None, uuid=False)[source]

Returns the local file path from a dset record from a REST query. Unlike to_eid, this function does not require ONE, and the dataset may not exist.

Parameters:

  • dset (dict, list) – Dataset dictionary or list of dictionaries from Alyx rest endpoint.

  • root_path (str, pathlib.Path, pathlib.PurePath) – The prefix path such as the ONE download directory or remote http server root.

  • repository (str, None) – Which data repository to use from the file_records list, defaults to first online repository.

  • uuid (bool) – If True, the file path will contain the dataset UUID.

Returns:

File path or list of paths.

Return type:

one.alf.path.ALFPath, list

path_from_filerecord(fr, root_path=PurePosixALFPath('/'), uuid=None)[source]

Returns a data file Path constructed from an Alyx file record.

The Path type returned depends on the type of root_path: If root_path is a string an ALFPath object is returned, otherwise if the root_path is a PurePath, a PureALFPath is returned.

Parameters:

  • fr (dict) – An Alyx file record dict.

  • root_path (str, pathlib.Path) – An optional root path.

  • uuid (str, uuid.UUID) – An optional dataset UUID to add to the file name.

Returns:

A filepath as a pathlib object.

Return type:

one.alf.path.ALFPath

session_record2path(session, root_dir=None)[source]

Convert a session record into a path.

If a lab key is present, the path will be in the form root_dir/lab/Subjects/subject/yyyy-mm-dd/nnn, otherwise root_dir/subject/yyyy-mm-dd/nnn.

Parameters:

  • session (Mapping) – A session record with keys (‘subject’, ‘date’, ‘number’[, ‘lab’]).

  • root_dir (str, pathlib.Path, pathlib.PurePath) – A root directory to prepend.

Returns:

A constructed path of the session.

Return type:

one.alf.path.ALFPath, one.alf.path.PureALFPath

Examples

>>> session_record2path({'subject': 'ALK01', 'date': '2020-01-01', 'number': 1})
PurePosixPath('ALK01/2020-01-01/001')

>>> record = {'date': datetime.datetime.fromisoformat('2020-01-01').date(),
...           'number': '001', 'lab': 'foo', 'subject': 'ALK01'}
>>> session_record2path(record, Path('/home/user'))
Path('/home/user/foo/Subjects/ALK01/2020-01-01/001')
ses2records(ses: dict)[source]

Extract session cache record and datasets cache from a remote session data record.

Parameters:

ses (dict) – Session dictionary from Alyx REST endpoint.

Returns:

  • pd.Series – Session record.

  • pd.DataFrame – Datasets frame.

datasets2records(datasets, additional=None) → DataFrame[source]

Extract datasets DataFrame from one or more Alyx dataset records.

Parameters:

  • datasets (dict, list) – One or more records from the Alyx ‘datasets’ endpoint.

  • additional (list of str) – A set of optional fields to extract from dataset records.

Returns:

Datasets frame.

Return type:

pd.DataFrame

Examples

>>> datasets = ONE().alyx.rest('datasets', 'list', subject='foobar')
>>> df = datasets2records(datasets)