one.converters

A module for inter-converting experiment identifiers.

There are multiple ways to uniquely identify an experiment:
  • eid (str) : An experiment UUID as a string

  • np (int64) : An experiment UUID encoded as 2 int64s

  • path (Path) : A pathlib ALF path of the form <lab>/Subjects/<subject>/<date>/<number>

  • ref (str) : An experiment reference string of the form yyyy-mm-dd_n_subject

  • url (str) : A remote http session path of the form <lab>/Subjects/<subject>/<date>/<number>

Functions

one_path_from_dataset

Returns local one file path from a dset record or a list of dsets records from REST.

parse_values

Convert str values in reference dict to appropriate type.

path_from_dataset

Returns the local file path from a dset record from a REST query.

path_from_filerecord

Returns a data file Path constructed from an Alyx file record.

recurse

Decorator to call decorated function recursively if first arg is non-string iterable.

session_record2path

Convert a session record into a path.

Classes

ConversionMixin

A mixin providing methods to interconvert experiment identifiers.

recurse(func)[source]

Decorator to call decorated function recursively if first arg is non-string iterable.

Allows decorated methods to accept both single values, and lists/tuples of values. When given the latter, a list is returned. This decorator is intended to work on class methods, therefore the first arg is assumed to be the object. Maps and pandas objects are not iterated over.

Parameters:

func (function) – A method to decorate.

Returns:

The decorated method.

Return type:

function

parse_values(func)[source]

Convert str values in reference dict to appropriate type.

Example

>>> parse_values(lambda x: x)({'date': '2020-01-01', 'sequence': '001'}, parse=True)
{'date': datetime.date(2020, 1, 1), 'sequence': 1}
class ConversionMixin[source]

Bases: object

A mixin providing methods to interconvert experiment identifiers.

to_eid(id: str | Path | UUID | dict | Sequence[str | Path | UUID | dict] = None, cache_dir: str | Path | None = None) str | Sequence[str][source]

Given any kind of experiment identifier, return a corresponding eid string.

NB: Currently does not support integer IDs.

Parameters:
  • id (str, pathlib.Path, UUID, dict, tuple, list) – An experiment identifier

  • cache_dir (pathlib.Path, str) – An optional cache directory path for intermittent conversion to path

Returns:

An experiment ID string or None if session not in cache

Return type:

str, None

Raises:

ValueError – Input ID invalid

eid2path(eid: str) Path | Sequence[Path] | None[source]

From an experiment id or a list of experiment ids, gets the local cache path.

Parameters:

eid (str, uuid.UUID) – Experiment ID (UUID) or list of UUIDs

Returns:

A session path

Return type:

pathlib.Path

path2eid(path_obj)[source]

From a local path, gets the experiment id.

Parameters:

path_obj (pathlib.Path, str) – Local path or list of local paths

Returns:

Experiment ID (eid) string or list of eids

Return type:

eid, list

path2record(path) Series[source]

Convert a file or session path to a dataset or session cache record.

NB: Assumes <lab>/Subjects/<subject>/<date>/<number> pattern

Parameters:

path (str, pathlib.Path) – Local path or HTTP URL

Returns:

A cache file record

Return type:

pandas.Series

path2url(filepath)[source]

Given a local file path, constructs the URL of the remote file.

Parameters:

filepath (str, pathlib.Path) – A local file path

Returns:

A remote URL string

Return type:

str

record2url(record)[source]

Convert a session or dataset record to a remote URL.

NB: Requires online instance

Parameters:

record (pd.Series, pd.DataFrame) – A datasets or sessions cache record. If DataFrame, iterate over and returns list.

Returns:

A dataset URL or list if input is DataFrame

Return type:

str, list

record2path(dataset) Path | None[source]

Given a set of dataset records, checks the corresponding exists flag in the cache correctly reflects the files system.

Parameters:

dataset (pd.DataFrame, pd.Series) – A datasets dataframe slice

Returns:

File path for the record

Return type:

pathlib.Path

eid2ref(eid: str | Iterable, as_dict=True, parse=True) str | Mapping | List[source]

Get human-readable session ref from path.

Parameters:
  • eid (str, uuid.UUID) – The experiment uuid to find reference for.

  • as_dict (bool) – If false a string is returned in the form ‘subject_sequence_yyyy-mm-dd’.

  • parse (bool) – If true, the reference date and sequence are parsed from strings to their respective data types.

Returns:

One or more objects with keys (‘subject’, ‘date’, ‘sequence’), or strings with the form yyyy-mm-dd_n_subject.

Return type:

dict, str, list

Examples

>>> eid = '4e0b3320-47b7-416e-b842-c34dc9004cf8'
>>> one.eid2ref(eid)
{'subject': 'flowers', 'date': datetime.date(2018, 7, 13), 'sequence': 1}
>>> one.eid2ref(eid, parse=False)
{'subject': 'flowers', 'date': '2018-07-13', 'sequence': '001'}
>>> one.eid2ref(eid, as_dict=False)
'2018-07-13_1_flowers'
>>> one.eid2ref(eid, as_dict=False, parse=False)
'2018-07-13_001_flowers'
>>> one.eid2ref([eid, '7dc3c44b-225f-4083-be3d-07b8562885f4'])
[{'subject': 'flowers', 'date': datetime.date(2018, 7, 13), 'sequence': 1},
 {'subject': 'KS005', 'date': datetime.date(2019, 4, 11), 'sequence': 1}]
ref2eid(ref: Mapping | str | Iterable) str | List[source]

Returns experiment uuid, given one or more experiment references.

Parameters:

ref (str, dict, list) – One or more objects with keys (‘subject’, ‘date’, ‘sequence’), or strings with the form yyyy-mm-dd_n_subject.

Returns:

One or more experiment uuid strings.

Return type:

str, list

Examples

>>> base = 'https://test.alyx.internationalbrainlab.org'
>>> one = ONE(username='test_user', password='TapetesBloc18', base_url=base)
Connected to...
>>> ref = {'date': datetime(2018, 7, 13).date(), 'sequence': 1, 'subject': 'flowers'}
>>> one.ref2eid(ref)
'4e0b3320-47b7-416e-b842-c34dc9004cf8'
>>> one.ref2eid(['2018-07-13_1_flowers', '2019-04-11_1_KS005'])
['4e0b3320-47b7-416e-b842-c34dc9004cf8',
 '7dc3c44b-225f-4083-be3d-07b8562885f4']
ref2path(ref)[source]

Convert one or more experiment references to session path(s).

Parameters:

ref (str, dict, list) – One or more objects with keys (‘subject’, ‘date’, ‘sequence’), or strings with the form yyyy-mm-dd_n_subject.

Returns:

Path object(s) for the experiment session(s).

Return type:

pathlib.Path

Examples

>>> base = 'https://test.alyx.internationalbrainlab.org'
>>> one = ONE(username='test_user', password='TapetesBloc18', base_url=base)
Connected to...
>>> ref = {'subject': 'flowers', 'date': datetime(2018, 7, 13).date(), 'sequence': 1}
>>> one.ref2path(ref)
WindowsPath('E:/FlatIron/zadorlab/Subjects/flowers/2018-07-13/001')
>>> one.ref2path(['2018-07-13_1_flowers', '2019-04-11_1_KS005'])
[WindowsPath('E:/FlatIron/zadorlab/Subjects/flowers/2018-07-13/001'),
 WindowsPath('E:/FlatIron/cortexlab/Subjects/KS005/2019-04-11/001')]
static path2ref(path_str: str | Path | Iterable, as_dict=True) Bunch | List[source]

Returns a human-readable experiment reference, given a session path.

The path need not exist.

Parameters:
  • path_str (str) – A path to a given session.

  • as_dict (bool) – If True a Bunch is returned, otherwise a string.

Returns:

One or more objects with keys (‘subject’, ‘date’, ‘sequence’).

Return type:

dict, str, list

Examples

>>> path_str = Path('E:/FlatIron/Subjects/zadorlab/flowers/2018-07-13/001')
>>> path2ref(path_str)
{'subject': 'flowers', 'date': datetime.date(2018, 7, 13), 'sequence': 1}
>>> path2ref(path_str, parse=False)
{'subject': 'flowers', 'date': '2018-07-13', 'sequence': '001'}
>>> path_str2 = Path('E:/FlatIron/Subjects/churchlandlab/CSHL046/2020-06-20/002')
>>> path2ref([path_str, path_str2])
[{'subject': 'flowers', 'date': datetime.date(2018, 7, 13), 'sequence': 1},
 {'subject': 'CSHL046', 'date': datetime.date(2020, 6, 20), 'sequence': 2}]
static is_exp_ref(ref: str | Mapping | Iterable) bool | List[bool][source]

Returns True is ref is a valid experiment reference.

Parameters:

ref (str, dict, list) – One or more objects with keys (‘subject’, ‘date’, ‘sequence’), or strings with the form yyyy-mm-dd_n_subject.

Returns:

True if ref is valid.

Return type:

bool, list of bool

Examples

>>> ref = {'date': datetime(2018, 7, 13).date(), 'sequence': 1, 'subject': 'flowers'}
>>> is_exp_ref(ref)
True
>>> is_exp_ref('2018-07-13_001_flowers')
True
>>> is_exp_ref('invalid_ref')
False
static ref2dict(ref: str | Mapping | Iterable) Bunch | List[source]

Returns a Bunch (dict-like) from a reference string (or list thereof).

Parameters:

ref (str, list) – One or more experiment reference strings.

Returns:

A Bunch in with keys (‘subject’, ‘sequence’, ‘date’).

Return type:

iblutil.util.Bunch

Examples

>>> ref2dict('2018-07-13_1_flowers')
{'date': datetime.date(2018, 7, 13), 'sequence': 1, 'subject': 'flowers'}
>>> ref2dict('2018-07-13_001_flowers', parse=False)
{'date': '2018-07-13', 'sequence': '001', 'subject': 'flowers'}
>>> ref2dict(['2018-07-13_1_flowers', '2020-01-23_002_ibl_witten_01'])
[{'date': datetime.date(2018, 7, 13), 'sequence': 1, 'subject': 'flowers'},
 {'date': datetime.date(2020, 1, 23), 'sequence': 2, 'subject': 'ibl_witten_01'}]
static dict2ref(ref_dict) str | List[source]

Convert an experiment reference dict to a string in the format yyyy-mm-dd_n_subject.

Parameters:

ref_dict (dict, Bunch, list, tuple) – A map with the keys (‘subject’, ‘date’, ‘sequence’).

Returns:

An experiment reference string, or list thereof.

Return type:

str, list

one_path_from_dataset(dset, one_cache)[source]

Returns local one file path from a dset record or a list of dsets records from REST. Unlike to_eid, this function does not require ONE, and the dataset may not exist.

Parameters:
  • dset (dict, list) – Dataset dictionary or list of dictionaries from Alyx rest endpoint.

  • one_cache (str, pathlib.Path, pathlib.PurePath) – The local ONE data cache directory.

Returns:

The local path for a given dataset.

Return type:

pathlib.Path

path_from_dataset(dset, root_path=PurePosixPath('/'), repository=None, uuid=False)[source]

Returns the local file path from a dset record from a REST query. Unlike to_eid, this function does not require ONE, and the dataset may not exist.

Parameters:
  • dset (dict, list) – Dataset dictionary or list of dictionaries from Alyx rest endpoint.

  • root_path (str, pathlib.Path, pathlib.PurePath) – The prefix path such as the ONE download directory or remote http server root.

  • repository (str, None) – Which data repository to use from the file_records list, defaults to first online repository.

  • uuid (bool) – If True, the file path will contain the dataset UUID.

Returns:

File path or list of paths.

Return type:

pathlib.Path, list

path_from_filerecord(fr, root_path=PurePosixPath('/'), uuid=None)[source]

Returns a data file Path constructed from an Alyx file record. The Path type returned depends on the type of root_path: If root_path is a string a Path object is returned, otherwise if the root_path is a PurePath, the same path type is returned.

Parameters:
  • fr (dict) – An Alyx file record dict.

  • root_path (str, pathlib.Path) – An optional root path.

  • uuid (str, uuid.UUID) – An optional dataset UUID to add to the file name.

Returns:

A filepath as a pathlib object.

Return type:

pathlib.Path

session_record2path(session, root_dir=None)[source]

Convert a session record into a path.

If a lab key is present, the path will be in the form root_dir/lab/Subjects/subject/yyyy-mm-dd/nnn, otherwise root_dir/subject/yyyy-mm-dd/nnn.

Parameters:
  • session (Mapping) – A session record with keys (‘subject’, ‘date’, ‘number’[, ‘lab’]).

  • root_dir (str, pathlib.Path, pathlib.PurePath) – A root directory to prepend.

Returns:

A constructed path of the session.

Return type:

pathlib.Path, Pathlib.PurePath

Examples

>>> session_record2path({'subject': 'ALK01', 'date': '2020-01-01', 'number': 1})
PurePosixPath('ALK01/2020-01-01/001')
>>> record = {'date': datetime.datetime.fromisoformat('2020-01-01').date(),
...           'number': '001', 'lab': 'foo', 'subject': 'ALK01'}
>>> session_record2path(record, Path('/home/user'))
Path('/home/user/foo/Subjects/ALK01/2020-01-01/001')