one.converters

A module for inter-converting experiment identifiers.

There are multiple ways to uniquely identify an experiment:
  • eid (str) : An experiment UUID as a string

  • np (int64) : An experiment UUID encoded as 2 int64s

  • path (Path) : A pathlib ALF path of the form <lab>/Subjects/<subject>/<date>/<number>

  • ref (str) : An experiment reference string of the form yyyy-mm-dd_n_subject

  • url (str) : A remote http session path of the form <lab>/Subjects/<subject>/<date>/<number>

Functions

one_path_from_dataset

Returns local one file path from a dset record or a list of dsets records from REST.

parse_values

Convert str values in reference dict to appropriate type.

path_from_dataset

Returns the local file path from a dset record from a REST query.

path_from_filerecord

Returns a data file Path constructed from an Alyx file record.

recurse

Decorator to call decorated function recursively if first arg is non-string iterable.

session_record2path

Convert a session record into a path.

Classes

ConversionMixin

A mixin providing methods to interconvert experiment identifiers

recurse(func)[source]

Decorator to call decorated function recursively if first arg is non-string iterable.

Allows decorated methods to accept both single values, and lists/tuples of values. When given the latter, a list is returned. This decorator is intended to work on class methods, therefore the first arg is assumed to be the object. Maps and pandas objects are not iterated over.

Parameters

func (function) – A method to decorate

Returns

The decorated method

Return type

function

parse_values(func)[source]

Convert str values in reference dict to appropriate type.

Example

>>> parse_values(lambda x: x)({'date': '2020-01-01', 'sequence': '001'}, parse=True)
{'date': datetime.date(2020, 1, 1), 'sequence': 1}
class ConversionMixin[source]

Bases: object

A mixin providing methods to interconvert experiment identifiers

to_eid(id: Union[str, Path, UUID, dict, Sequence[Union[str, Path, UUID, dict]]] = None, cache_dir: Optional[Union[str, Path]] = None) Union[str, Sequence[str]][source]

Given any kind of experiment identifier, return a corresponding eid string.

NB: Currently does not support integer IDs.

Parameters
  • id (str, pathlib.Path, UUID, dict, tuple, list) – An experiment identifier

  • cache_dir (pathlib.Path, str) – An optional cache directory path for intermittent conversion to path

Returns

An experiment ID string or None if session not in cache

Return type

str, None

Raises

ValueError – Input ID invalid

eid2path(eid: str) Optional[Union[Path, Sequence[Path]]][source]

From an experiment id or a list of experiment ids, gets the local cache path

Parameters

eid (str, uuid.UUID) – Experiment ID (UUID) or list of UUIDs

Returns

A session path

Return type

pathlib.Path

path2eid(path_obj)[source]

From a local path, gets the experiment id

Parameters

path_obj (pathlib.Path, str) – Local path or list of local paths

Returns

Experiment ID (eid) string or list of eids

Return type

eid, list

path2record(path) Series[source]

Convert a file or session path to a dataset or session cache record.

NB: Assumes <lab>/Subjects/<subject>/<date>/<number> pattern

Parameters

path (str, pathlib.Path) – Local path or HTTP URL

Returns

A cache file record

Return type

pandas.Series

path2url(filepath)[source]

Given a local file path, constructs the URL of the remote file.

Parameters

filepath (str, pathlib.Path) – A local file path

Returns

A remote URL string

Return type

str

record2url(record)[source]

Convert a session or dataset record to a remote URL

NB: Requires online instance

Parameters

record (pd.Series, pd.DataFrame) – A datasets or sessions cache record. If DataFrame, iterate over and returns list.

Returns

A dataset URL or list if input is DataFrame

Return type

str, list

record2path(dataset) Optional[Path][source]

Given a set of dataset records, checks the corresponding exists flag in the cache correctly reflects the files system

Parameters

dataset (pd.DataFrame, pd.Series) – A datasets dataframe slice

Returns

File path for the record

Return type

pathlib.Path

eid2ref(eid: Union[str, Iterable], as_dict=True, parse=True) Union[str, Mapping, List][source]

Get human-readable session ref from path

Parameters
  • eid (str, uuid.UUID) – The experiment uuid to find reference for

  • as_dict (bool) – If false a string is returned in the form ‘subject_sequence_yyyy-mm-dd’

  • parse (bool) – If true, the reference date and sequence are parsed from strings to their respective data types

Returns

One or more objects with keys (‘subject’, ‘date’, ‘sequence’), or strings with the form yyyy-mm-dd_n_subject

Return type

dict, str, list

Examples

>>> eid = '4e0b3320-47b7-416e-b842-c34dc9004cf8'
>>> one.eid2ref(eid)
{'subject': 'flowers', 'date': datetime.date(2018, 7, 13), 'sequence': 1}
>>> one.eid2ref(eid, parse=False)
{'subject': 'flowers', 'date': '2018-07-13', 'sequence': '001'}
>>> one.eid2ref(eid, as_dict=False)
'2018-07-13_1_flowers'
>>> one.eid2ref(eid, as_dict=False, parse=False)
'2018-07-13_001_flowers'
>>> one.eid2ref([eid, '7dc3c44b-225f-4083-be3d-07b8562885f4'])
[{'subject': 'flowers', 'date': datetime.date(2018, 7, 13), 'sequence': 1},
 {'subject': 'KS005', 'date': datetime.date(2019, 4, 11), 'sequence': 1}]
ref2eid(ref: Union[Mapping, str, Iterable]) Union[str, List][source]

Returns experiment uuid, given one or more experiment references

Parameters

ref (str, dict, list) – One or more objects with keys (‘subject’, ‘date’, ‘sequence’), or strings with the form yyyy-mm-dd_n_subject

Returns

One or more experiment uuid strings

Return type

str, list

Examples

>>> base = 'https://test.alyx.internationalbrainlab.org'
>>> one = ONE(username='test_user', password='TapetesBloc18', base_url=base)
Connected to...
>>> ref = {'date': datetime(2018, 7, 13).date(), 'sequence': 1, 'subject': 'flowers'}
>>> one.ref2eid(ref)
'4e0b3320-47b7-416e-b842-c34dc9004cf8'
>>> one.ref2eid(['2018-07-13_1_flowers', '2019-04-11_1_KS005'])
['4e0b3320-47b7-416e-b842-c34dc9004cf8',
 '7dc3c44b-225f-4083-be3d-07b8562885f4']
ref2path(ref)[source]

Convert one or more experiment references to session path(s)

Parameters

ref (str, dict, list) – One or more objects with keys (‘subject’, ‘date’, ‘sequence’), or strings with the form yyyy-mm-dd_n_subject

Returns

Path object(s) for the experiment session(s)

Return type

pathlib.Path

Examples

>>> base = 'https://test.alyx.internationalbrainlab.org'
>>> one = ONE(username='test_user', password='TapetesBloc18', base_url=base)
Connected to...
>>> ref = {'subject': 'flowers', 'date': datetime(2018, 7, 13).date(), 'sequence': 1}
>>> one.ref2path(ref)
WindowsPath('E:/FlatIron/zadorlab/Subjects/flowers/2018-07-13/001')
>>> one.ref2path(['2018-07-13_1_flowers', '2019-04-11_1_KS005'])
[WindowsPath('E:/FlatIron/zadorlab/Subjects/flowers/2018-07-13/001'),
 WindowsPath('E:/FlatIron/cortexlab/Subjects/KS005/2019-04-11/001')]
static path2ref(path_str: Union[str, Path, Iterable], as_dict=True) Union[Bunch, List][source]

Returns a human readable experiment reference, given a session path. The path need not exist.

Parameters
  • path_str (str) – A path to a given session

  • as_dict (bool) – If True a Bunch is returned, otherwise a string

Returns

One or more objects with keys (‘subject’, ‘date’, ‘sequence’)

Return type

dict, str, list

Examples

>>> path_str = Path('E:/FlatIron/Subjects/zadorlab/flowers/2018-07-13/001')
>>> path2ref(path_str)
{'subject': 'flowers', 'date': datetime.date(2018, 7, 13), 'sequence': 1}
>>> path2ref(path_str, parse=False)
{'subject': 'flowers', 'date': '2018-07-13', 'sequence': '001'}
>>> path_str2 = Path('E:/FlatIron/Subjects/churchlandlab/CSHL046/2020-06-20/002')
>>> path2ref([path_str, path_str2])
[{'subject': 'flowers', 'date': datetime.date(2018, 7, 13), 'sequence': 1},
 {'subject': 'CSHL046', 'date': datetime.date(2020, 6, 20), 'sequence': 2}]
ref2dj(ref: Union[str, Mapping, Iterable])[source]

Return an ibl-pipeline sessions table, restricted by experiment reference(s)

Parameters

ref (str, list, dict) – One or more objects with keys (‘subject’, ‘date’, ‘sequence’), or strings with the form yyyy-mm-dd_n_subject

Returns

An acquisition.Session table corresponding to the ref

Return type

acquisition.Session

Examples

>>> ref2dj('2020-06-20_2_CSHL046').fetch1()
Connecting...
{'subject_uuid': UUID('dffc24bc-bd97-4c2a-bef3-3e9320dc3dd7'),
 'session_start_time': datetime.datetime(2020, 6, 20, 13, 31, 47),
 'session_number': 2,
 'session_date': datetime.date(2020, 6, 20),
 'subject_nickname': 'CSHL046'}
>>> len(ref2dj({'date':'2020-06-20', 'sequence':'002', 'subject':'CSHL046'}))
1
>>> len(ref2dj(['2020-06-20_2_CSHL046', '2019-11-01_1_ibl_witten_13']))
2
static is_exp_ref(ref: Union[str, Mapping, Iterable]) Union[bool, List[bool]][source]

Returns True is ref is a valid experiment reference

Parameters

ref (str, dict, list) – One or more objects with keys (‘subject’, ‘date’, ‘sequence’), or strings with the form yyyy-mm-dd_n_subject

Returns

True if ref is valid

Return type

bool

Examples

>>> ref = {'date': datetime(2018, 7, 13).date(), 'sequence': 1, 'subject': 'flowers'}
>>> is_exp_ref(ref)
True
>>> is_exp_ref('2018-07-13_001_flowers')
True
>>> is_exp_ref('invalid_ref')
False
static ref2dict(ref: Union[str, Mapping, Iterable]) Union[Bunch, List][source]

Returns a Bunch (dict-like) from a reference string (or list thereof)

Parameters

ref (str, list) – One or more experiment reference strings

Returns

A Bunch in with keys (‘subject’, ‘sequence’, ‘date’)

Return type

iblutil.util.Bunch

Examples

>>> ref2dict('2018-07-13_1_flowers')
{'date': datetime.date(2018, 7, 13), 'sequence': 1, 'subject': 'flowers'}
>>> ref2dict('2018-07-13_001_flowers', parse=False)
{'date': '2018-07-13', 'sequence': '001', 'subject': 'flowers'}
>>> ref2dict(['2018-07-13_1_flowers', '2020-01-23_002_ibl_witten_01'])
[{'date': datetime.date(2018, 7, 13), 'sequence': 1, 'subject': 'flowers'},
 {'date': datetime.date(2020, 1, 23), 'sequence': 2, 'subject': 'ibl_witten_01'}]
static dict2ref(ref_dict) Union[str, List][source]

Convert an experiment reference dict to a string in the format yyyy-mm-dd_n_subject.

Parameters

ref_dict (dict, Bunch, list, tuple) – A map with the keys (‘subject’, ‘date’, ‘sequence’)

Returns

An experiment reference string, or list thereof

Return type

str, list

one_path_from_dataset(dset, one_cache)[source]

Returns local one file path from a dset record or a list of dsets records from REST. Unlike to_eid, this function does not require ONE, and the dataset may not exist.

Parameters
  • dset (dict, list) – Dataset dictionary or list of dictionaries from Alyx rest endpoint

  • one_cache (str, pathlib.Path, pathlib.PurePath) – The local ONE data cache directory

Returns

The local path for a given dataset

Return type

pathlib.Path

path_from_dataset(dset, root_path=PurePosixPath('/'), repository=None, uuid=False)[source]

Returns the local file path from a dset record from a REST query. Unlike to_eid, this function does not require ONE, and the dataset may not exist.

Parameters
  • dset (dict, list) – Dataset dictionary or list of dictionaries from Alyx rest endpoint

  • root_path (str, pathlib.Path, pathlib.PurePath) – The prefix path such as the ONE download directory or remote http server root

  • repository (str, None) – Which data repository to use from the file_records list, defaults to first online repository

  • uuid (bool) – If True, the file path will contain the dataset UUID

Returns

File path or list of paths

Return type

pathlib.Path, list

path_from_filerecord(fr, root_path=PurePosixPath('/'), uuid=None)[source]

Returns a data file Path constructed from an Alyx file record. The Path type returned depends on the type of root_path: If root_path is a string a Path object is returned, otherwise if the root_path is a PurePath, the same path type is returned.

Parameters
  • fr (dict) – An Alyx file record dict

  • root_path (str, pathlib.Path) – An optional root path

  • uuid (str, uuid.UUID) – An optional dataset UUID to add to the file name

Returns

A filepath as a pathlib object

Return type

pathlib.Path

session_record2path(session, root_dir=None)[source]

Convert a session record into a path.

If a lab key is present, the path will be in the form root_dir/lab/Subjects/subject/yyyy-mm-dd/nnn, otherwise root_dir/subject/yyyy-mm-dd/nnn.

Parameters
  • session (Mapping) – A session record with keys (‘subject’, ‘date’, ‘number’[, ‘lab’]).

  • root_dir (str, pathlib.Path, pathlib.PurePath) – A root directory to prepend.

Returns

A constructed path of the session.

Return type

pathlib.Path, Pathlib.PurePath

Examples

>>> session_record2path({'subject': 'ALK01', 'date': '2020-01-01', 'number': 1})
PurePosixPath('ALK01/2020-01-01/001')
>>> record = {'date': datetime.datetime.fromisoformat('2020-01-01').date(),
...           'number': '001', 'lab': 'foo', 'subject': 'ALK01'}
>>> session_record2path(record, Path('/home/user'))
Path('/home/user/foo/Subjects/ALK01/2020-01-01/001')