one.converters

A module for inter-converting experiment identifiers.

There are multiple ways to uniquely identify an experiment:
  • eid (str) : An experiment UUID as a string

  • np (int64) : An experiment UUID encoded as 2 int64s

  • path (Path) : A pathlib ALF path of the form <lab>/Subjects/<subject>/<date>/<number>

  • ref (str) : An experiment reference string of the form yyyy-mm-dd_n_subject

  • url (str) : An remote http session path of the form <lab>/Subjects/<subject>/<date>/<number>

Functions

one_path_from_dataset

Returns local one file path from a dset record or a list of dsets records from REST.

parse_values

Convert str values in reference dict to appropriate type

path_from_dataset

Returns the local file path from a dset record from a REST query Unlike to_eid, this function does not require ONE, and the dataset may not exist.

path_from_filerecord

Returns a data file Path constructed from an Alyx file record.

recurse

Classes

ConversionMixin

recurse(func)[source]
parse_values(func)[source]

Convert str values in reference dict to appropriate type

class ConversionMixin[source]

Bases: object

to_eid(id: Union[str, pathlib.Path, uuid.UUID, dict, Sequence[Union[str, pathlib.Path, uuid.UUID, dict]]] = None, cache_dir: Optional[Union[str, pathlib.Path]] = None) → Union[str, Sequence[str]][source]
eid2path(eid: str) → Optional[Union[pathlib.Path, Sequence[pathlib.Path]]][source]

From an experiment id or a list of experiment ids, gets the local cache path

Parameters

eid (str, uuid.UUID) – Experiment ID (UUID) or list of UUIDs

Returns

Return type

A pathlib.Path of the session

path2eid(path_obj)[source]

From a local path, gets the experiment id

Parameters

path_obj (pathlib.Path, str) – Local path or list of local paths

Returns

Return type

Experiment ID (eid) string or list of eids

path2record(filepath)[source]

TODO Return Series instead of DataFrame NB: Assumes <lab>/Subjects/<subject>/<date>/<number> pattern

Parameters

filepath (str, pathlib.Path) – File path or HTTP URL

Returns

Return type

pandas.Series corresponding to the file

path2url(filepath)[source]

Given a local file path, constructs the URL of the remote file.

Parameters

filepath (str, pathlib.Path) – A local file path

Returns

Return type

A remote URL string

record2url(dataset)[source]
record2path(dataset) → Optional[pathlib.Path][source]

Given a set of dataset records, checks the corresponding exists flag in the cache correctly reflects the files system

Parameters

dataset (pd.DataFrame, pd.Series) – A datasets dataframe slice

Returns

Return type

File path for the record

eid2ref(eid: Union[str, Iterable], as_dict=True, parse=True) → Union[str, Mapping, List][source]

Get human-readable session ref from path

Parameters
  • eid (str, uuid.UUID) – The experiment uuid to find reference for

  • as_dict (bool) – If false a string is returned in the form ‘subject_sequence_yyyy-mm-dd’

  • parse (bool) – If true, the reference date and sequence are parsed from strings to their respective data types

Returns

  • One or more objects with keys (‘subject’, ‘date’, ‘sequence’), or strings with the

  • form yyyy-mm-dd_n_subject

Examples

>>> eid = '4e0b3320-47b7-416e-b842-c34dc9004cf8'
>>> one.eid2ref(eid)
{'subject': 'flowers', 'date': datetime.date(2018, 7, 13), 'sequence': 1}
>>> one.eid2ref(eid, parse=False)
{'subject': 'flowers', 'date': '2018-07-13', 'sequence': '001'}
>>> one.eid2ref(eid, as_dict=False)
'2018-07-13_1_flowers'
>>> one.eid2ref(eid, as_dict=False, parse=False)
'2018-07-13_001_flowers'
>>> one.eid2ref([eid, '7dc3c44b-225f-4083-be3d-07b8562885f4'])
[{'subject': 'flowers', 'date': datetime.date(2018, 7, 13), 'sequence': 1},
 {'subject': 'KS005', 'date': datetime.date(2019, 4, 11), 'sequence': 1}]
ref2eid(ref: Union[Mapping, str, Iterable]) → Union[str, List][source]

Returns experiment uuid, given one or more experiment references

Parameters

ref (str, dict, list) – One or more objects with keys (‘subject’, ‘date’, ‘sequence’), or strings with the form yyyy-mm-dd_n_subject

Returns

Return type

One or more experiment uuid strings

Examples

>>> base = 'https://test.alyx.internationalbrainlab.org'
>>> one = ONE(username='test_user', password='TapetesBloc18', base_url=base)
Connected to...
>>> ref = {'date': datetime(2018, 7, 13).date(), 'sequence': 1, 'subject': 'flowers'}
>>> one.ref2eid(ref)
'4e0b3320-47b7-416e-b842-c34dc9004cf8'
>>> one.ref2eid(['2018-07-13_1_flowers', '2019-04-11_1_KS005'])
['4e0b3320-47b7-416e-b842-c34dc9004cf8',
 '7dc3c44b-225f-4083-be3d-07b8562885f4']
ref2path(ref)[source]

Convert one or more experiment references to session path(s)

Parameters

ref (str, dict, list) – One or more objects with keys (‘subject’, ‘date’, ‘sequence’), or strings with the form yyyy-mm-dd_n_subject

Returns

Return type

Path object(s) for the experiment session(s)

Examples

>>> base = 'https://test.alyx.internationalbrainlab.org'
>>> one = ONE(username='test_user', password='TapetesBloc18', base_url=base)
Connected to...
>>> ref = {'subject': 'flowers', 'date': datetime(2018, 7, 13).date(), 'sequence': 1}
>>> one.ref2path(ref)
WindowsPath('E:/FlatIron/zadorlab/Subjects/flowers/2018-07-13/001')
>>> one.ref2path(['2018-07-13_1_flowers', '2019-04-11_1_KS005'])
[WindowsPath('E:/FlatIron/zadorlab/Subjects/flowers/2018-07-13/001'),
 WindowsPath('E:/FlatIron/cortexlab/Subjects/KS005/2019-04-11/001')]
static path2ref(*args, **kwargs) → Union[iblutil.util.Bunch, List][source]

Returns a human readable experiment reference, given a session path. The path need not exist.

Parameters
  • path_str (str) – A path to a given session

  • as_dict (bool) – If True a Bunch is returned, otherwise a string

Returns

Return type

One or more objects with keys (‘subject’, ‘date’, ‘sequence’)

Examples

>>> path_str = Path('E:/FlatIron/Subjects/zadorlab/flowers/2018-07-13/001')
>>> path2ref(path_str)
{'subject': 'flowers', 'date': datetime.date(2018, 7, 13), 'sequence': 1}
>>> path2ref(path_str, parse=False)
{'subject': 'flowers', 'date': '2018-07-13', 'sequence': '001'}
>>> path_str2 = Path('E:/FlatIron/Subjects/churchlandlab/CSHL046/2020-06-20/002')
>>> path2ref([path_str, path_str2])
[{'subject': 'flowers', 'date': datetime.date(2018, 7, 13), 'sequence': 1},
 {'subject': 'CSHL046', 'date': datetime.date(2020, 6, 20), 'sequence': 2}]
ref2dj(ref: Union[str, Mapping, Iterable])[source]

Return an ibl-pipeline sessions table, restricted by experiment reference(s)

Parameters

ref (str, list, dict) – One or more objects with keys (‘subject’, ‘date’, ‘sequence’), or strings with the form yyyy-mm-dd_n_subject

Returns

Return type

An acquisition.Session table

Examples

>>> ref2dj('2020-06-20_2_CSHL046').fetch1()
Connecting...
{'subject_uuid': UUID('dffc24bc-bd97-4c2a-bef3-3e9320dc3dd7'),
 'session_start_time': datetime.datetime(2020, 6, 20, 13, 31, 47),
 'session_number': 2,
 'session_date': datetime.date(2020, 6, 20),
 'subject_nickname': 'CSHL046'}
>>> len(ref2dj({'date':'2020-06-20', 'sequence':'002', 'subject':'CSHL046'}))
1
>>> len(ref2dj(['2020-06-20_2_CSHL046', '2019-11-01_1_ibl_witten_13']))
2
static is_exp_ref(ref: Union[str, Mapping, Iterable]) → Union[bool, List[bool]][source]

Returns True is ref is a valid experiment reference

Parameters

ref (str, dict, list) – One or more objects with keys (‘subject’, ‘date’, ‘sequence’), or strings with the form yyyy-mm-dd_n_subject

Returns

Return type

True if ref is valid

Examples

>>> ref = {'date': datetime(2018, 7, 13).date(), 'sequence': 1, 'subject': 'flowers'}
>>> is_exp_ref(ref)
True
>>> is_exp_ref('2018-07-13_001_flowers')
True
>>> is_exp_ref('invalid_ref')
False
path2pid(path)[source]

Returns a portion of the path that represents the session and probe label

static ref2dict(*args, **kwargs) → Union[iblutil.util.Bunch, List][source]

Returns a Bunch (dict-like) from a reference string (or list thereof)

Parameters

ref (str, list) – One or more experiment reference strings

Returns

Return type

A Bunch in with keys (‘subject’, ‘sequence’, ‘date’)

Examples

>>> ref2dict('2018-07-13_1_flowers')
{'date': datetime.date(2018, 7, 13), 'sequence': 1, 'subject': 'flowers'}
>>> ref2dict('2018-07-13_001_flowers', parse=False)
{'date': '2018-07-13', 'sequence': '001', 'subject': 'flowers'}
>>> ref2dict(['2018-07-13_1_flowers', '2020-01-23_002_ibl_witten_01'])
[{'date': datetime.date(2018, 7, 13), 'sequence': 1, 'subject': 'flowers'},
 {'date': datetime.date(2020, 1, 23), 'sequence': 2, 'subject': 'ibl_witten_01'}]
static dict2ref(ref_dict)[source]
one_path_from_dataset(dset, one_cache)[source]

Returns local one file path from a dset record or a list of dsets records from REST. Unlike to_eid, this function does not require ONE, and the dataset may not exist.

Parameters
  • dset (dict, list) – Dataset dictionary or list of dictionaries from Alyx rest endpoint

  • one_cache (str, pathlib.Path, pathlib.PurePath) – The local ONE data cache directory

Returns

Return type

The local path for a given dataset

path_from_dataset(dset, root_path=PurePosixPath('/'), repository=None, uuid=False)[source]

Returns the local file path from a dset record from a REST query Unlike to_eid, this function does not require ONE, and the dataset may not exist.

Parameters
  • dset (dict, list) – Dataset dictionary or list of dictionaries from Alyx rest endpoint

  • root_path (str, pathlib.Path, pathlib.PurePath) – The prefix path such as the ONE download directory or remote http server root

  • repository (str, None) – Which data repository to use from the file_records list, defaults to first online repository

  • uuid (bool) – If True, the file path will contain the dataset UUID

Returns

Return type

File path or list of paths

path_from_filerecord(fr, root_path=PurePosixPath('/'), uuid=None)[source]

Returns a data file Path constructed from an Alyx file record. The Path type returned depends on the type of root_path: If root_path is a string a Path object is returned, otherwise if the root_path is a PurePath, the same path type is returned.

Parameters
  • fr (dict) – An Alyx file record dict

  • root_path (str, pathlib.Path) – An optional root path

  • uuid (str, uuid.UUID) – An optional dataset UUID to add to the file name

Returns

Return type

A filepath as a pathlib object