one.converters
A module for inter-converting experiment identifiers.
- There are multiple ways to uniquely identify an experiment:
eid (str) : An experiment UUID as a string
np (int64) : An experiment UUID encoded as 2 int64s
path (Path) : A pathlib ALF path of the form <lab>/Subjects/<subject>/<date>/<number>
ref (str) : An experiment reference string of the form yyyy-mm-dd_n_subject
url (str) : A remote http session path of the form <lab>/Subjects/<subject>/<date>/<number>
Functions
Returns local one file path from a dset record or a list of dsets records from REST. |
|
Convert str values in reference dict to appropriate type. |
|
Returns the local file path from a dset record from a REST query. |
|
Returns a data file Path constructed from an Alyx file record. |
|
Decorator to call decorated function recursively if first arg is non-string iterable. |
|
Convert a session record into a path. |
Classes
A mixin providing methods to interconvert experiment identifiers. |
- recurse(func)[source]
Decorator to call decorated function recursively if first arg is non-string iterable.
Allows decorated methods to accept both single values, and lists/tuples of values. When given the latter, a list is returned. This decorator is intended to work on class methods, therefore the first arg is assumed to be the object. Maps and pandas objects are not iterated over.
- Parameters:
func (function) – A method to decorate.
- Returns:
The decorated method.
- Return type:
function
- parse_values(func)[source]
Convert str values in reference dict to appropriate type.
Example
>>> parse_values(lambda x: x)({'date': '2020-01-01', 'sequence': '001'}, parse=True) {'date': datetime.date(2020, 1, 1), 'sequence': 1}
- class ConversionMixin[source]
Bases:
object
A mixin providing methods to interconvert experiment identifiers.
- to_eid(id: str | Path | UUID | dict | Sequence[str | Path | UUID | dict] = None, cache_dir: str | Path | None = None) str | Sequence[str] [source]
Given any kind of experiment identifier, return a corresponding eid string.
NB: Currently does not support integer IDs.
- Parameters:
id (str, pathlib.Path, UUID, dict, tuple, list) – An experiment identifier
cache_dir (pathlib.Path, str) – An optional cache directory path for intermittent conversion to path
- Returns:
An experiment ID string or None if session not in cache
- Return type:
str, None
- Raises:
ValueError – Input ID invalid
- eid2path(eid: str) Path | Sequence[Path] | None [source]
From an experiment id or a list of experiment ids, gets the local cache path.
- Parameters:
eid (str, uuid.UUID) – Experiment ID (UUID) or list of UUIDs
- Returns:
A session path
- Return type:
pathlib.Path
- path2eid(path_obj)[source]
From a local path, gets the experiment id.
- Parameters:
path_obj (pathlib.Path, str) – Local path or list of local paths.
- Returns:
Experiment ID (eid) string or list of eids.
- Return type:
eid, list
- path2record(path) Series [source]
Convert a file or session path to a dataset or session cache record.
NB: Assumes <lab>/Subjects/<subject>/<date>/<number> pattern
- Parameters:
path (str, pathlib.Path) – Local path or HTTP URL
- Returns:
A cache file record
- Return type:
pandas.Series
- path2url(filepath)[source]
Given a local file path, constructs the URL of the remote file.
- Parameters:
filepath (str, pathlib.Path) – A local file path
- Returns:
A remote URL string
- Return type:
str
- record2url(record)[source]
Convert a session or dataset record to a remote URL.
NB: Requires online instance
- Parameters:
record (pd.Series, pd.DataFrame) – A datasets or sessions cache record. If DataFrame, iterate over and returns list.
- Returns:
A dataset URL or list if input is DataFrame
- Return type:
str, list
- record2path(dataset) Path | None [source]
Given a set of dataset records, returns the corresponding paths
- Parameters:
dataset (pd.DataFrame, pd.Series) – A datasets dataframe slice
- Returns:
File path for the record
- Return type:
pathlib.Path
- eid2ref(eid: str | Iterable, as_dict=True, parse=True) str | Mapping | List [source]
Get human-readable session ref from path.
- Parameters:
eid (str, uuid.UUID) – The experiment uuid to find reference for.
as_dict (bool) – If false a string is returned in the form ‘subject_sequence_yyyy-mm-dd’.
parse (bool) – If true, the reference date and sequence are parsed from strings to their respective data types.
- Returns:
One or more objects with keys (‘subject’, ‘date’, ‘sequence’), or strings with the form yyyy-mm-dd_n_subject.
- Return type:
dict, str, list
Examples
>>> eid = '4e0b3320-47b7-416e-b842-c34dc9004cf8' >>> one.eid2ref(eid) {'subject': 'flowers', 'date': datetime.date(2018, 7, 13), 'sequence': 1} >>> one.eid2ref(eid, parse=False) {'subject': 'flowers', 'date': '2018-07-13', 'sequence': '001'} >>> one.eid2ref(eid, as_dict=False) '2018-07-13_1_flowers' >>> one.eid2ref(eid, as_dict=False, parse=False) '2018-07-13_001_flowers' >>> one.eid2ref([eid, '7dc3c44b-225f-4083-be3d-07b8562885f4']) [{'subject': 'flowers', 'date': datetime.date(2018, 7, 13), 'sequence': 1}, {'subject': 'KS005', 'date': datetime.date(2019, 4, 11), 'sequence': 1}]
- ref2eid(ref: Mapping | str | Iterable) str | List [source]
Returns experiment uuid, given one or more experiment references.
- Parameters:
ref (str, dict, list) – One or more objects with keys (‘subject’, ‘date’, ‘sequence’), or strings with the form yyyy-mm-dd_n_subject.
- Returns:
One or more experiment uuid strings.
- Return type:
str, list
Examples
>>> base = 'https://test.alyx.internationalbrainlab.org' >>> one = ONE(username='test_user', password='TapetesBloc18', base_url=base) Connected to... >>> ref = {'date': datetime(2018, 7, 13).date(), 'sequence': 1, 'subject': 'flowers'} >>> one.ref2eid(ref) '4e0b3320-47b7-416e-b842-c34dc9004cf8' >>> one.ref2eid(['2018-07-13_1_flowers', '2019-04-11_1_KS005']) ['4e0b3320-47b7-416e-b842-c34dc9004cf8', '7dc3c44b-225f-4083-be3d-07b8562885f4']
- ref2path(ref)[source]
Convert one or more experiment references to session path(s).
- Parameters:
ref (str, dict, list) – One or more objects with keys (‘subject’, ‘date’, ‘sequence’), or strings with the form yyyy-mm-dd_n_subject.
- Returns:
Path object(s) for the experiment session(s).
- Return type:
pathlib.Path
Examples
>>> base = 'https://test.alyx.internationalbrainlab.org' >>> one = ONE(username='test_user', password='TapetesBloc18', base_url=base) Connected to... >>> ref = {'subject': 'flowers', 'date': datetime(2018, 7, 13).date(), 'sequence': 1} >>> one.ref2path(ref) WindowsPath('E:/FlatIron/zadorlab/Subjects/flowers/2018-07-13/001') >>> one.ref2path(['2018-07-13_1_flowers', '2019-04-11_1_KS005']) [WindowsPath('E:/FlatIron/zadorlab/Subjects/flowers/2018-07-13/001'), WindowsPath('E:/FlatIron/cortexlab/Subjects/KS005/2019-04-11/001')]
- static path2ref(path_str: str | Path | Iterable, as_dict=True) Bunch | List [source]
Returns a human-readable experiment reference, given a session path.
The path need not exist.
- Parameters:
path_str (str) – A path to a given session.
as_dict (bool) – If True a Bunch is returned, otherwise a string.
- Returns:
One or more objects with keys (‘subject’, ‘date’, ‘sequence’).
- Return type:
dict, str, list
Examples
>>> path_str = Path('E:/FlatIron/Subjects/zadorlab/flowers/2018-07-13/001') >>> path2ref(path_str) {'subject': 'flowers', 'date': datetime.date(2018, 7, 13), 'sequence': 1} >>> path2ref(path_str, parse=False) {'subject': 'flowers', 'date': '2018-07-13', 'sequence': '001'} >>> path_str2 = Path('E:/FlatIron/Subjects/churchlandlab/CSHL046/2020-06-20/002') >>> path2ref([path_str, path_str2]) [{'subject': 'flowers', 'date': datetime.date(2018, 7, 13), 'sequence': 1}, {'subject': 'CSHL046', 'date': datetime.date(2020, 6, 20), 'sequence': 2}]
- static is_exp_ref(ref: str | Mapping | Iterable) bool | List[bool] [source]
Returns True is ref is a valid experiment reference.
- Parameters:
ref (str, dict, list) – One or more objects with keys (‘subject’, ‘date’, ‘sequence’), or strings with the form yyyy-mm-dd_n_subject.
- Returns:
True if ref is valid.
- Return type:
bool, list of bool
Examples
>>> ref = {'date': datetime(2018, 7, 13).date(), 'sequence': 1, 'subject': 'flowers'} >>> is_exp_ref(ref) True >>> is_exp_ref('2018-07-13_001_flowers') True >>> is_exp_ref('invalid_ref') False
- static ref2dict(ref: str | Mapping | Iterable) Bunch | List [source]
Returns a Bunch (dict-like) from a reference string (or list thereof).
- Parameters:
ref (str, list) – One or more experiment reference strings.
- Returns:
A Bunch in with keys (‘subject’, ‘sequence’, ‘date’).
- Return type:
iblutil.util.Bunch
Examples
>>> ref2dict('2018-07-13_1_flowers') {'date': datetime.date(2018, 7, 13), 'sequence': 1, 'subject': 'flowers'} >>> ref2dict('2018-07-13_001_flowers', parse=False) {'date': '2018-07-13', 'sequence': '001', 'subject': 'flowers'} >>> ref2dict(['2018-07-13_1_flowers', '2020-01-23_002_ibl_witten_01']) [{'date': datetime.date(2018, 7, 13), 'sequence': 1, 'subject': 'flowers'}, {'date': datetime.date(2020, 1, 23), 'sequence': 2, 'subject': 'ibl_witten_01'}]
- static dict2ref(ref_dict) str | List [source]
Convert an experiment reference dict to a string in the format yyyy-mm-dd_n_subject.
- Parameters:
ref_dict (dict, Bunch, list, tuple) – A map with the keys (‘subject’, ‘date’, ‘sequence’).
- Returns:
An experiment reference string, or list thereof.
- Return type:
str, list
- one_path_from_dataset(dset, one_cache)[source]
Returns local one file path from a dset record or a list of dsets records from REST. Unlike to_eid, this function does not require ONE, and the dataset may not exist.
- Parameters:
dset (dict, list) – Dataset dictionary or list of dictionaries from Alyx rest endpoint.
one_cache (str, pathlib.Path, pathlib.PurePath) – The local ONE data cache directory.
- Returns:
The local path for a given dataset.
- Return type:
pathlib.Path
- path_from_dataset(dset, root_path=PurePosixPath('/'), repository=None, uuid=False)[source]
Returns the local file path from a dset record from a REST query. Unlike to_eid, this function does not require ONE, and the dataset may not exist.
- Parameters:
dset (dict, list) – Dataset dictionary or list of dictionaries from Alyx rest endpoint.
root_path (str, pathlib.Path, pathlib.PurePath) – The prefix path such as the ONE download directory or remote http server root.
repository (str, None) – Which data repository to use from the file_records list, defaults to first online repository.
uuid (bool) – If True, the file path will contain the dataset UUID.
- Returns:
File path or list of paths.
- Return type:
pathlib.Path, list
- path_from_filerecord(fr, root_path=PurePosixPath('/'), uuid=None)[source]
Returns a data file Path constructed from an Alyx file record. The Path type returned depends on the type of root_path: If root_path is a string a Path object is returned, otherwise if the root_path is a PurePath, the same path type is returned.
- Parameters:
fr (dict) – An Alyx file record dict.
root_path (str, pathlib.Path) – An optional root path.
uuid (str, uuid.UUID) – An optional dataset UUID to add to the file name.
- Returns:
A filepath as a pathlib object.
- Return type:
pathlib.Path
- session_record2path(session, root_dir=None)[source]
Convert a session record into a path.
If a lab key is present, the path will be in the form root_dir/lab/Subjects/subject/yyyy-mm-dd/nnn, otherwise root_dir/subject/yyyy-mm-dd/nnn.
- Parameters:
session (Mapping) – A session record with keys (‘subject’, ‘date’, ‘number’[, ‘lab’]).
root_dir (str, pathlib.Path, pathlib.PurePath) – A root directory to prepend.
- Returns:
A constructed path of the session.
- Return type:
pathlib.Path, Pathlib.PurePath
Examples
>>> session_record2path({'subject': 'ALK01', 'date': '2020-01-01', 'number': 1}) PurePosixPath('ALK01/2020-01-01/001')
>>> record = {'date': datetime.datetime.fromisoformat('2020-01-01').date(), ... 'number': '001', 'lab': 'foo', 'subject': 'ALK01'} >>> session_record2path(record, Path('/home/user')) Path('/home/user/foo/Subjects/ALK01/2020-01-01/001')