one.util

Decorators and small standalone functions for api module

Functions

Listable

Return a typing.Union if the input and sequence of input

autocomplete

Validate search term and return complete name, e.g.

cache_int2str

Convert int ids to str ids for cache table.

datasets2records

Extract datasets DataFrame from one or more Alyx dataset records

ensure_list

Ensure input is a list

filter_datasets

Filter the datasets cache table by the relative path (dataset name, collection and revision).

filter_revision_last_before

Filter datasets by revision, returning previous revision in ordered list if revision doesn't exactly match.

index_last_before

Returns the index of string that occurs directly before the provided revision string when lexicographic sorted.

parse_id

Ensures the input experiment identifier is an experiment UUID string

patch_cache

Reformat older cache tables to comply with this version of ONE.

refresh

Refresh cache depending of query_type kwarg

ses2records

Extract session cache record and datasets cache from a remote session data record.

validate_date_range

Validates and arrange date range in a 2 elements list

Classes

LazyId

Using a paginated response object or list of session records, extracts eid string when required

Listable(t)[source]

Return a typing.Union if the input and sequence of input

ses2records(ses: dict, int_id=False)[source]

Extract session cache record and datasets cache from a remote session data record.

Parameters
  • ses (dict) – Session dictionary from Alyx REST endpoint

  • int_id (bool) – If True, the UUIDs are converted to two int64s

Returns

  • pd.Series – Session record

  • pd.DataFrame – Datasets frame

datasets2records(datasets, int_id=False) DataFrame[source]

Extract datasets DataFrame from one or more Alyx dataset records

Parameters

datasets (dict, list) – One or more records from the Alyx ‘datasets’ endpoint

Returns

  • pd.DataFrame – Datasets frame

  • int_id (bool) – If True, the UUIDs are converted to two int64s

Examples

>>> datasets = ONE().alyx.rest('datasets', 'list', subject='foobar')
>>> df = datasets2records(datasets)
parse_id(method)[source]

Ensures the input experiment identifier is an experiment UUID string

Parameters

method (function) – An ONE method whose second arg is an experiment ID

Returns

A wrapper function that parses the ID to the expected string

Return type

function

Raises

ValueError – Unable to convert input to a valid experiment ID

refresh(method)[source]

Refresh cache depending of query_type kwarg

validate_date_range(date_range) -> (<class 'pandas._libs.tslibs.timestamps.Timestamp'>, <class 'pandas._libs.tslibs.timestamps.Timestamp'>)[source]

Validates and arrange date range in a 2 elements list

Parameters

date_range (str, datetime.date, datetime.datetime, pd.Timestamp, np.datetime64, list, None) – A single date or tuple/list of two dates. None represents no bound.

Returns

The start and end timestamps

Return type

tuple of pd.Timestamp

Examples

>>> validate_date_range('2020-01-01')  # On this day
>>> validate_date_range(datetime.date(2020, 1, 1))
>>> validate_date_range(np.array(['2022-01-30', '2022-01-30'], dtype='datetime64[D]'))
>>> validate_date_range(pd.Timestamp(2020, 1, 1))
>>> validate_date_range(np.datetime64(2021, 3, 11))
>>> validate_date_range(['2020-01-01'])  # from date
>>> validate_date_range(['2020-01-01', None])  # from date
>>> validate_date_range([None, '2020-01-01'])  # up to date
Raises

ValueError – Size of date range tuple must be 1 or 2

filter_datasets(all_datasets, filename=None, collection=None, revision=None, revision_last_before=True, assert_unique=True, wildcards=False)[source]

Filter the datasets cache table by the relative path (dataset name, collection and revision). When None is passed, all values will match. To match on empty parts, use an empty string. When revision_last_before is true, None means return latest revision.

Parameters
  • all_datasets (pandas.DataFrame) – A datasets cache table

  • filename (str, dict, None) – A filename str or a dict of alf parts. Regular expressions permitted.

  • collection (str, None) – A collection string. Regular expressions permitted.

  • revision (str, None) – A revision string to match. If revision_last_before is true, regular expressions are not permitted.

  • revision_last_before (bool) – When true and no exact match exists, the (lexicographically) previous revision is used instead. When false the revision string is matched like collection and filename, with regular expressions permitted.

  • assert_unique (bool) – When true an error is raised if multiple collections or datasets are found

  • wildcards (bool) – If true, use unix shell style matching instead of regular expressions

Returns

A slice of all_datasets that match the filters

Return type

pd.DataFrame

Examples

Filter by dataset name and collection

>>> datasets = filter_datasets(all_datasets, '.*spikes.times.*', 'alf/probe00')

Filter datasets not in a collection

>>> datasets = filter_datasets(all_datasets, collection='')

Filter by matching revision

>>> datasets = filter_datasets(all_datasets, 'spikes.times.npy',
...                            revision='2020-01-12', revision_last_before=False)

Filter by filename parts

>>> datasets = filter_datasets(all_datasets, dict(object='spikes', attribute='times'))
filter_revision_last_before(datasets, revision=None, assert_unique=True)[source]

Filter datasets by revision, returning previous revision in ordered list if revision doesn’t exactly match.

Parameters
  • datasets (pandas.DataFrame) – A datasets cache table

  • revision (str) – A revision string to match (regular expressions not permitted)

  • assert_unique (bool) – When true an alferr.ALFMultipleRevisionsFound exception is raised when multiple default revisions are found; an alferr.ALFError when no default revision is found

Returns

A datasets DataFrame with 0 or 1 row per unique dataset

Return type

pd.DataFrame

index_last_before(revisions: List[str], revision: Optional[str]) Optional[int][source]

Returns the index of string that occurs directly before the provided revision string when lexicographic sorted. If revision is None, the index of the most recent revision is returned.

Parameters
  • revisions (list of strings) – A list of revision strings

  • revision (None, str) – The revision string to match on

Returns

Index of revision before matching string in sorted list or None

Return type

int, None

Examples

>>> idx = index_last_before([], '2020-08-01')
autocomplete(term, search_terms) str[source]

Validate search term and return complete name, e.g. autocomplete(‘subj’) == ‘subject’

ensure_list(value)[source]

Ensure input is a list

class LazyId(pg)[source]

Bases: Mapping

Using a paginated response object or list of session records, extracts eid string when required

static ses2eid(ses)[source]
Parameters

ses (one.webclient._PaginatedResponse, dict, list) – A collection of Alyx REST sessions endpoint records

Returns

One or more experiment ID strings

Return type

str, list

cache_int2str(table: DataFrame) DataFrame[source]

Convert int ids to str ids for cache table.

Parameters

table (pd.DataFrame) – A cache table (from One._cache)

patch_cache(table: DataFrame, min_api_version=None) DataFrame[source]

Reformat older cache tables to comply with this version of ONE.

Currently this function will 1. convert integer UUIDs to string UUIDs; 2. rename the ‘project’ column to ‘projects’.

Parameters
  • table (pd.DataFrame) – A cache table (from One._cache).

  • min_api_version (str) – The minimum API version supported by this cache table.