ONE API modes

Online vs Offline

ONE can be instantiated in either the online and offline state. When online, ONE can query the Alyx database to retrieve information about dataset types , search with metadata such as session QC, task performance, histology and session narrative. Advanced queries may also be made via REST. Other online methods include pid2eid and describe_revision.

When the mode is not specified, it is usually set to ‘auto’, unless a cache_dir is specified for which no database has been configured.

[2]:

import one.webclient as webclient
from one.api import ONE

one = ONE()
print(one, one.mode)  # online, 'auto' mode
assert not one.offline

one_offline = ONE(cache_dir=r'C:\data')
print(one_offline, one_offline.mode)  # offline, 'local' mode
assert one_offline.offline

One (online, https://alyx.internationalbrainlab.org) auto
One (offline, C:\data) local

Query modes

In ‘auto’ mode, the list, search and load methods will use the local cache tables and not connect to Alyx, however the option to use Alyx is always there. If the cache tables can’t be downloaded from Alyx, or authentication fails, the mode will fall back to ‘local’.

If ‘remote’ mode is specified, ONE will only query the remote database and will not use the local cache tables. Avoiding the database whenever possible is recommended as it doesn’t rely on a stable internet connection and reduces the load on the remote Alyx.

While in ‘remote’ mode, the local cache may be used by providing the query_type=’local’ keyword argument to any method. Likewise, in ‘auto’/’local’ mode, a remote query can be made by specifying query_type='remote'

NB: The ‘remote’ query type is not valid in offline mode as there is no database associated to the local cache directory.

[3]:

eids = one.search(lab='cortexlab', query_type='remote')  # Search Alyx instead of the local cache

REST caching

In remote mode ONE makes a REST query instead of using the local cache tables. The results of the remote REST queries are also cached for 24 hours. This means that making the same remote REST query twice in a row will only hit the database once. The default cache expiry can be set by changing the relevant AlyxClient property:

[ ]:

from datetime import timedelta
one.alyx.default_expiry = timedelta(days=20)  # Cache results for up to 20 days

You can temporarily deactivate the REST cache using the no_cache function in one.webclient. When in this context no REST responses are cached and any existing cache files are not used. Use this when the most up-to-date information is required:

[ ]:

with webclient.no_cache(one.alyx):
    eids, det = one.search(lab='cortexlab', query_type='remote')

When calling the alyx rest method directly you can deactivate the cache with the no_cache keyword argument:

[ ]:

ses = one.alyx.rest('sessions', 'list', lab='cortexlab', no_cache=True)

Caching greatly improves performance and should be used whenever possible. For more information on ONE REST queries, see this guide.

You can turn off REST caching when instantiating ONE with the cache_rest keyword argument:

[ ]:

one = ONE(cache_rest=None, mode='remote')

Refreshing the cache tables

By default ONE will try to update the cache tables once every 24 hours. This can be set by changing the ‘cache_expiry’ property:

[4]:

from datetime import timedelta
one.cache_expiry = timedelta(hours=1)  # Check for new remote cache every hour

# The time when the cache was generated can be found in the cache metadata:
one._cache._meta

Out[4]:

{'expired': False,
 'created_time': datetime.datetime(2021, 9, 14, 13, 0),
 'loaded_time': datetime.datetime(2021, 9, 14, 18, 15, 54, 384591),
 'raw': {'datasets': {'date_created': '2021-09-14 13:00', 'origin': 'alyx'},
  'sessions': {'date_created': '2021-09-14 13:00', 'origin': 'alyx'}}}

Note that the cache won’t be downloaded if the remote cache hasn’t been updated since the last download. The cache can be explicitly refreshed in two ways:

[5]:

loaded_time = one.refresh_cache('refresh')  # Explicitly refresh the cache
eids = one.search(lab='cortexlab', query_type='refresh')  # Calls `refresh_cache` before searching

Summary

Mode overview:

Mode	Function
local	only use current cache tables i.e. work entirely offline*
remote	make REST queries instead of using cache tables
auto	use cache tables wherever possible and keep them updated
refresh	always check for new cache tables

*no remote cache tables are downloaded; local cache files must be present in this mode.

I want to make a REST query for up-to-the-minute information Use one.alyx.rest(..., no_cache=True).

I want to use a remote ONE query with up-to-the-minute information Call the ONE method from within the webclient.no_cache context.