ONE API modes

Online vs Offline

ONE can be instantiated in either the online and offline state. When online, ONE can query the Alyx database to retrieve information about dataset types , search with metadata such as session QC, task performance, histology and session narrative. Advanced queries may also be made via REST. Other online methods include pid2eid and describe_revision.

When the mode is not specified, it is usually set to ‘auto’, unless a cache_dir is specified for which no database has been configured.

[2]:
import one.webclient as webclient
from one.api import ONE

one = ONE()
print(one, one.mode)  # online, 'auto' mode
assert not one.offline

one_offline = ONE(cache_dir=r'C:\data')
print(one_offline, one_offline.mode)  # offline, 'local' mode
assert one_offline.offline
One (online, https://alyx.internationalbrainlab.org) auto
One (offline, C:\data) local

Query modes

In ‘auto’ mode, the list, search and load methods will use the local cache tables and not connect to Alyx, however the option to use Alyx is always there. If the cache tables can’t be downloaded from Alyx, or authentication fails, the mode will fall back to ‘local’.

If ‘remote’ mode is specified, ONE will only query the remote database and will not use the local cache tables. Avoiding the database whenever possible is recommended as it doesn’t rely on a stable internet connection and reduces the load on the remote Alyx.

While in ‘remote’ mode, the local cache may be used by providing the query_type=‘local’ keyword argument to any method. Likewise, in ‘auto’/‘local’ mode, a remote query can be made by specifying query_type='remote'

NB: The ‘remote’ query type is not valid in offline mode as there is no database associated to the local cache directory.

[3]:
eids = one.search(lab='cortexlab', query_type='remote')  # Search Alyx instead of the local cache

REST caching

In remote mode ONE makes a REST query instead of using the local cache tables. The results of the remote REST queries are also cached for 24 hours. This means that making the same remote REST query twice in a row will only hit the database once. The default cache expiry can be set by changing the relevant AlyxClient property:

[ ]:
from datetime import timedelta
one.alyx.default_expiry = timedelta(days=20)  # Cache results for up to 20 days

You can temporarily deactivate the REST cache using the no_cache function in one.webclient. When in this context no REST responses are cached and any existing cache files are not used. Use this when the most up-to-date information is required:

[ ]:
with webclient.no_cache(one.alyx):
    eids, det = one.search(lab='cortexlab', query_type='remote')

When calling the alyx rest method directly you can deactivate the cache with the no_cache keyword argument:

[ ]:
ses = one.alyx.rest('sessions', 'list', lab='cortexlab', no_cache=True)

Caching greatly improves performance and should be used whenever possible. For more information on ONE REST queries, see this guide.

Refreshing the cache tables

By default ONE will try to update the cache tables once every 24 hours. This can be set by changing the ‘cache_expiry’ property:

[4]:
from datetime import timedelta
one.cache_expiry = timedelta(hours=1)  # Check for new remote cache every hour

# The time when the cache was generated can be found in the cache metadata:
one._cache._meta
Out[4]:
{'expired': False,
 'created_time': datetime.datetime(2021, 9, 14, 13, 0),
 'loaded_time': datetime.datetime(2021, 9, 14, 18, 15, 54, 384591),
 'raw': {'datasets': {'date_created': '2021-09-14 13:00', 'origin': 'alyx'},
  'sessions': {'date_created': '2021-09-14 13:00', 'origin': 'alyx'}}}

Note that the cache won’t be downloaded if the remote cache hasn’t been updated since the last download. The cache can be explicitly refreshed in two ways:

[5]:
loaded_time = one.refresh_cache('refresh')  # Explicitly refresh the cache
eids = one.search(lab='cortexlab', query_type='refresh')  # Calls `refresh_cache` before searching

Summary

Mode overview:

Mode

Function

local

only use current cache tables i.e. work entirely offline*

remote

make REST queries instead of using cache tables

auto

use cache tables wherever possible and keep them updated

refresh

always check for new cache tables

*no remote cache tables are downloaded; local cache files must be present in this mode.

I want to make a REST query for up-to-the-minute information
Use one.alyx.rest(..., no_cache=True).
I want to use a remote ONE query with up-to-the-minute information
Call the ONE method from within the webclient.no_cache context.