ONE API modes
Online vs Offline
ONE can be instantiated in either the online and offline state. When online, ONE can query the Alyx database to retrieve information about dataset types , search with metadata such as session QC, task performance, histology and session narrative. Advanced queries may also be made via REST. Other online methods include pid2eid
and describe_revision
.
When the mode is not specified, it is usually set to ‘auto’, unless a cache_dir is specified for which no database has been configured.
[2]:
import one.webclient as webclient
from one.api import ONE
one = ONE()
print(one, one.mode) # online, 'auto' mode
assert not one.offline
one_offline = ONE(cache_dir=r'C:\data')
print(one_offline, one_offline.mode) # offline, 'local' mode
assert one_offline.offline
One (online, https://alyx.internationalbrainlab.org) auto
One (offline, C:\data) local
Query modes
In ‘auto’ mode, the list, search and load methods will use the local cache tables and not connect to Alyx, however the option to use Alyx is always there. If the cache tables can’t be downloaded from Alyx, or authentication fails, the mode will fall back to ‘local’.
If ‘remote’ mode is specified, ONE will only query the remote database and will not use the local cache tables. Avoiding the database whenever possible is recommended as it doesn’t rely on a stable internet connection and reduces the load on the remote Alyx.
While in ‘remote’ mode, the local cache may be used by providing the query_type=’local’ keyword argument to any method. Likewise, in ‘auto’/’local’ mode, a remote query can be made by specifying query_type='remote'
NB: The ‘remote’ query type is not valid in offline mode as there is no database associated to the local cache directory.
[3]:
eids = one.search(lab='cortexlab', query_type='remote') # Search Alyx instead of the local cache
REST caching
In remote mode ONE makes a REST query instead of using the local cache tables. The results of the remote REST queries are also cached for 24 hours. This means that making the same remote REST query twice in a row will only hit the database once. The default cache expiry can be set by changing the relevant AlyxClient property:
[ ]:
from datetime import timedelta
one.alyx.default_expiry = timedelta(days=20) # Cache results for up to 20 days
You can temporarily deactivate the REST cache using the no_cache
function in one.webclient
. When in this context no REST responses are cached and any existing cache files are not used. Use this when the most up-to-date information is required:
[ ]:
with webclient.no_cache(one.alyx):
eids, det = one.search(lab='cortexlab', query_type='remote')
When calling the alyx rest
method directly you can deactivate the cache with the no_cache
keyword argument:
[ ]:
ses = one.alyx.rest('sessions', 'list', lab='cortexlab', no_cache=True)
Caching greatly improves performance and should be used whenever possible. For more information on ONE REST queries, see this guide.
You can turn off REST caching when instantiating ONE with the cache_rest
keyword argument:
[ ]:
one = ONE(cache_rest=None, mode='remote')
Refreshing the cache tables
By default ONE will try to update the cache tables once every 24 hours. This can be set by changing the ‘cache_expiry’ property:
[4]:
from datetime import timedelta
one.cache_expiry = timedelta(hours=1) # Check for new remote cache every hour
# The time when the cache was generated can be found in the cache metadata:
one._cache._meta
Out[4]:
{'expired': False,
'created_time': datetime.datetime(2021, 9, 14, 13, 0),
'loaded_time': datetime.datetime(2021, 9, 14, 18, 15, 54, 384591),
'raw': {'datasets': {'date_created': '2021-09-14 13:00', 'origin': 'alyx'},
'sessions': {'date_created': '2021-09-14 13:00', 'origin': 'alyx'}}}
Note that the cache won’t be downloaded if the remote cache hasn’t been updated since the last download. The cache can be explicitly refreshed in two ways:
[5]:
loaded_time = one.refresh_cache('refresh') # Explicitly refresh the cache
eids = one.search(lab='cortexlab', query_type='refresh') # Calls `refresh_cache` before searching
Summary
Mode overview:
Mode |
Function |
---|---|
local |
only use current cache tables i.e. work entirely offline* |
remote |
make REST queries instead of using cache tables |
auto |
use cache tables wherever possible and keep them updated |
refresh |
always check for new cache tables |
*no remote cache tables are downloaded; local cache files must be present in this mode.
I want to make a REST query for up-to-the-minute information Use one.alyx.rest(..., no_cache=True)
.
I want to use a remote ONE query with up-to-the-minute information Call the ONE method from within the webclient.no_cache
context.