{ "cells": [ { "cell_type": "markdown", "source": [ "# ONE API modes\n", "## Online vs Offline\n", "ONE can be instantiated in either the online and offline state. When online, ONE can query the\n", "Alyx database to retrieve information about dataset types , search with metadata such as\n", "session QC, task performance, histology and session narrative. Advanced queries may also be\n", "made [via REST](./one_advanced/one_advanced.html). Other online methods include `pid2eid` and\n", "`describe_revision`.\n", "\n", "When the mode is not specified, it is usually set to 'auto', unless a cache_dir is specified for\n", "which no database has been configured." ], "metadata": { "collapsed": false } }, { "cell_type": "code", "execution_count": 2, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "One (online, https://alyx.internationalbrainlab.org) auto\n", "One (offline, C:\\data) local\n" ] } ], "source": [ "import one.webclient as webclient\n", "from one.api import ONE\n", "\n", "one = ONE()\n", "print(one, one.mode) # online, 'auto' mode\n", "assert not one.offline\n", "\n", "one_offline = ONE(cache_dir=r'C:\\data')\n", "print(one_offline, one_offline.mode) # offline, 'local' mode\n", "assert one_offline.offline" ], "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } } }, { "cell_type": "markdown", "source": [ "## Query modes\n", "In 'auto' mode, the list, search and load methods will use the local cache tables and not\n", "connect to Alyx, however the option to use Alyx is always there. If the cache tables can't be\n", "downloaded from Alyx, or authentication fails, the mode will fall back to 'local'.\n", "\n", "If 'remote' mode is specified, ONE will only query the remote database and will not use the\n", "local cache tables. Avoiding the database whenever possible is recommended as it doesn't rely\n", "on a stable internet connection and reduces the load on the remote Alyx.\n", "\n", "While in 'remote' mode, the local cache may be used by providing the query_type='local' keyword\n", "argument to any method. Likewise, in 'auto'/'local' mode, a remote query can be made by\n", "specifying `query_type='remote'`\n", "\n", "
\n", "NB: The 'remote' query type is not valid in offline mode as there is no database associated to\n", "the local cache directory.\n", "
" ], "metadata": { "collapsed": false, "pycharm": { "name": "#%% md\n" } } }, { "cell_type": "code", "execution_count": 3, "outputs": [], "source": [ "eids = one.search(lab='cortexlab', query_type='remote') # Search Alyx instead of the local cache" ], "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } } }, { "cell_type": "markdown", "source": [ "## REST caching\n", "In remote mode ONE makes a REST query instead of using the local cache tables. The results of\n", "the remote REST queries are also cached for 24 hours. This means that making the same remote\n", "REST query twice in a row will only hit the database once. The default cache expiry can be set\n", "by changing the relevant AlyxClient property:" ], "metadata": { "collapsed": false, "pycharm": { "name": "#%% md\n" } } }, { "cell_type": "code", "execution_count": null, "outputs": [], "source": [ "from datetime import timedelta\n", "one.alyx.default_expiry = timedelta(days=20) # Cache results for up to 20 days" ], "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } } }, { "cell_type": "markdown", "source": [ "You can temporarily deactivate the REST cache using the `no_cache` function in `one.webclient`.\n", "When in this context no REST responses are cached and any existing cache files are not used.\n", "Use this when the most up-to-date information is required:" ], "metadata": { "collapsed": false, "pycharm": { "name": "#%% md\n" } } }, { "cell_type": "code", "execution_count": null, "outputs": [], "source": [ "with webclient.no_cache(one.alyx):\n", " eids, det = one.search(lab='cortexlab', query_type='remote')" ], "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } } }, { "cell_type": "markdown", "source": [ "When calling the alyx `rest` method directly you can deactivate the cache with the `no_cache`\n", "keyword argument:\n" ], "metadata": { "collapsed": false, "pycharm": { "name": "#%% md\n" } } }, { "cell_type": "code", "execution_count": null, "outputs": [], "source": [ "ses = one.alyx.rest('sessions', 'list', lab='cortexlab', no_cache=True)" ], "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } } }, { "cell_type": "markdown", "source": [ "Caching greatly improves performance and should be used whenever possible.\n", "For more information on ONE REST queries, see [this guide](./one_advanced/one_advanced.html).\n", "\n", "You can turn off REST caching when instantiating ONE with the `cache_rest` keyword argument:" ], "metadata": { "collapsed": false, "pycharm": { "name": "#%% md\n" } } }, { "cell_type": "code", "execution_count": null, "outputs": [], "source": [ "one = ONE(cache_rest=None, mode='remote')" ], "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } } }, { "cell_type": "markdown", "source": [ "## Refreshing the cache tables\n", "By default ONE will try to update the cache tables once every 24 hours. This can be set by\n", "changing the 'cache_expiry' property:" ], "metadata": { "collapsed": false, "pycharm": { "name": "#%% md\n" } } }, { "cell_type": "code", "execution_count": 4, "outputs": [ { "data": { "text/plain": "{'expired': False,\n 'created_time': datetime.datetime(2021, 9, 14, 13, 0),\n 'loaded_time': datetime.datetime(2021, 9, 14, 18, 15, 54, 384591),\n 'raw': {'datasets': {'date_created': '2021-09-14 13:00', 'origin': 'alyx'},\n 'sessions': {'date_created': '2021-09-14 13:00', 'origin': 'alyx'}}}" }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from datetime import timedelta\n", "one.cache_expiry = timedelta(hours=1) # Check for new remote cache every hour\n", "\n", "# The time when the cache was generated can be found in the cache metadata:\n", "one._cache._meta" ], "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } } }, { "cell_type": "markdown", "source": [ "Note that the cache won't be downloaded if the remote cache hasn't been updated since the last\n", "download. The cache can be explicitly refreshed in two ways:" ], "metadata": { "collapsed": false, "pycharm": { "name": "#%% md\n" } } }, { "cell_type": "code", "execution_count": 5, "outputs": [], "source": [ "loaded_time = one.refresh_cache('refresh') # Explicitly refresh the cache\n", "eids = one.search(lab='cortexlab', query_type='refresh') # Calls `refresh_cache` before searching" ], "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } } }, { "cell_type": "markdown", "source": [ "## Summary\n", "Mode overview:\n", "\n", "| **Mode** | **Function** |\n", "|----------|-----------------------------------------------------------|\n", "| local | only use current cache tables i.e. work entirely offline* |\n", "| remote | make REST queries instead of using cache tables |\n", "| auto | use cache tables wherever possible and keep them updated |\n", "| refresh | always check for new cache tables |\n", "\n", "\\*no remote cache tables are downloaded; local cache files must be present in this mode.\n", "\n", "**I want to make a REST query for up-to-the-minute information**\n", "Use `one.alyx.rest(..., no_cache=True)`.\n", "\n", "**I want to use a remote ONE query with up-to-the-minute information**\n", "Call the ONE method from within the `webclient.no_cache` context." ], "metadata": { "collapsed": false, "pycharm": { "name": "#%% md\n" } } } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 2 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython2", "version": "2.7.6" } }, "nbformat": 4, "nbformat_minor": 0 }