{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Exploring the IBL Data Pipeline\n", "\n", "Here we will introduce some useful DataJoint tools and concepts to help you explore the IBL data pipeline. Before proceeding make sure that you have installed the [IBL python environment](../../02_installation.md) and set up your [Datajoint credentials](../../dj_docs/dj_credentials.md)\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## A few definitions\n", "First of all, let's define a few basic definitions:\n", "\n", "- Table - collection of rows and columns that contain data\n", "- Schema - a collection of related tables \n", "- Module - script where schema and associated tables are defined\n", "- Pipeline - collection of schemas\n", "\n", "Example nomenclature would be to say that we want to get data from the `Subject table` stored in the `ibl_subjects schema` which are together defined in the `subject module` in the `IBL pipeline`. \n", "\n", "In practice, `modules` are often referred to as schemas and so we would refer to the `subject module` as the `subject schema`" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's now proceed by importing Datajoint and some schemas from the IBL pipeline" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "execution": { "iopub.execute_input": "2022-11-07T11:24:17.985769Z", "iopub.status.busy": "2022-11-07T11:24:17.985769Z", "iopub.status.idle": "2022-11-07T11:24:42.187777Z", "shell.execute_reply": "2022-11-07T11:24:42.187777Z" } }, "outputs": [], "source": [ "import datajoint as dj\n", "dj.config['display.limit'] = 5\n", "from ibl_pipeline import reference, subject, action, acquisition, data, behavior\n", "from ibl_pipeline.analyses import behavior as behavior_analyses" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 1. Browsing schemas\n", "The IBL pipeline contains a number of different schemas. To browse which schemas are available, we can use the `dj.list_schemas()` command," ] }, { "cell_type": "code", "execution_count": null, "metadata": { "execution": { "iopub.execute_input": "2022-11-07T11:24:42.203268Z", "iopub.status.busy": "2022-11-07T11:24:42.203268Z", "iopub.status.idle": "2022-11-07T11:24:42.343839Z", "shell.execute_reply": "2022-11-07T11:24:42.343839Z" } }, "outputs": [], "source": [ "dj.list_schemas()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "### Major schemas include: \n", "Meta data from **Alyx**: `ibl_reference`, `ibl_subject`, `ibl_action`, `ibl_acquisition`, and `ibl_data` \n", "Imported data from **FlatIron**: `ibl_behavior` and `ibl_ephys` \n", "Computed analzyed results: `ibl_analyses_behavior`\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 2. Browsing tables in a schemas\n", "We can see what tables are defined in a schema using the `dj.Diagram` command. For example, to see the tables defined in the `subject` schema we can type," ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "```\n", "dj.Diagram(subject)\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "