S3 Data Architecture

All IBL electrophysiology atlas data lives in a single private AWS S3 bucket:

s3://ibl-brain-wide-map-private/aggregates/atlas/

Authentication uses one.api.ONE via Alyx credentials — pass a logged-in ONE instance as the one argument to every download function.

Folder Layout

aggregates/atlas/
│
├── features/{project}/{label}/agg_full/     ← channel-level ephys features
│   ├── raw_ephys_features_denoised.pqt      denoised channel features  (~200 MB)
│   ├── raw_ephys_features.pqt               raw channel features
│   ├── channels.pqt                         channel metadata (coordinates, probe info)
│   └── channels_labels.pqt                  region labels per channel
│
├── features/{project}/{label}_extended/     ← large optional data (cross-correlograms …)
│
├── encoding_volumes/{project}/{label}/      ← 4-D CCF volume, one file per resolution
│   ├── brainwide_ephys_atlas_25um.npz       (456 × 528 × 320 × N_features), ~500 MB
│   └── brainwide_ephys_atlas_50um.npz       (228 × 264 × 160 × N_features), ~240 MB
│
├── models/{model_name}/                     ← trained region classifier
│   ├── model.ubj
│   └── meta.yaml
│
└── projects/{project}/                      ← per-project cluster/LFP aggregates
    ├── df_probe_details.pqt                 one row per probe insertion
    ├── cells_aggregates/
    │   ├── clusters.table.pqt               all clusters (n_clusters × ~59)
    │   ├── clusters_good.table.pqt          QC-passing clusters (n_good × ~61)
    │   ├── clusters.acgs_log.npy            log-binned ACGs, normalised  (n_clusters × 128) float16
    │   ├── acgs_log.times.npy               ACG bin centres in seconds   (128,) float64
    │   ├── clusters.waveforms_peak.npy      peak-channel waveform        (n_clusters × 128) float16
    │   ├── clusters_good.stpc.npy           spike-triggered population coupling  (n_good × 1000) float16
    │   ├── clusters_good.stlfp.npy          spike-triggered LFP                  (n_good × 250)  float16
    │   ├── waveforms.voltage.npy            neighbourhood traces (~8 GB)         (n_traces × 128) float16
    │   └── waveforms.table.pqt              pid/cluster_id/abs_channel index     (n_traces × 3)
    └── lfp_aggregates/                      ← merged LFP archives v04, one group per pid (lfpack ≥ 1.0)
        ├── lf_compressed_v04_a14_small_all.h5    small   (ε=100, α=14)   ~11 GB
        ├── lf_compressed_v04_a07_default_all.h5  default (ε=100, α=7)    ~21.5 GB
        └── lf_compressed_v04_a2p5_fine_all.h5    fine    (ε=100, α=2.5)  ~46 GB

Versioning

Channel features and encoding volumes are versioned with a weekly label of the form YYYY_Www (e.g. 2025_W28, 2026_W12). Encoding volumes are additionally versioned by voxel resolution (res_um, e.g. 25 or 50); if omitted, it auto-resolves to the finest resolution available on S3 for that label — see ephysatlas.data.download_encoding_volume().

import ephysatlas.data
from one.api import ONE

one = ONE(base_url='https://alyx.internationalbrainlab.org')
labels = ephysatlas.data.list_available_labels(one=one, project='ea_active')
latest = ephysatlas.data.get_latest_label(one=one, project='ea_active')

Projects

Two main projects exist:

  • ea_active — default project, updated weekly with the latest features

  • ibl_neuropixel_brainwide_01 — frozen brainwide map dataset

Download Functions

Function

S3 path

ephysatlas.data.download_tables()

features/{project}/{label}/agg_full/

ephysatlas.data.download_encoding_volume()

encoding_volumes/{project}/{label}/

ephysatlas.data.download_probe_details()

projects/{project}/df_probe_details.pqt

ephysatlas.data.download_cells_features()

projects/{project}/cells_aggregates/

ephysatlas.data.download_project_data()

probe details + cell aggregates (convenience wrapper)

ephysatlas.data.download_lfp_features()

projects/{project}/lfp_aggregates/

See also