Download and decompress raw ephys data

This code demonstrates how to decompress raw ephys (binary) data - This is necessary for some client codes (such as Matlab spike sorting KS2 algorithm) to run

(example taken for nidq.cbin file, but also applicable for lf.cbin and ap.cbin files)

# Author: Olivier, Gaelle, Mayo
from pprint import pprint

from ibllib.io import spikeglx
from one.api import ONE

one = ONE(base_url='https://openalyx.internationalbrainlab.org', silent=True)
# Download a dataset of interest
eid = one.search(subject='KS023', date_range='2019-12-10')[0]

# Optionally list the raw ephys data for this session
pprint([x for x in one.list_datasets(eid) if 'ephysData' in x])

files = one.load_object(eid, 'ephysData_g0_t0',
                        attribute='nidq', collection='raw_ephys_data', download_only=True)

# Get file path of interest
efile = next(x for x in files if str(x).endswith('.cbin'))

# Read the files and get the data
# Enough to do analysis
sr = spikeglx.Reader(efile)

# Decompress the data
# Used by client code, e.g. Matlab for spike sorting
# Give new path output name
sr.decompress_file(keep_original=True, overwrite=True)  # Keep the original file and overwrite any
# previously decompressed file

# For ap/lf data from a given probe
# probe_label = 'probe00'
# files = one.load_object(eid, 'ephysData_g0_t0', download_only=True,
#                         attribute='nidq', collection=f'raw_ephys_data/{probe_label}')

['raw_ephys_data/_spikeglx_ephysData_g0_t0.nidq.cbin',
 'raw_ephys_data/_spikeglx_ephysData_g0_t0.nidq.ch',
 'raw_ephys_data/_spikeglx_ephysData_g0_t0.nidq.meta',
 'raw_ephys_data/_spikeglx_ephysData_g0_t0.nidq.wiring.json',
 'raw_ephys_data/probe01/_spikeglx_ephysData_g0_t0.imec1.ap.cbin',
 'raw_ephys_data/probe01/_spikeglx_ephysData_g0_t0.imec1.ap.ch',
 'raw_ephys_data/probe01/_spikeglx_ephysData_g0_t0.imec1.ap.meta',
 'raw_ephys_data/probe01/_spikeglx_ephysData_g0_t0.imec1.lf.cbin',
 'raw_ephys_data/probe01/_spikeglx_ephysData_g0_t0.imec1.lf.ch',
 'raw_ephys_data/probe01/_spikeglx_ephysData_g0_t0.imec1.lf.meta',
 'raw_ephys_data/probe01/_spikeglx_ephysData_g0_t0.imec1.sync.npy',
 'raw_ephys_data/probe01/_spikeglx_ephysData_g0_t0.imec1.timestamps.npy',
 'raw_ephys_data/probe01/_spikeglx_ephysData_g0_t0.imec1.wiring.json']

Decompressing: 100%|█████████████████████████████████████████████████████████████████| 410/410 [00:04<00:00, 82.24it/s]
INFO:mtscomp:Wrote C:\Users\Mayo\Downloads\ONE\openalyx.internationalbrainlab.org\cortexlab\Subjects\KS023\2019-12-10\001\raw_ephys_data\_spikeglx_ephysData_g0_t0.nidq.bin (0.4 GB).
Checking: 100%|███████████████████████████████████████████████████████████████████| 3278/3278 [00:07<00:00, 467.29it/s]

WindowsPath('C:/Users/Mayo/Downloads/ONE/openalyx.internationalbrainlab.org/cortexlab/Subjects/KS023/2019-12-10/001/raw_ephys_data/_spikeglx_ephysData_g0_t0.nidq.bin')