Loading SpikeSorting Data
Spikesorted output of electrophysiology data.
Relevant Alf objects
channels
clusters
spikes
Loading
[2]:
from one.api import ONE
from brainbox.io.one import SpikeSortingLoader
from ibllib.atlas import AllenAtlas
one = ONE()
ba = AllenAtlas()
pid = 'da8dfec1-d265-44e8-84ce-6ae9c109b8bd'
sl = SpikeSortingLoader(pid=pid, one=one, atlas=ba)
spikes, clusters, channels = sl.load_spike_sorting()
clusters = sl.merge_clusters(spikes, clusters, channels)
/home/runner/work/iblenv/ibllib-repo/ibllib/atlas/regions.py:439: RuntimeWarning: invalid value encountered in cast
level=df_regions.depth.to_numpy().astype(np.uint16),
/home/runner/work/iblenv/ibllib-repo/ibllib/atlas/regions.py:441: RuntimeWarning: invalid value encountered in cast
order=df_regions.graph_order.to_numpy().astype(np.uint16))
Downloading: /home/runner/Downloads/ONE/openalyx.internationalbrainlab.org/histology/ATLAS/Needles/Allen/average_template_25.nrrd Bytes: 32998960
100%|██████████| 31.470260620117188/31.470260620117188 [00:03<00:00, 10.00it/s]
Downloading: /home/runner/Downloads/ONE/openalyx.internationalbrainlab.org/histology/ATLAS/Needles/Allen/annotation_25.nrrd Bytes: 4035363
100%|██████████| 3.848422050476074/3.848422050476074 [00:01<00:00, 2.04it/s]
/home/runner/Downloads/ONE/openalyx.internationalbrainlab.org/hoferlab/Subjects/SWC_043/2020-09-21/001/alf/probe00/pykilosort/spikes.amps.npy: 100%|██████████| 141M/141M [00:01<00:00, 83.0MB/s]
/home/runner/Downloads/ONE/openalyx.internationalbrainlab.org/hoferlab/Subjects/SWC_043/2020-09-21/001/alf/probe00/pykilosort/spikes.clusters.npy: 100%|██████████| 70.3M/70.3M [00:00<00:00, 112MB/s]
/home/runner/Downloads/ONE/openalyx.internationalbrainlab.org/hoferlab/Subjects/SWC_043/2020-09-21/001/alf/probe00/pykilosort/spikes.times.npy: 100%|██████████| 141M/141M [00:00<00:00, 178MB/s]
/home/runner/Downloads/ONE/openalyx.internationalbrainlab.org/hoferlab/Subjects/SWC_043/2020-09-21/001/alf/probe00/pykilosort/spikes.depths.npy: 100%|██████████| 141M/141M [00:00<00:00, 166MB/s]
/home/runner/Downloads/ONE/openalyx.internationalbrainlab.org/hoferlab/Subjects/SWC_043/2020-09-21/001/alf/probe00/pykilosort/clusters.depths.npy: 100%|██████████| 3.78k/3.78k [00:00<00:00, 21.1kB/s]
/home/runner/Downloads/ONE/openalyx.internationalbrainlab.org/hoferlab/Subjects/SWC_043/2020-09-21/001/alf/probe00/pykilosort/clusters.uuids.csv: 100%|██████████| 33.8k/33.8k [00:00<00:00, 177kB/s]
/home/runner/Downloads/ONE/openalyx.internationalbrainlab.org/hoferlab/Subjects/SWC_043/2020-09-21/001/alf/probe00/pykilosort/clusters.channels.npy: 100%|██████████| 7.44k/7.44k [00:00<00:00, 55.8kB/s]
/home/runner/Downloads/ONE/openalyx.internationalbrainlab.org/hoferlab/Subjects/SWC_043/2020-09-21/001/alf/probe00/pykilosort/clusters.metrics.pqt: 100%|██████████| 108k/108k [00:00<00:00, 615kB/s]
/home/runner/Downloads/ONE/openalyx.internationalbrainlab.org/hoferlab/Subjects/SWC_043/2020-09-21/001/alf/probe00/pykilosort/channels.mlapdv.npy: 100%|██████████| 4.74k/4.74k [00:00<00:00, 35.5kB/s]
/home/runner/Downloads/ONE/openalyx.internationalbrainlab.org/hoferlab/Subjects/SWC_043/2020-09-21/001/alf/probe00/pykilosort/channels.rawInd.npy: 100%|██████████| 3.20k/3.20k [00:00<00:00, 28.0kB/s]
/home/runner/Downloads/ONE/openalyx.internationalbrainlab.org/hoferlab/Subjects/SWC_043/2020-09-21/001/alf/probe00/pykilosort/channels.localCoordinates.npy: 100%|██████████| 3.20k/3.20k [00:00<00:00, 17.6kB/s]
/home/runner/Downloads/ONE/openalyx.internationalbrainlab.org/hoferlab/Subjects/SWC_043/2020-09-21/001/alf/probe00/pykilosort/channels.brainLocationIds_ccf_2017.npy: 100%|██████████| 3.20k/3.20k [00:00<00:00, 30.9kB/s]
/home/runner/Downloads/ONE/openalyx.internationalbrainlab.org/hoferlab/Subjects/SWC_043/2020-09-21/001/alf/probe00/electrodeSites.brainLocationIds_ccf_2017.npy: 100%|██████████| 3.20k/3.20k [00:00<00:00, 28.6kB/s]
/home/runner/Downloads/ONE/openalyx.internationalbrainlab.org/hoferlab/Subjects/SWC_043/2020-09-21/001/alf/probe00/electrodeSites.mlapdv.npy: 100%|██████████| 4.74k/4.74k [00:00<00:00, 32.0kB/s]
/home/runner/Downloads/ONE/openalyx.internationalbrainlab.org/hoferlab/Subjects/SWC_043/2020-09-21/001/alf/probe00/electrodeSites.localCoordinates.npy: 100%|██████████| 3.20k/3.20k [00:00<00:00, 22.7kB/s]
Alternatively, one can instantiate the spike sorting loader using the session unique identifier eid and the probe name pname:
[3]:
eid, pname = one.pid2eid(pid)
sl = SpikeSortingLoader(eid=eid, pname=pname, one=one, atlas=ba)
spikes, clusters, channels = sl.load_spike_sorting()
clusters = sl.merge_clusters(spikes, clusters, channels)
More details
Available spikesorting algorithms
pykilosort (All data)
Kilosort 2.5 (Most data collected before August 2021)
Important information
Data sorted with pykilosort is loaded by default. If the pykilosort spikesorting data is not available, the matlab kilosort 2.5 version will be loaded. See Example 1 for more information.
The channel locations in the brain can come from several sources. it will load the most advanced version of the histology available, regardless of the spike sorting version loaded. The steps, from most to least advanced, are:
alf: the final version of channel locations, same as resolved with the difference that data has been written out to files
resolved: channel location alignments have been agreed upon
aligned: channel locations have been aligned, but review or other alignments are pending, potentially not accurate
traced: the histology track has been recovered from microscopy, however the depths may not match, inacurate data
The attributes
mlapdv
,atlas_ids
andacronyms
in theclusters
andchannels
objects are only available for probe insertions wheresl.histology
is equal to traced, aligned, resolved or alf.The cluster and channel locations in the brain are only considered final for probe insertions with
sl.histology='resolved'
orsl.histology='alf'
.
Useful modules
Exploring spikesorting data
Example 1: Loading different spikesorting versions
[4]:
# By default, if available, the data spikesorted with pykilosort is loaded.
# To find the spikesorting version that is loaded we can use
sl.collection
# To see all available spikesorted data for this probe insertion we can list the collections.
# N.B. ks2.5 matlab spikesorted data is stored in the alf/probe00 folder
sl.collections
# The following can be used to load a specific version of spikesorting
# pykilosort version
spikes, clusters, channels = sl.load_spike_sorting(spike_sorter='pykilosort')
# ks2.5 matlab version
spikes, clusters, channels = sl.load_spike_sorting(spike_sorter='')
/home/runner/Downloads/ONE/openalyx.internationalbrainlab.org/hoferlab/Subjects/SWC_043/2020-09-21/001/alf/probe00/spikes.depths.npy: 100%|██████████| 161M/161M [00:02<00:00, 78.3MB/s]
/home/runner/Downloads/ONE/openalyx.internationalbrainlab.org/hoferlab/Subjects/SWC_043/2020-09-21/001/alf/probe00/spikes.clusters.npy: 100%|██████████| 80.4M/80.4M [00:00<00:00, 124MB/s]
/home/runner/Downloads/ONE/openalyx.internationalbrainlab.org/hoferlab/Subjects/SWC_043/2020-09-21/001/alf/probe00/spikes.amps.npy: 100%|██████████| 161M/161M [00:00<00:00, 176MB/s]
/home/runner/Downloads/ONE/openalyx.internationalbrainlab.org/hoferlab/Subjects/SWC_043/2020-09-21/001/alf/probe00/spikes.times.npy: 100%|██████████| 161M/161M [00:00<00:00, 206MB/s]
/home/runner/Downloads/ONE/openalyx.internationalbrainlab.org/hoferlab/Subjects/SWC_043/2020-09-21/001/alf/probe00/clusters.channels.npy: 100%|██████████| 6.48k/6.48k [00:00<00:00, 52.8kB/s]
/home/runner/Downloads/ONE/openalyx.internationalbrainlab.org/hoferlab/Subjects/SWC_043/2020-09-21/001/alf/probe00/clusters.uuids.csv: 100%|██████████| 29.4k/29.4k [00:00<00:00, 189kB/s]
/home/runner/Downloads/ONE/openalyx.internationalbrainlab.org/hoferlab/Subjects/SWC_043/2020-09-21/001/alf/probe00/clusters.metrics.pqt: 100%|██████████| 97.0k/97.0k [00:00<00:00, 472kB/s]
/home/runner/Downloads/ONE/openalyx.internationalbrainlab.org/hoferlab/Subjects/SWC_043/2020-09-21/001/alf/probe00/clusters.depths.npy: 100%|██████████| 6.48k/6.48k [00:00<00:00, 41.8kB/s]
/home/runner/Downloads/ONE/openalyx.internationalbrainlab.org/hoferlab/Subjects/SWC_043/2020-09-21/001/alf/probe00/channels.rawInd.npy: 100%|██████████| 3.12k/3.12k [00:00<00:00, 18.6kB/s]
/home/runner/Downloads/ONE/openalyx.internationalbrainlab.org/hoferlab/Subjects/SWC_043/2020-09-21/001/alf/probe00/channels.localCoordinates.npy: 100%|██████████| 6.06k/6.06k [00:00<00:00, 38.5kB/s]
/home/runner/Downloads/ONE/openalyx.internationalbrainlab.org/hoferlab/Subjects/SWC_043/2020-09-21/001/alf/probe00/channels.brainLocationIds_ccf_2017.npy: 100%|██████████| 3.12k/3.12k [00:00<00:00, 22.4kB/s]
/home/runner/Downloads/ONE/openalyx.internationalbrainlab.org/hoferlab/Subjects/SWC_043/2020-09-21/001/alf/probe00/channels.mlapdv.npy: 100%|██████████| 4.62k/4.62k [00:00<00:00, 34.8kB/s]
Example 2: Loading additional data
[5]:
# The default spikes and cluster attributes loaded are:
# spikes - amps, clusters, depths, times
# cluster - channels, depths, metrics
#Other attributes can additionally be loaded in the following way
spikes, clusters, channels = sl.load_spike_sorting(dataset_types=['clusters.amps', 'spikes.samples'])
clusters = sl.merge_clusters(spikes, clusters, channels)
/home/runner/Downloads/ONE/openalyx.internationalbrainlab.org/hoferlab/Subjects/SWC_043/2020-09-21/001/alf/probe00/pykilosort/spikes.samples.npy: 100%|██████████| 141M/141M [00:01<00:00, 73.3MB/s]
/home/runner/Downloads/ONE/openalyx.internationalbrainlab.org/hoferlab/Subjects/SWC_043/2020-09-21/001/alf/probe00/pykilosort/#2022-10-31#/clusters.amps.npy: 100%|██████████| 7.44k/7.44k [00:00<00:00, 45.2kB/s]
Example 3: Compute firing rate across session
[6]:
from brainbox.ephys_plots import image_fr_plot
from brainbox.processing import bincount2D
import numpy as np
time_bin = 0.05 # time bin in seconds
depth_bin = 10 # depth bin in um
# Remove any nan values
kp_idx = np.bitwise_and(~np.isnan(spikes['times']), ~np.isnan(spikes['depths']))
fr, time, depth = bincount2D(spikes['times'][kp_idx], spikes['depths'][kp_idx], time_bin, depth_bin)
Example 4: Find clusters labelled as good
[7]:
good_clusterIDs = clusters['cluster_id'][clusters['label'] == 1]
Example 5: Plot a raster for all units
[8]:
sl.raster(spikes, channels)
/home/runner/work/iblenv/ibllib-repo/ibllib/atlas/regions.py:439: RuntimeWarning: invalid value encountered in cast
level=df_regions.depth.to_numpy().astype(np.uint16),
/home/runner/work/iblenv/ibllib-repo/ibllib/atlas/regions.py:441: RuntimeWarning: invalid value encountered in cast
order=df_regions.graph_order.to_numpy().astype(np.uint16))
/home/runner/Downloads/ONE/openalyx.internationalbrainlab.org/hoferlab/Subjects/SWC_043/2020-09-21/001/alf/probe00/pykilosort/drift.times.npy: 100%|██████████| 15.8k/15.8k [00:00<00:00, 119kB/s]
/home/runner/Downloads/ONE/openalyx.internationalbrainlab.org/hoferlab/Subjects/SWC_043/2020-09-21/001/alf/probe00/pykilosort/drift.um.npy: 100%|██████████| 142k/142k [00:00<00:00, 563kB/s]
Out[8]:
(<Figure size 1600x900 with 4 Axes>,
array([[<Axes: title={'center': '2020-09-21_1_SWC_043_probe00, None \n17_575_242 spikes, 914 clusters'}>,
<Axes: >],
[<Axes: xlabel='time (secs)', ylabel='depth (um)'>,
<Axes: title={'center': 'alf'}>]], dtype=object))

Other relevant examples
COMING SOON