ibl_alignment_gui.loaders.data_loader

Classes

`DataLoader`	Abstract base class for loading processed data for the ibl_alignment_gui.
`DataLoaderLocal`	Data loader using local file system.
`DataLoaderOne`	Data loader using ONE.
`FeatureLoader`	Abstract base class for loading ephys atlas features.
`FeatureLoaderLocal`	Feature loader using the local file system.
`FeatureLoaderOne`	Feature loader using ONE.
`SpikeGLXLoader`	Abstract base class for loading SpikeGLX metadata and AP band snippets.
`SpikeGLXLoaderLocal`	SpikeGLX loader using local file system.
`SpikeGLXLoaderOne`	SpikeGLX loader using ONE.

class ibl_alignment_gui.loaders.data_loader.DataLoader[source] 

Bases: ABC

Abstract base class for loading processed data for the ibl_alignment_gui.

Loads spikesorted and raw ephys data.

Subclasses must implement the follow abstract methods: - load_passive_data - load_raw_passive_data - load_ephys_data - load_spikes_data

check_channel_order: bool = False

filter_raw_by_chns(data)[source] 

Filter ephys data to only include data relevant to channels present on selected shank.

The columns are taken to hold the raw channels in file order, unless the data give the raw channel of each column as rawInd.

Parameters:: data (dict) – Raw ephys data
Returns:: data – Filtered ephys data
Return type:: Bunch

filter_spikes_by_chns(spikes, clusters, channels)[source] 

Filter spikes to only include data relevant to channels present on selected shank.

Return type:

tuple[Bunch[str, Any], Bunch[str, Any], Bunch[str, Any]]

Returns:

spikes (Bunch) – Filtered spikes data
clusters (Bunch) – Filtered clusters data
channels (Bunch) – Filtered channels data

static filter_spikes_by_fr(spikes, clusters, min_fr=0.013888888888888888)[source] 

Remove low-firing clusters and filter spikes accordingly.

Parameters:

spikes (Bunch) – Spike data.
clusters (Bunch) – Cluster data.
min_fr (float) – Minimum firing rate in Hz.

Return type:

tuple[Bunch[str, Any], Bunch[str, Any]]

Returns:

spikes (Bunch) – Spikes data above fr threshold
clusters (Bunch) – Clusters data above fr threshold

get_data(shank_sites)[source] 

Load all relevant data associated with the probe.

Parameters:: shank_sites (Bunch) – A Bunch object containing the channels that correspond to the shank
Returns:: data – A Bunch object containing spikes, clusters, channels, RMS and PSD data, and passive stimulus data.
Return type:: Bunch

get_passive_data()[source] 

Load passive visual stimulus data including RF map, visual stimuli, and gabor events.

Return type:

tuple[Bunch[str, Any], Bunch[str, Any], Bunch[str, Any]]

Returns:

rf_data (Bunch) – RF map data including frames.
stim_data (Bunch) – Visual stimulus data.
vis_stim (Bunch) – Gabor event data for left and right gabors.

get_psd_data(alf_object)[source] 

Load power spectral density data for specified band.

Only returns data on channels present on selected shank.

Parameters:: alf_object (str) – The alf object to load
Returns:: psd_data – PSD data
Return type:: Bunch

get_rms_data(alf_object)[source] 

Load RMS data for specified band.

Only returns data on channels present on selected shank.

Parameters:: alf_object (str) – The alf object to load
Returns:: rms_data – RMS data
Return type:: Bunch

get_spikes_data()[source] 

Load spike sorting data and optionally filter by minimum firing rate threshold.

Only returns data on channels present on selected shank.

Return type:

tuple[Bunch[str, Any], Bunch[str, Any], Bunch[str, Any]]

Returns:

spikes (Bunch) – spikes data
clusters (Bunch) – clusters data
channels (Bunch) – channels data

static load_data(load_function, *args, raise_message=None, raise_exception=(<class 'one.alf.exceptions.ALFObjectNotFound'>, <class 'one.alf.exceptions.ALFMultipleCollectionsFound'>), raise_error=False, **kwargs)[source] 

Safely load data using a provided function.

Parameters:

load_function (Callable) – Function to load the data (e.g., ONE or alfio).
args (tuple) – Arguments for the loading function.
raise_message (str or None) – Message to log if an exception is raised.
raise_exception (Exception or ALFObjectNotFound) – Type of exception to catch.
raise_error (bool) – Whether to raise the exception after logging.

Returns:

The loaded data with an ‘exists’ flag.

Return type:

Bunch

abstractmethod load_ephys_data(alf_object, **kwargs)[source] 

Abstract method to load ephys data.

Return type:: Bunch[str, Any]

abstractmethod load_passive_data(alf_object, **kwargs)[source] 

Abstract method to load passive data.

Return type:: Bunch[str, Any]

abstractmethod load_raw_passive_data(alf_object, **kwargs)[source] 

Abstract method to load raw passive data.

Return type:: Bunch[str, Any]

abstractmethod load_spikes_data(alf_object, attributes, **kwargs)[source] 

Abstract method to load spike sorting data.

Return type:: Bunch[str, Any]

note_channel_order(data, alf_object)[source] 

Note RMS or spectra data that do not record the order of their channels (rawInd).

Parameters:

data (Bunch) – The RMS or spectra data.
alf_object (str) – The ALF object the data were loaded from.

Return type:

None

class ibl_alignment_gui.loaders.data_loader.DataLoaderLocal(data_paths)[source] 

Bases: DataLoader

Data loader using local file system.

Data are loaded from files on disk, from the dataset paths resolved from the session yaml.

Parameters:: data_paths (DatasetPaths) – The resolved dataset paths for the probe, holding the spike sorting, processed ephys, task, raw task and raw ephys directories.

get_passive_data()[source] 

Load passive data from local path.

Only attempts to load if both task_path and raw_task_path are defined.

Return type:: Bunch or None

load_ephys_data(alf_object, **kwargs)[source] 

Load ephys data from local path.

Return type:: Bunch

load_passive_data(alf_object, **kwargs)[source] 

Load passive data from local path.

Return type:: Bunch

load_raw_passive_data(alf_object, **kwargs)[source] 

Load raw passive data from local path.

Return type:: Bunch or None

load_spikes_data(alf_object, attributes, **kwargs)[source] 

Load spike sorting data from local path.

Returns:: The data, with exists False when the session has no spike sorting.
Return type:: Bunch

property spike_sorting_path: Path 

Return the path to the folder holding the spike sorting data.

Returns:: The path to the spike sorting folder.
Return type:: Path

class ibl_alignment_gui.loaders.data_loader.DataLoaderOne(insertion, one, session_path=None, spike_collection=None)[source] 

Bases: DataLoader

Data loader using ONE.

Data are downloaded using Alyx/ ONE.

Parameters:

insertion (dict) – Dictionary representing a probe insertion (must include ‘session’ and ‘name’).
one (ONE) – An ONE instance used to access data.
session_path (Path or None) – Path to the session folder. If None, it is resolved using the eid via one.eid2path.
spike_collection (str or None) – Spike sorting algorithm to load (e.g. ‘pykilosort’, ‘iblsorter’).

get_spike_sorting_path()[source] 

Determine the path to the spike sorting output.

Returns:: probe_path – A Path to the spike sorting folder for the probe.
Return type:: Path

load_ephys_data(alf_object, **kwargs)[source] 

Load ephys data using ONE.

Return type:: Bunch

load_passive_data(alf_object, **kwargs)[source] 

Load passive data using ONE.

Return type:: Bunch

load_raw_passive_data(alf_object, **kwargs)[source] 

Load raw passive data using ONE.

Return type:: Bunch

load_spikes_data(alf_object, attributes, **kwargs)[source] 

Load spike sorting data using ONE.

Return type:: Bunch

property spike_sorting_path: Path 

Return the path to the folder holding the spike sorting data.

Returns:: The path to the spike sorting folder.
Return type:: Path

class ibl_alignment_gui.loaders.data_loader.FeatureLoader[source] 

Bases: ABC

Abstract base class for loading ephys atlas features.

abstractmethod load_features(shank_sites=None)[source] 

Abstract method to load ephys atlas features.

Parameters:: shank_sites (Bunch or None) – The sites for the shank being loaded (see GeometryLoader.get_sites_for_shank()). When given, implementations that read a combined multi-shank file use it to keep only this shank’s channels.
Return type:: Bunch[str, Any]

class ibl_alignment_gui.loaders.data_loader.FeatureLoaderLocal(features_path)[source] 

Bases: FeatureLoader

Feature loader using the local file system.

Mirrors FeatureLoaderOne but reads a single per-channel feature parquet from disk instead of downloading the ephys-atlas feature table via ONE. Used in offline/yaml mode where the features file is supplied directly (e.g. by the local channel-prediction plugin dialog or the launcher script).

Parameters:: features_path (Path) – Path to a per-channel features parquet file (one row per channel).

load_features(shank_sites=None)[source] 

Load the per-channel features parquet from disk.

A single features file may hold every channel of a multi-shank probe. When shank_sites is given, the rows are split down to this shank by matching the file’s channel column against shank_sites['raw_ind'] (the raw-data channel indices for the shank). For a single-shank file (or a file already restricted to this shank) this is a no-op.

Parameters:: shank_sites (Bunch or None) – The sites for the shank being loaded (see GeometryLoader.get_sites_for_shank()). When None the full file is returned.
Returns:: feature_data – A Bunch with df (the features DataFrame) and exists=True if the file was found and non-empty, otherwise Bunch(exists=False).
Return type:: Bunch

class ibl_alignment_gui.loaders.data_loader.FeatureLoaderOne(insertion, one, feature_version, multi_area=False)[source] 

Bases: FeatureLoader

Feature loader using ONE.

Loads feature from ephys feature table.

download_features(project, feature, save_path)[source] 

Download the latest ephys atlas features from S3.

Parameters:

project (str) – The project name
feature (str) – The feature version
save_path (Path) – A path to save the downloaded file

Return type:

None

load_dataframe(project, feature, save_path)[source] 

Load the ephys-atlas feature table filtered to this insertion’s pid.

Parameters:

project (str) – The project name.
feature (str) – The feature version label.
save_path (Path) – Directory under which the <project>/<feature> tables are stored.

Returns:

The feature table rows for this insertion’s pid.

Return type:

pd.DataFrame

load_features(shank_sites=None)[source] 

Load ephys atlas features from ONE.

Parameters:: shank_sites (Bunch or None) – Unused here — the ONE table is already filtered to this insertion’s pid. Accepted to keep the FeatureLoader interface uniform across loaders.
Returns:: feature_data – A Bunch containing a dataFrame containing ephys atlas features for the probe.
Return type:: Bunch

read_full_dataframe(project, feature, save_path)[source] 

Read the full ephys-atlas feature table for all insertions, downloading if needed.

Parameters:

project (str) – The project name.
feature (str) – The feature version label.
save_path (Path) – Directory under which the <project>/<feature> tables are stored.

Returns:

The full feature table across all insertions, or an empty frame if the table is not cached and cannot be downloaded (ephysatlas is not installed, or the release has no such table).

Return type:

pd.DataFrame

class ibl_alignment_gui.loaders.data_loader.SpikeGLXLoader(save_path=None)[source] 

Bases: ABC

Abstract base class for loading SpikeGLX metadata and AP band snippets.

Subclasses must implement the follow abstract methods: - load_meta_data - load_ap_data

Parameters:: save_path (Path or None) – Directory where cached snippet data will be saved.

get_meta_data()[source] 

Load and parse metadata for the AP band.

Returns:: A Bunch containing geometry info and an ‘exists’ flag.
Return type:: Bunch

static get_time_snippets(sr, n=3, pad=200, twin=0)[source] 

Return n time points across the file duration, excluding pad seconds from start and end.

Parameters:

sr (spikeglx.Reader or Streamer) – The raw data reader
n (int) – Number of time points to extract.
pad (int) – Time (seconds) to exclude from beginning and end.
twin (float) – The length (seconds) of the snippet read from each time point. The time points are kept early enough for the snippet to end inside the file.

Returns:

Array of time points in seconds.

Return type:

np.ndarray

abstractmethod load_ap_data()[source] 

Abstract method to return a SpikeGLX reader or Streamer object.

Return type:: spikeglx.Reader | Streamer | None

load_ap_snippets(twin=1)[source] 

Load AP snippets centered around selected time points.

Also computes channel quality metrics across snippets to detect dead, noisy and outside channels.

Parameters:: twin (float) – Time window in seconds for each snippet.
Returns:: data – Snippets of raw data for three timepoints in addition to metadata (exists, fs). The channels of the snippets and the channel quality metrics are in file order.
Return type:: Bunch

abstractmethod load_lf_data()[source] 

Abstract method to return a SpikeGLX reader or Streamer object.

Return type:: spikeglx.Reader | Streamer | None

load_lf_snippets(twin=5)[source] 

Load LF snippets centered around selected time points.

Also computes channel quality metrics across snippets to detect dead, noisy and outside channels.

Parameters:: twin (float) – Time window in seconds for each snippet.
Returns:: data – Snippets of raw data for three timepoints in addition to metadata (exists, fs). The channels of the snippets are in file order.
Return type:: Bunch

abstractmethod load_meta_data()[source] 

Abstract method to load AP metadata.

Return type:: dict | None

class ibl_alignment_gui.loaders.data_loader.SpikeGLXLoaderLocal(meta_path)[source] 

Bases: SpikeGLXLoader

SpikeGLX loader using local file system.

Parameters:

probe_path (Path) – Root directory of probe data
meta_collection (str) – Name of subfolder containing meta and binary files.

load_ap_data()[source] 

Load binary AP data from local path.

Returns:: A spikeglx.Reader instance to load the raw data, or None if not found.
Return type:: spikeglx.Reader or None

load_lf_data()[source] 

Load binary LF data from local path.

Returns:: A spikeglx.Reader instance to load the raw data, or None if not found.
Return type:: spikeglx.Reader or None

load_meta_data()[source] 

Load AP metadata from local path.

Returns:: A dict containing the spikeglx AP band metadata, or None if not found.
Return type:: dict or None

class ibl_alignment_gui.loaders.data_loader.SpikeGLXLoaderOne(insertion, one, session_path=None, force=False)[source] 

Bases: SpikeGLXLoader

SpikeGLX loader using ONE.

Raw data is streamed via ONE

Parameters:

insertion (dict) – Dictionary representing a probe insertion (must include ‘session’ and ‘name’).
one (ONE) – An ONE instance used to access data.
session_path (Path or None) – Path to the session folder. If None, it is resolved using the eid via one.eid2path.
force (bool) – If True, forces removal of cached data before streaming.

load_ap_data()[source] 

Load AP data using ONE.

Returns:: A streamer object for AP band.
Return type:: Streamer

load_lf_data()[source] 

Load LF data using ONE.

Returns:: A streamer object for LF band.
Return type:: Streamer

load_meta_data()[source] 

Load AP metadata using ONE.

Returns:: A dict containing the spikeglx AP band metadata, or None if not found.
Return type:: dict or None