compress_to_h5

compress_to_h5(
    cadzow_npy,
    out_h5,
    recording,
    scale=0,
    sglx_meta=None,
    h=None,
    chunk=_COMPRESS_CHUNK,
    overlap=_COMPRESS_OVERLAP,
    epsilon=100.0,
    alpha=7.0,
    fs=250.0,
    t0_sync=None,
    fs_sync=None,
    channels=None,
    n_jobs=4,
    saturation_intervals=None,
    saturation_attrs=None,
    basis_size=_container.BASIS_SIZE,
    processing=None,
)

Compress a decimated (ns, nc) .npy into a format-2 HDF5 archive, chunk by chunk.

Each written chunk of chunk samples is extended by overlap samples on each side before SVD + wavelet-packet compression. Only the central chunk columns of Vh_hat are stored, eliminating wavelet-reconstruction boundary artefacts. Decompressed chunks are concatenated without overlap during reading.

HDF5 layout (format 2)

///meta attrs: nc, ns_total, fs, compress_chunk, compress_overlap, epsilon, alpha, format_version, sglx_meta (JSON), geometry_x, geometry_y, lfpack_version, lfpack_commit, ibl_neuropixel_version, processing (JSON) ///codec/ datasets: chunk_table, u_values, vh_deltas, vh_values, and basis + u_norms with a shared basis (see lfpack._container)

//saturation dataset (n_intervals, 2) int64 of [start_sample, stop_sample] at the raw LFP rate, written once per recording (scale 00 only). attrs: fs, ns_total, n_saturated_samples, saturated_fraction, v_per_sec, proportion, mute_window_samples, muted

where = f’{scale:02d}‘, e.g. ’00’, ‘01’, … Multiple recordings and/or scales can coexist in a single file; merging two files is a plain group copy.

Files are written with the module-level _H5_LIBVER constant (currently ("earliest", "v110")), which pins the HDF5 format to features available since HDF5 1.10 (2017) and makes the output readable by any modern HDF5 installation without version negotiation.

Parameters

Name	Type	Description	Default
cadzow_npy	path - like	Path to the (ns, nc) float32 array at the rate to store (time-first), typically the decimated, optionally Cadzow-denoised checkpoint of compress_bin_to_h5.	required
out_h5	path - like	Output HDF5 file (created or overwritten).	required
recording	str	Unique key for this recording (e.g. a probe-insertion UUID). Top-level HDF5 group name; allows multiple recordings to coexist in one file.	required
scale	int	Resolution level (zero-padded to two digits in the path). 0 = base resolution. Default 0.	0
sglx_meta	dict or None	Original spikeglx metadata (sr.meta). Stored verbatim as JSON.	None
h	dict or None	Probe header. Defaults to NP1 geometry for nc channels.	None
chunk	int	Written chunk size (samples). Default 2048 = 2^11.	_COMPRESS_CHUNK
overlap	int	Guard-band samples each side. Default 128.	_COMPRESS_OVERLAP
epsilon	float	SVD threshold multiplier. Default 100.	100.0
alpha	float	WP threshold multiplier. Default 7.	7.0
fs	float	Sampling rate [Hz] written into metadata. Default 250.	250.0
channels	dict or None	Optional per-channel brain location annotations. Accepted keys (all optional): x (ML, metres), y (AP, metres), z (DV, metres), atlas_id (int32 array), acronym (list of str), labels (int8 array of bad-channel quality flags: 0=good, 1=dead, 2=noisy, 3=outside brain). Matches the dict returned by LFPackReader.channels_full. Written only to the 00 scale; ignored when scale != 0.	None
saturation_intervals	np.ndarray or None	(n_intervals, 2) int64 array of [start_sample, stop_sample] saturated spans at the raw LFP rate. Written once at /<recording>/saturation (scale 00 only). Default None (no saturation table).	None
saturation_attrs	dict or None	Attributes attached to the saturation dataset (fs, ns_total, saturated_fraction, detection parameters, whether muting was applied). Default None.	None
basis_size	int	Number of vectors of the shared spatial basis (one per recording and scale; each chunk stores float16 coefficients in it). Default 32.	_container.BASIS_SIZE
processing	dict or None	Settings of the pre-processing that produced the input (e.g. the Cadzow keyword arguments), stored as JSON in meta.attrs["processing"] next to the software versions (see lfpack._provenance). Default None (empty).	None