aggregation.produce_output_dataframes

aggregation.produce_output_dataframes(snippets_df, input_dir, output_dir=None)

Orchestrate the complete pipeline to produce aggregated and denoised electrophysiological data.

This function serves as the main entry point for processing electrophysiological data from multiple probes. It coordinates the entire pipeline: aggregating channel metadata, processing raw features, applying denoising, and remapping waveform features to the sparse ModelSpikeShapeFeatures schema. The function handles both the data processing and optional file output.

Args: snippets_df (pandas.DataFrame): DataFrame containing snippet information for multiple PIDs. Must have columns: - ‘pid’: Probe ID - ‘base_level_dir’: Base directory path for each snippet - ‘snippet_level_dir’: Relative path to the snippet directory input_dir (Path): Root directory containing probe data. The function searches for ‘channels.pqt’ files in subdirectories of this path. output_dir (Path or None, optional): If provided, all output DataFrames are saved as Parquet files in this directory: - ‘snippets_df.pqt’: Input snippets DataFrame - ‘channels.pqt’: Aggregated channel metadata - ‘raw_ephys_features.pqt’: Aggregated raw features - ‘raw_ephys_features_denoised.pqt’: Denoised features with remapped waveform features The directory is created if it does not exist. Default is None (no files are written).

Returns: tuple: A tuple containing three DataFrames: - df_channels: Aggregated channel metadata with position information - df_raw_ephys: Aggregated raw electrophysiological features - df_features_denoise: Denoised electrophysiological features

Example: >>> import pandas as pd >>> from pathlib import Path >>> snippets_df = pd.DataFrame({ … ‘pid’: [‘probe1’, ‘probe2’], … ‘base_level_dir’: [‘/data/probe1’, ‘/data/probe2’], … ‘snippet_level_dir’: [‘snippet_001’, ‘snippet_001’] … }) >>> channels, raw_features, denoised_features = produce_output_dataframes( … snippets_df, Path(‘/data’), Path(‘output’) … )

Note: - The function searches for ‘channels.pqt’ files recursively in the input directory. - All processing steps are performed sequentially: channel aggregation, feature aggregation, denoising, then waveform spike feature remapping. - If output_dir is provided, all intermediate and final results are saved as Parquet files. - This function serves as a high-level wrapper around the individual processing functions.