aggregation.aggregate_all_probes

aggregation.aggregate_all_probes(
    path_list,
    base_level_dir=None,
    n_jobs=-1,
    verbose=1,
)

Aggregate snippet-level data from multiple probe directories into a single DataFrame.

This function processes a list of probe directory paths in parallel using joblib, calling get_aggregated_snippets_df on each to extract and concatenate their snippet-level data into a unified DataFrame. Optionally, it can annotate the result with a base_level_dir column in case the base_directory is moved.

Args: path_list (List[pathlib.Path]): List of Path objects, each pointing to a probe directory containing snippet-level data to aggregate. Each directory must be compatible with get_aggregated_snippets_df. base_level_dir (Path or str or None, optional): If provided, this value will update the column (‘base_level_dir’) to the resulting DataFrame for all rows.

Returns: pandas.DataFrame: A DataFrame containing the concatenated snippet-level data from all probes in path_list.

Example: >>> from pathlib import Path >>> probe_dirs = [Path(‘/data/probe1’), Path(‘/data/probe2’)] >>> df = aggregate_all_probes(probe_dirs, base_level_dir=‘/data’) >>> print(df.head())

Note: - Each path in path_list should point to a directory structure compatible with get_aggregated_snippets_df. - The function processes all paths in parallel using all available CPU cores (n_jobs=-1). - The function ignores index continuity and resets the index in the returned DataFrame. - If base_level_dir is provided, it is stored as a string in the ‘base_level_dir’ column for all rows. - This function does not perform validation on the contents of the aggregated DataFrames beyond concatenation.