ibllib.pipes.training_status

Functions

add_training_lines
check_up_to_date	Check which sessions on local file system are missing from the computed training table.
compute_session_duration_delay_location	Get meta information about task.
compute_training_status	Compute the training status for compute date based on training from that session and two previous days.
find_earliest_recompute_date	Find the earliest date that we need to compute the training status from.
get_data_collection	Return the location of the raw behavioral data and extracted trials data for a given session.
get_latest_training_information	Extracts the latest training status.
get_sess_dict
get_training_info_for_session	Extract the training information needed for plots for each session.
get_training_table_from_aws	If aws credentials exist on the local server download the latest training table from aws s3 private bucket
load_combined_trials	Load and concatenate trials for multiple sessions.
load_existing_dataframe	Load training dataframe from disk, if dataframe doesn't exist returns None.
load_trials	Load trials data for session.
make_plots
pass_through_training_hierachy	Makes sure that the new training status is not less than the one from the previous day.
plot_fit_params
plot_heatmap_performance_over_days
plot_over_days
plot_performance_easy_median_reaction_time
plot_psychometric_curve
plot_trial_count_and_session_duration
save_dataframe	Save training dataframe to disk.
save_path
upload_training_table_to_aws	If aws credentials exist on the local server upload the training table to aws s3 private bucket

get_training_table_from_aws(lab, subject)

If aws credentials exist on the local server download the latest training table from aws s3 private bucket

Parameters:

• lab

• subject

Returns:

upload_training_table_to_aws(lab, subject)

If aws credentials exist on the local server upload the training table to aws s3 private bucket

Parameters:

• lab

• subject

Returns:

save_path(subj_path)

save_dataframe(df, subj_path)

Save training dataframe to disk.

Parameters:

• df – dataframe to save

• subj_path – path to subject folder

Returns:

load_existing_dataframe(subj_path)

Load training dataframe from disk, if dataframe doesn’t exist returns None.

Parameters:

subj_path – path to subject folder

Returns:

load_trials(sess_path, one, collections=None, force=True, mode='raise')

Load trials data for session. First attempts to load from local session path, if this fails will attempt to download via ONE, if this also fails, will then attempt to re-extract locally

Parameters:

• sess_path – session path

• one – ONE instance

• force – when True and if the session trials can’t be found, will attempt to re-extract from the disk

• mode – ‘raise’ or ‘warn’, if ‘raise’, will error when forcing re-extraction of past sessions

Returns:

load_combined_trials(sess_paths, one, force=True)

Load and concatenate trials for multiple sessions. Used when we want to concatenate trials for two sessions on the same day

Parameters:

• sess_paths – list of paths to sessions

• one – ONE instance

Returns:

get_latest_training_information(sess_path, one)

Extracts the latest training status.

Parameters:

• sess_path (pathlib.Path) – The session path from which to load the data.

• one (one.api.One) – An ONE instance.

Returns:

A table of training information.

Return type:

pandas.DataFrame

find_earliest_recompute_date(df)

Find the earliest date that we need to compute the training status from. Training status depends on previous sessions so if a session was missing and now has been added we need to recompute everything from that date onwards

Parameters:

df

Returns:

compute_training_status(df, compute_date, one, force=True)

Compute the training status for compute date based on training from that session and two previous days.

When true and if the session trials can’t be found, will attempt to re-extract from disk. :return:

Parameters:

• df (pandas.DataFrame) – A training data frame, e.g. one generated from get_training_info_for_session().

• compute_date (str, datetime.datetime, pandas.Timestamp) – The date to compute training on.

• one (one.api.One) – An instance of ONE for loading trials data.

• force (bool) – When true and if the session trials can’t be found, will attempt to re-extract from disk.

Returns:

The input data frame with a ‘training_status’ column populated for compute_date.

Return type:

pandas.DataFrame

pass_through_training_hierachy(status_new, status_old)

Makes sure that the new training status is not less than the one from the previous day. e.g Subject cannot regress in performance

Parameters:

• status_new – latest training status

• status_old – previous training status

Returns:

compute_session_duration_delay_location(sess_path, collections=None, **kwargs)

Get meta information about task. Extracts session duration, delay before session start and location of session

Parameters:

• sess_path (pathlib.Path, str) – The session path with the pattern subject/yyyy-mm-dd/nnn.

• collections (list) – The location within the session path directory of task settings and data.

Returns:

• int – The session duration in minutes, rounded to the nearest minute.

• int – The delay between session start time and the first trial in seconds.

• str {‘ephys_rig’, ‘training_rig’} – The location of the session.

get_data_collection(session_path)

Return the location of the raw behavioral data and extracted trials data for a given session.

For multiple locations in one session (e.g. chained protocols), returns all collections. Passive protocols are excluded.

Parameters:

session_path (pathlib.Path) – A session path in the form subject/date/number.

Returns:

• list of str – A list of sub-directory names that contain raw behaviour data.

• list of str – A list of sub-directory names that contain ALF trials data.

Examples

An iblrig v7 session

>>> get_data_collection(Path(r'C:/data/subject/2023-01-01/001'))
['raw_behavior_data'], ['alf']

An iblrig v8 session where two protocols were run

>>> get_data_collection(Path(r'C:/data/subject/2023-01-01/001'))
['raw_task_data_00', 'raw_task_data_01], ['alf/task_00', 'alf/task_01']

get_sess_dict(session_path, one, protocol, alf_collections=None, raw_collections=None, force=True)

get_training_info_for_session(session_paths, one, force=True)

Extract the training information needed for plots for each session.

Parameters:

• session_paths (list of pathlib.Path) – List of session paths on same date.

• one (one.api.One) – An ONE instance.

• force (bool) – When true and if the session trials can’t be found, will attempt to re-extract from disk.

Returns:

A list of dictionaries the length of session_paths containing individual and aggregate performance information.

Return type:

list of dict

check_up_to_date(subj_path, df)

Check which sessions on local file system are missing from the computed training table.

Parameters:

• subj_path (pathlib.Path) – The path to the subject’s dated session folders.

• df (pandas.DataFrame) – The computed training table.

Returns:

A table of dates and session paths that are missing from the computed training table.

Return type:

pandas.DataFrame

plot_trial_count_and_session_duration(df, subject)

plot_performance_easy_median_reaction_time(df, subject)

plot_fit_params(df, subject)

plot_psychometric_curve(df, subject, one)

plot_over_days(df, subject, y1, y2=None, ax=None, legend=True, title=True, training_lines=True)

add_training_lines(df, ax)

plot_heatmap_performance_over_days(df, subject)

make_plots(session_path, one, df=None, save=False, upload=False, task_collection='raw_behavior_data')