brainbox.behavior.training
Computing and testing IBL training status criteria.
For an indepth description of each training status, see Appendix 2 of the IBL Protocol For Mice Training.
Examples
Plot the psychometric curve for a given session.
>>> trials = ONE().load_object(eid, 'trials')
>>> fix, ax = plot_psychometric(trials)
Compute ‘response times’, defined as the duration of openloop for each contrast.
>>> reaction_time, contrasts, n_contrasts = compute_reaction_time(trials)
Compute ‘reaction times’, defined as the time between go cue and first detected movement. NB: These may be negative!
>>> reaction_time, contrasts, n_contrasts = compute_reaction_time(
... trials, stim_on_type='goCue_times', stim_off_type='firstMovement_times')
Compute ‘response times’, defined as the time between first detected movement and response.
>>> reaction_time, contrasts, n_contrasts = compute_reaction_time(
... trials, stim_on_type='firstMovement_times', stim_off_type='response_times')
Compute ‘movement times’, defined as the time between last detected movement and response threshold.
>>> import brainbox.behavior.wheel as wh
>>> wheel_moves = ONE().load_object(eid, 'wheeMoves')
>>> trials['lastMovement_times'] = wh.get_movement_onset(wheel_moves.intervals, trial_data.response_times)
>>> reaction_time, contrasts, n_contrasts = compute_reaction_time(
... trials, stim_on_type='lastMovement_times', stim_off_type='response_times')
Module attributes
The required keys in the trials object for computing training status. 
Functions
Compute all relevant performance metrics for when subject is on biasedChoiceWorld 

Compute median response time on zero contrast trials from trials object 

Compute number of trials in trials object 

Compute performance on all trials at each contrast level from trials object 

Compute performance on easy trials (stimulus >= 50 %) from trials object 

Compute psychometric fit parameters for trials object. 

Compute median response time for all contrasts. 

Compute all relevant performance metrics for when subject is on trainingChoiceWorld. 

Concatenate trials from different training sessions. 

Returns bool indicating whether criteria for status 'trained_1a' are met. 

Returns bool indicating whether criteria for trained_1b are met. 

Returns bool indicating whether criteria for 'ready4delay' is met. 

Returns bool indicating whether criteria for ready4ephysrig or ready4recording are met. 

Display training status of subject to terminal. 

Computes the training status of all alive and water restricted subjects in a specified lab. 

Download and load in training data for a specified subject. 

Compute signed contrast from trials object 

Computes the training status of specified subject and prints results to std out. 

Compute training status of a subject from consecutive training datasets. 

Function to plot psychometric curve plots a la datajoint webpage. 

Function to plot reaction time against contrast a la datajoint webpage. 

Function to plot reaction time with trial number a la datajoint webpage. 

Get the session for which a given training criterion was met. 
Classes
Standard IBL training criteria. 
 TRIALS_KEYS = ['contrastLeft', 'contrastRight', 'feedbackType', 'probabilityLeft', 'choice', 'response_times', 'stimOn_times']
The required keys in the trials object for computing training status.
 Type:
list of str
 class TrainingStatus(value)[source]
Bases:
IntFlag
Standard IBL training criteria.
Enumeration allows for comparisons between training status.
Examples
>>> status = 'ready4delay' ... assert TrainingStatus[status.upper()] is TrainingStatus.READY4DELAY ... assert TrainingStatus[status.upper()] not in TrainingStatus.FAILED, 'Subject failed training' ... assert TrainingStatus[status.upper()] >= TrainingStatus.TRAINED, 'Subject untrained' ... assert TrainingStatus[status.upper()] > TrainingStatus.IN_TRAINING, 'Subject untrained' ... assert TrainingStatus[status.upper()] in ~TrainingStatus.FAILED, 'Subject untrained' ... assert TrainingStatus[status.upper()] in TrainingStatus.TRAINED ^ TrainingStatus.READY
Get the next training status
>>> next(member for member in sorted(TrainingStatus) if member > TrainingStatus[status.upper()]) <TrainingStatus.READY4RECORDING: 128>
Notes
~TrainingStatus.TRAINED means any status but trained 1a or trained 1b.
A subject may acheive both TRAINED_1A and TRAINED_1B within a single session, therefore it is possible to have skipped the TRAINED_1A session status.
 UNTRAINABLE = 1
 UNBIASABLE = 2
 IN_TRAINING = 4
 TRAINED_1A = 8
 TRAINED_1B = 16
 READY4EPHYSRIG = 32
 READY4DELAY = 64
 READY4RECORDING = 128
 FAILED = 3
 READY = 224
 TRAINED = 24
 get_lab_training_status(lab, date=None, details=True, one=None)[source]
Computes the training status of all alive and water restricted subjects in a specified lab.
The response are printed to std out.
 Parameters:
lab (str) – Lab name (must match the name registered on Alyx).
date (str) – The ISO date from which to compute training status. If not specified will compute from the latest date with available data. Format should be ‘YYYYMMDD’.
details (bool) – Whether to display all information about training status computation e.g. performance, number of trials, psychometric fit parameters.
one (one.api.OneAlyx) – An instance of ONE.
 get_subject_training_status(subj, date=None, details=True, one=None)[source]
Computes the training status of specified subject and prints results to std out.
 Parameters:
subj (str) – Subject nickname (must match the name registered on Alyx).
date (str) – The ISO date from which to compute training status. If not specified will compute from the latest date with available data. Format should be ‘YYYYMMDD’.
details (bool) – Whether to display all information about training status computation e.g. performance, number of trials, psychometric fit parameters.
one (one.api.OneAlyx) – An instance of ONE.
 get_sessions(subj, date=None, one=None)[source]
Download and load in training data for a specified subject. If a date is given it will load data from the three (or as many as are available) previous sessions up to the specified date. If not it will load data from the last three training sessions that have data available.
 Parameters:
subj (str) – Subject nickname (must match the name registered on Alyx).
date (str) – The ISO date from which to compute training status. If not specified will compute from the latest date with available data. Format should be ‘YYYYMMDD’.
one (one.api.OneAlyx) – An instance of ONE.
 Returns:
iblutil.util.Bunch – Dictionary of trials objects where each key is the ISO session date string.
list of str – List of the task protocol used for each of the sessions.
list of str – List of ISO date strings where training was conducted on ephys rig. Empty list if all sessions on training rig.
n_delay (int) – Number of sessions on ephys rig that had delay prior to starting session > 15min. Returns 0 if no sessions detected.
 get_training_status(trials, task_protocol, ephys_sess_dates, n_delay)[source]
Compute training status of a subject from consecutive training datasets.
For IBL, training status is calculated using trials from the last three consecutive sessions.
 Parameters:
trials (dict of str) – Dictionary of trials objects where each key is the ISO session date string.
task_protocol (list of str) – Task protocol used for each training session in trials, can be ‘training’, ‘biased’ or ‘ephys’.
ephys_sess_dates (list of str) – List of ISO date strings where training was conducted on ephys rig. Empty list if all sessions on training rig.
n_delay (int) – Number of sessions on ephys rig that had delay prior to starting session > 15min. Returns 0 if no sessions detected.
 Returns:
str – Training status of the subject.
iblutil.util.Bunch – Bunch containing performance metrics that decide training status i.e. performance on easy trials, number of trials, psychometric fit parameters, reaction time.
 display_status(subj, sess_dates, status, perf_easy=None, n_trials=None, psych=None, psych_20=None, psych_80=None, rt=None)[source]
Display training status of subject to terminal.
 Parameters:
subj (str) – Subject nickname (must match the name registered on Alyx).
sess_dates (list of str) – ISO date strings of training sessions used to determine training status.
status (str) – Training status of subject.
perf_easy (numpy.array) – Proportion of correct high contrast trials for each training session.
n_trials (numpy.array) – Total number of trials for each training session.
psych (numpy.array) – Psychometric parameters fit to data from all training sessions  bias, threshold, lapse high, lapse low.
psych_20 (numpy.array) – The fit psychometric parameters for the blocks where probability of a left stimulus is 0.2.
psych_80 (numpy.array) – The fit psychometric parameters for the blocks where probability of a left stimulus is 0.8.
rt (float) – The median response time for zero contrast trials across all training sessions. NaN indicates no zero contrast stimuli in training sessions.
 concatenate_trials(trials)[source]
Concatenate trials from different training sessions.
 Parameters:
trials (dict of str) – Dictionary of trials objects where each key is the ISO session date string.
 Returns:
Trials object with data concatenated over three training sessions.
 Return type:
 compute_training_info(trials, trials_all)[source]
Compute all relevant performance metrics for when subject is on trainingChoiceWorld.
 Parameters:
trials (dict of str) – Dictionary of trials objects where each key is the ISO session date string.
trials_all (one.alf.io.AlfBunch) – Trials object with data concatenated over three training sessions.
 Returns:
numpy.array – Proportion of correct high contrast trials for each session.
numpy.array – Total number of trials for each training session.
numpy.array – Array of psychometric parameters fit to all_trials  bias, threshold, lapse high, lapse low.
float – The median response time for all zerocontrast trials across all sessions. Returns NaN if no trials zerocontrast trials).
 compute_bias_info(trials, trials_all)[source]
Compute all relevant performance metrics for when subject is on biasedChoiceWorld
 Parameters:
trials – dict containing trials objects from three consecutive training sessions,
keys are session dates :type trials: Bunch :param trials_all: trials object with data concatenated over three training sessions :type trials_all: Bunch :returns:
perf_easy  performance of easy trials for each session
n_trials  number of trials in each session
psych_20  parameters for psychometric curve fit to trials in 20 block over all sessions
psych_80  parameters for psychometric curve fit to trials in 80 block over all sessions
rt  median reaction time for zero contrast stimuli over all sessions
 get_signed_contrast(trials)[source]
Compute signed contrast from trials object
 Parameters:
trials (dict) – trials object that must contain contrastLeft and contrastRight keys
returns: array of signed contrasts in percent, where ve values are on the left
 compute_performance_easy(trials)[source]
Compute performance on easy trials (stimulus >= 50 %) from trials object
 Parameters:
trials – trials object that must contain contrastLeft, contrastRight and feedbackType
keys :type trials: dict returns: float containing performance on easy contrast trials
 compute_performance(trials, signed_contrast=None, block=None, prob_right=False)[source]
Compute performance on all trials at each contrast level from trials object
 Parameters:
trials – trials object that must contain contrastLeft, contrastRight and feedbackType
keys :type trials: dict returns: float containing performance on easy contrast trials
 compute_n_trials(trials)[source]
Compute number of trials in trials object
 Parameters:
trials (dict) – trials object
returns: int containing number of trials in session
 compute_psychometric(trials, signed_contrast=None, block=None, plotting=False, compute_ci=False, alpha=0.032)[source]
Compute psychometric fit parameters for trials object.
 Parameters:
trials (one.alf.io.AlfBunch) – An ALF trials object containing the keys {‘probabilityLeft’, ‘contrastLeft’, ‘contrastRight’, ‘feedbackType’, ‘choice’, ‘response_times’, ‘stimOn_times’}.
signed_contrast (numpy.array) – An array of signed contrasts in percent the length of trials, where left contrasts are ve. If None, these are computed from the trials object.
block (float) – The block type to compute. If None, all trials are included, otherwise only trials where probabilityLeft matches this value are included. For biasedChoiceWorld, the probabilityLeft set is {0.5, 0.2, 0.8}.
plotting (bool) – Which set of psychofit model parameters to use (see notes).
compute_ci (bool) – If true, computes and returns the confidence intervals for response at each contrast.
alpha (float, default=0.032) – Significance level for confidence interval. Must be in (0, 1). If compute_ci is false, this value is ignored.
 Returns:
numpy.array – Array of psychometric fit parameters  bias, threshold, lapse high, lapse low.
(tuple of numpy.array) – If compute_ci is true, a tuple of
See also
statsmodels.stats.proportion.proportion_confint
interval.
psychofit.mle_fit_psycho
Notes
The psychofit starting parameters and model constraints used for the fit when computing the training status (e.g. trained_1a, etc.) are suboptimal and can produce a poor fit. To keep the precise criteria the same for all subjects, these parameters have not changed. To produce a better fit for plotting purposes, or to calculate the training status in a manner inconsistent with the IBL training pipeline, use plotting=True.
 compute_median_reaction_time(trials, stim_on_type='stimOn_times', contrast=None, signed_contrast=None)[source]
Compute median response time on zero contrast trials from trials object
 Parameters:
trials (one.alf.io.AlfBunch) – An ALF trials object containing the keys {‘probabilityLeft’, ‘contrastLeft’, ‘contrastRight’, ‘feedbackType’, ‘choice’, ‘response_times’, ‘stimOn_times’}.
stim_on_type (str, default='stimOn_times') – The trials key to use when calculating the response times. The difference between this and ‘feedback_times’ is used (see notes).
contrast (float) – If None, the median response time is calculated for all trials, regardless of contrast, otherwise only trials where the matching signed percent contrast was presented are used.
signed_contrast (numpy.array) – An array of signed contrasts in percent the length of trials, where left contrasts are ve. If None, these are computed from the trials object.
 Returns:
The median response time for trials with contrast (returns NaN if no trials matching contrast in trials object).
 Return type:
float
Notes
The stim_on_type is ‘stimOn_times’ by default, however for IBL rig data, the photodiode is sometimes not calibrated properly which can lead to inaccurate (or absent, i.e. NaN) stim on times. Therefore, it is sometimes more accurate to use the ‘stimOnTrigger_times’ (the time of the stimulus onset command), if available, or the ‘goCue_times’ (the time of the soundcard output TTL when the audio go cue is played) or the ‘goCueTrigger_times’ (the time of the audio go cue command).
The response/reaction time here is defined as the time between stim on and feedback, i.e. the entire openloop trial duration.
 compute_reaction_time(trials, stim_on_type='stimOn_times', stim_off_type='response_times', signed_contrast=None, block=None, compute_ci=False, alpha=0.32)[source]
Compute median response time for all contrasts.
 Parameters:
trials (one.alf.io.AlfBunch) – An ALF trials object containing the keys {‘probabilityLeft’, ‘contrastLeft’, ‘contrastRight’, ‘feedbackType’, ‘choice’, ‘response_times’, ‘stimOn_times’}.
stim_on_type (str, default='stimOn_times') – The trials key to use when calculating the response times. The difference between this and stim_off_type is used (see notes).
stim_off_type (str, default='response_times') – The trials key to use when calculating the response times. The difference between this and stim_on_type is used (see notes).
signed_contrast (numpy.array) – An array of signed contrasts in percent the length of trials, where left contrasts are ve. If None, these are computed from the trials object.
block (float) – The block type to compute. If None, all trials are included, otherwise only trials where probabilityLeft matches this value are included. For biasedChoiceWorld, the probabilityLeft set is {0.5, 0.2, 0.8}.
compute_ci (bool) – If true, computes and returns the confidence intervals for response time at each contrast.
alpha (float, default=0.32) – Significance level for confidence interval. Must be in (0, 1). If compute_ci is false, this value is ignored.
 Returns:
numpy.array – The median response times for each unique signed contrast.
numpy.array – The set of unique signed contrasts.
numpy.array – The number of trials for each unique signed contrast.
(numpy.array) – If compute_ci is true, an array of confidence intervals is return in the shape (n_trials, 2).
Notes
The response/reaction time by default is the time between stim on and response, i.e. the entire openloop trial duration. One could use ‘stimOn_times’ and ‘firstMovement_times’ to get the true reaction time, or ‘firstMovement_times’ and ‘response_times’ to get the true response times, or calculate the last movement onset times and calculate the true movement times. See module examples for how to calculate this.
See also
scipy.stats.bootstrap
 criterion_1a(psych, n_trials, perf_easy)[source]
Returns bool indicating whether criteria for status ‘trained_1a’ are met.
Criteria
Bias is less than 16
Threshold is less than 19
Lapse rate on both sides is less than 0.2
The total number of trials is greater than 200 for each session
Performance on easy contrasts > 80% for all sessions
 param psych:
The fit psychometric parameters three consecutive sessions. Parameters are bias, threshold, lapse high, lapse low.
 type psych:
numpy.array
 param n_trials:
The number for trials for each session.
 type n_trials:
numpy.array of int
 param perf_easy:
The proportion of correct high contrast trials for each session.
 type perf_easy:
numpy.array of float
 returns:
True if the criteria are met for ‘trained_1a’.
 rtype:
bool
Notes
The parameter thresholds chosen here were originally determined by averaging the parameter fits for a number of sessions determined to be of ‘good’ performance by an experimenter.
 criterion_1b(psych, n_trials, perf_easy, rt)[source]
Returns bool indicating whether criteria for trained_1b are met.
Criteria
Bias is less than 10
Threshold is less than 20 (see notes)
Lapse rate on both sides is less than 0.1
The total number of trials is greater than 400 for each session
Performance on easy contrasts > 90% for all sessions
The median response time across all zero contrast trials is less than 2 seconds
 param psych:
The fit psychometric parameters three consecutive sessions. Parameters are bias, threshold, lapse high, lapse low.
 type psych:
numpy.array
 param n_trials:
The number for trials for each session.
 type n_trials:
numpy.array of int
 param perf_easy:
The proportion of correct high contrast trials for each session.
 type perf_easy:
numpy.array of float
 param rt:
The median response time for zero contrast trials.
 type rt:
float
 returns:
True if the criteria are met for ‘trained_1b’.
 rtype:
bool
Notes
The parameter thresholds chosen here were originally chosen to be slightly stricter than 1a, however it was decided to use round numbers so that readers would not assume a level of precision that isn’t there (remember, these parameters were not chosen with any rigor). This regrettably means that the maximum threshold fit for 1b is greater than for 1a, meaning the slope of the psychometric curve may be slightly less steep than 1a.
 criterion_ephys(psych_20, psych_80, n_trials, perf_easy, rt)[source]
Returns bool indicating whether criteria for ready4ephysrig or ready4recording are met.
NB: The difference between these two is whether the sessions were acquired ot a recording rig with a delay before the first trial. Neither of these two things are tested here.
Criteria
Lapse on both sides < 0.1 for both bias blocks
Bias shift between blocks > 5
Total number of trials > 400 for all sessions
Performance on easy contrasts > 90% for all sessions
Median response time for zero contrast stimuli < 2 seconds
 param psych_20:
The fit psychometric parameters for the blocks where probability of a left stimulus is 0.2. Parameters are bias, threshold, lapse high, lapse low.
 type psych_20:
numpy.array
 param psych_80:
The fit psychometric parameters for the blocks where probability of a left stimulus is 0.8. Parameters are bias, threshold, lapse high, lapse low.
 type psych_80:
numpy.array
 param n_trials:
The number of trials for each session (typically three consecutive sessions).
 type n_trials:
numpy.array
 param perf_easy:
The proportion of correct high contrast trials for each session (typically three consecutive sessions).
 type perf_easy:
numpy.array
 param rt:
The median response time for zero contrast trials.
 type rt:
float
 returns:
True if subject passes the ready4ephysrig or ready4recording criteria.
 rtype:
bool
 criterion_delay(n_trials, perf_easy)[source]
Returns bool indicating whether criteria for ‘ready4delay’ is met.
Criteria
Total number of trials for any of the sessions is greater than 400
Performance on easy contrasts is greater than 90% for any of the sessions
 param n_trials:
The number of trials for each session (typically three consecutive sessions).
 type n_trials:
numpy.array of int
 param perf_easy:
The proportion of correct high contrast trials for each session (typically three consecutive sessions).
 type perf_easy:
numpy.array
 returns:
True if subject passes the ‘ready4delay’ criteria.
 rtype:
bool
 plot_psychometric(trials, ax=None, title=None, plot_ci=False, ci_alpha=0.032, **kwargs)[source]
Function to plot psychometric curve plots a la datajoint webpage.
 Parameters:
trials (one.alf.io.AlfBunch) – An ALF trials object containing the keys {‘probabilityLeft’, ‘contrastLeft’, ‘contrastRight’, ‘feedbackType’, ‘choice’, ‘response_times’, ‘stimOn_times’}.
ax (matplotlib.pyplot.Axes) – An axis object to plot onto.
title (str) – An optional plot title.
plot_ci (bool) – If true, computes and plots the confidence intervals for response at each contrast.
ci_alpha (float, default=0.032) – Significance level for confidence interval. Must be in (0, 1). If plot_ci is false, this value is ignored.
**kwargs – If ax is None, these arguments are passed to matplotlib.pyplot.subplots.
 Returns:
matplotlib.pyplot.Figure – The figure handle containing the plot.
matplotlib.pyplot.Axes – The plotted axes.
See also
statsmodels.stats.proportion.proportion_confint
interval.
psychofit.mle_fit_psycho
psychofit.erf_psycho_2gammas
using the fit parameters.
 plot_reaction_time(trials, ax=None, title=None, plot_ci=False, ci_alpha=0.32, **kwargs)[source]
Function to plot reaction time against contrast a la datajoint webpage.
The reaction times are plotted individually for the following three blocks: {0.5, 0.2, 0.8}.
 Parameters:
trials (one.alf.io.AlfBunch) – An ALF trials object containing the keys {‘probabilityLeft’, ‘contrastLeft’, ‘contrastRight’, ‘feedbackType’, ‘choice’, ‘response_times’, ‘stimOn_times’}.
ax (matplotlib.pyplot.Axes) – An axis object to plot onto.
title (str) – An optional plot title.
plot_ci (bool) – If true, computes and plots the confidence intervals for response at each contrast.
ci_alpha (float, default=0.32) – Significance level for confidence interval. Must be in (0, 1). If plot_ci is false, this value is ignored.
**kwargs – If ax is None, these arguments are passed to matplotlib.pyplot.subplots.
 Returns:
matplotlib.pyplot.Figure – The figure handle containing the plot.
matplotlib.pyplot.Axes – The plotted axes.
See also
scipy.stats.bootstrap
 plot_reaction_time_over_trials(trials, stim_on_type='stimOn_times', ax=None, title=None, **kwargs)[source]
Function to plot reaction time with trial number a la datajoint webpage.
 Parameters:
trials (one.alf.io.AlfBunch) – An ALF trials object containing the keys {‘probabilityLeft’, ‘contrastLeft’, ‘contrastRight’, ‘feedbackType’, ‘choice’, ‘response_times’, ‘stimOn_times’}.
stim_on_type (str, default='stimOn_times') – The trials key to use when calculating the response times. The difference between this and ‘feedback_times’ is used (see notes for compute_median_reaction_time).
ax (matplotlib.pyplot.Axes) – An axis object to plot onto.
title (str) – An optional plot title.
**kwargs – If ax is None, these arguments are passed to matplotlib.pyplot.subplots.
 Returns:
matplotlib.pyplot.Figure – The figure handle containing the plot.
matplotlib.pyplot.Axes – The plotted axes.
 query_criterion(subject, status, from_status=None, one=None, validate=True)[source]
Get the session for which a given training criterion was met.
 Parameters:
subject (str) – The subject name.
status (str) – The training status to query for.
from_status (str, optional) – Count number of sessions and days from reaching from_status to status.
one (one.api.OneAlyx, optional) – An instance of ONE.
validate (bool) – If true, check if status in TrainingStatus enumeration. Set to false for nonstandard training pipelines.
 Returns:
str – The eID of the first session where this training status was reached.
int – The number of sessions it took to reach status (optionally from reaching from_status).
int – The number of days it tool to reach status (optionally from reaching from_status).