API Reference#

Start from the group you are working in, or search every public object at once.

All objects

Every public class and function in the benchmark, in one filterable table.

Evaluation contract

What a submission must contain.

ibl_bwb_eval: Evaluation contract for the IBL BrainWideBench benchmark.
ibl_bwb_eval.metrics: Metric definitions the benchmark reports.
ibl_bwb_eval.multi_unit: The multi-unit readout: TS3’s second scored submission variant.
ibl_bwb_eval.predictions: Writing a submission to disk.
ibl_bwb_eval.entity_ids: The ids a submission is aligned by, and how they reach a .safetensors file.

Scoring

How a directory of submissions becomes the reported numbers.

ibl_bwb_eval.scoring.ts1: TS1 scoring utility.
ibl_bwb_eval.scoring.ts2: TS2 scoring utility.
ibl_bwb_eval.scoring.ts3: TS3 scoring utility.
ibl_bwb_eval.scoring.aggregation: Score aggregation and ranking across all suites.

Core

The pieces every task suite builds on.

core.data: The unit and recording bookkeeping a build carries into every task suite.
core.dataset: The dataset every task suite and pretraining run reads the benchmark through.
core.finetuning: How much of a pretrained encoder a downstream run is allowed to move.
core.model: The interface every benchmark model implements.
core.trainer: The training loop every model’s trainer is built on.
core.samplers: Samplers for batching across sessions of unequal size.
core.transforms: Unit selection and spike-count augmentations, shared by every task suite.
core.nn: Layers shared by more than one model, and the attention backends they run on.
core.nn.loss: Losses shared by more than one model’s objective.
core.nn.metrics: Training-time diagnostics. The reported metrics live in ibl_bwb_eval.metrics.

Pretraining

Encoders pretrained once, then evaluated by each suite.

pretrain.models: The pretrained encoders, their maskers, and the pretrainers that fit them.
pretrain.datasets: What a pretraining run reads, shared by the models that fit the same objective.

TS1: Behavior Prediction

Decoding behavior from neural population activity.

ts1: Task Suite 1: Behavior Prediction.
ts1.models.single_session: Baselines fit on one session, from scratch.
ts1.models.pretrained: Eval trainers that decode behavior from a pretrained encoder.

TS2: Neural Activity Prediction

Predicting activity across time and across neurons.

ts2: Task Suite 2: Neural Activity Prediction.
ts2.models.single_session: Baselines fit on one session, from statistical to sequence models.
ts2.models.pretrained: Eval trainers that predict activity from a pretrained encoder.

TS3: Brain Region Prediction

Predicting the region a single neuron was recorded in.

ts3: Task Suite 3: Brain Region Prediction.
ts3.models.inductive: Extractors whose unit embedding is a function of the unit’s own data.
ts3.models.transductive: Extractors whose unit embedding is a free parameter indexed by unit identity.
ts3.models.supervised: Models trained in-suite on the region labels, which emit probabilities, not embeddings.
ts3.probes: The classifiers fit on a saved embeddings file, one embedding per unit.