Aims

The aims of the course are to:

• To introduce students to machine learning approaches to modeling of natural phenomena and hypothesis testing.
• To apply generative modeling techniques to neurobiological data in order to infer underlying mechanisms.
• To gain practical experience with model inference and validation, and hypothesis testing via model selection.
• To gain experience with issues such as overfitting, and possible lack of robustness of conclusions due to model misspecification.

Content

In this projects students will study two proposed mechanisms hypothesised to underlie the firing rate patterns of neurons recorded in an area of the monkey cortex thought to be involved in evidence integration for decision making. The ultimate goal is to infer which of the two mechanisms (as two competing hypotheses) had generated a simulated dataset of neural responses provided to the students.

The students will also be provided with a package written in Python allowing them to simulate two generative models as concrete formalisations of the two conceptual hypotheses.
The students will first explore the behaviour of the outputs (neural spike trains) of the these two models in different regions of their parameter space. They will be guided to appreciate
how some key aspects of the data, commonly relied on in neuroscience, can look near identical in the data generated by the two simulators despite their qualitatively different mechanisms.
This motivates the use of Bayesian statistical techniques for inferring the models and their parameters from whole datasets (as opposed to summary statistics).

Students will then carry out model fitting and Bayesian inference of latent variables and model parameters. This is partly done by writing their own code, and partly using provided Python programmes.
Students will also carry out model validation using simulated data generated by ground-truth models, in order to gain insight into factors affecting model recovery and overfitting, and approaches for mitigating it.
Students will explore the issue of "brittleness" and non-robustness of hypothesis testing, when auxiliary features of the models formalising the hypotheses do not match those in the ground-truth model.  Students will then apply their gained knowledge to infer the mechanism underlying a dataset of neural responses.

Note that this project is being offered for the first time and some of the details above may be subject to change.

Format

Week 1

Explore the behaviour of the two generative models and the effect of different parameters. Understand how and why trial-average firing rates generated by the two models can look similar.

Week 2

Students will generate datasets and carry out model inference (inference of parameters) by implementing the expectation-maximization and variational inference algorithms. Students will assess overfitting using cross-validation and study its behaviour with growing dataset size.

Week 3

Students will be introduced to information criteria for model selection, and will carry out model-recovery experiments to assess whether a given dataset allows for reliable inference of underlying model.

Week 4

Students will explore simulating and fitting models which realise the same conceptual mechanisms but differ in other aspects, in order to explore the effect of those differences and mismatches on model selection. At the end students apply their gain experience to infer the mechanism that generated a dataset provided to them.