Aims

The aims of the course are to:

• Provide a thorough introduction into the topic of statistical inference including maximum-likelihood and Bayesian approaches
• Introduce inference algorithms for regression, classification, clustering and sequence modelling
• Introduce basic concepts in optimisation, dynamic programming and Monte Carlo sampling

Objectives

As specific objectives, by the end of the course students should be able to:

• Understand the use of maximum-likelihood and Bayesian inference and the strengths and weaknesses of both approaches.
• Implement methods to solve simple regression, classification, dimensionality reduction, clustering and sequence modelling problems.
• Implement simple optimisation methods (gradient and coordinate descent, stochastic gradient descent) and dynamic programming (Kalman filter or forward algorithm).

Content

Introduction to inference (2L)

• Revision of maximum likelihood and Bayesian estimation
• Revision of Bayesian decision theory
• Outline of the course

Regression (3L)

• What are regression problems
• Revision of properties of Gaussian probability density
• Maximum likelihood and Bayesian fitting of Gaussians
• Linear regression and non-linear regression

Classification (2L)

• Classification problems
• Logistic regression probabilistic model
• Training logistic regression using optimisation
• Stochastic optimisation methods
• Non-linear feature expansions for logistic regression​​

Dimensionality Reduction (2L)

• What is dimensionality reduction
• Principal component analysis as minimising reconstruction cost
• Principal component analysis as inference​

Clustering (3L)

• What is clustering
• The k-means algorithm
• Gaussian Mixture Models
• The Expectation Maximisation (EM) Algorithm

Sequence models (3L)

• Sequence modelling problems
• Markov Models and Hidden Markov models
• Inference in Hidden Markov Models using dynamic programming

Very Basic Monte Carlo (introduced through the lectures above)

• Simple Monte Carlo