Timing and Structure
Michaelmas term. 75% exam / 25% coursework.
3F3, 3F8, 3M1
The aims of the course are to:
- Introduce students to foundational theoretical concepts and methodological tools essential for the successful development, analysis, and application of advanced Machine Learning and Computational Statistical methods.
As specific objectives, by the end of the course students should be able to:
- Introduce the students to the required statistical and mathematical concepts that underpin all rigorously designed Machine Learning and Computational Statistical methods that can be used practically across all the contemporary engineering sciences
- Introduce the students to advanced computational statistical inference methods required to design Machine Learning solutions to a range of challenging large scale engineering problems where data and models are synthesised
By successful completion of this course the student will have an appreciation and basic understanding of the mathematical, probabilistic, and statistical foundations of modern Computational Statistical Methods and recent developments in Machine Learning algorithms.
Computational Statistics and Machine Learning
: Numerically computing integrals, the law of large numbers for Monte Carlo estimators, The Central Limit Theorem for Monte Carlo estimators.
: Improving MC estimators, Importance Sampling, Control Variates to reduce variance of estimates.
: Difference between Riemann and Lebesgue Integral, why Lebesgue integral is required for machine learning and engineering, definition of Lebesgue integral.
: Definition of Lebesgue Measure, Radon-Nikodym derivative and change of measure, Measure theoretic basis of Probability (Kolmogorov), Random Variables.
: Definition of Markov chain and invariant distributions, presentation of the Metropolis and Hastings method.
: Metropolis Hastings in multiple dimensions, the Gibbs Sampler.
: generalisation of Euclidean space in R^3 to infinite dimensional spaces, Completion of space and definition of Banach space of functions.
: Inner product space, definition of Hilbert space, Cauchy sequences and function approximation, Reproducing kernel Hilbert Space and function approximation.
: Definition of weak derivatives, understanding rates of convergence of function approximations based on properties of Sobolev space (smoothness)
: Illustrating non-existence of Lebesgue Measure in function space, construction of finite Gaussian measure in Hilbert space, definition of Bayes rule (via Radon-Nikodym derivative) in Hilbert space employing Gaussian measure as reference - GP's.
: defining dimension invariant Markov transition kernel in Hilbert space and how overcomes degeneracy in high dimensions.
- use of RKHS to obtain super-root-N convergence of Monte Carlo Estimators.
and MCMC for them using pseudo-marginal MCMC methodology.
simulation and inference for doubly intractable probability measures
Machine Learning methods are having a major impact in every area of the engineering sciences. Machine Learning models and methods rely predominantly on Computational Statistics methods for model calibration, estimation, prediction and updating. Together Computational Statistics and Machine Learning are providing a revolution in the way mankind lives, works, communicates, and transacts.
Simulation Based Inference on Engineering Problem
The synthesis of both data and formal mathematical models in defining a digital twin of an engineering problem will be presented. The design of the machine learning and computational statistical methods to characterise uncertainty in predictions and forecasts from the digital twin will be the main focus of this exercise.
Wed week 9
Shima, H. Functional Analysis for Physics and Engineering: An Introduction, CRC Press.
Biegler, L., Biros, G., Ghattas, O., Heinkenschloss, M., Keyes, D., Mallickj, B., Tenorio, L., van Bloemen Waanders, B., Willcox, K., and Marouk, Y. (2010). Large-Scale Inverse Problems and Quantification of Uncertainty. Wiley.
Brooks, S., Gelman, A., Jones, G. L., and Meng, X. (2011). Handbook of Markov Chain Monte Carlo. CRC.
Cotter, C. and Reich, S. (2015). Probabilistic Forecasting and Bayesian Data Assimilation. Cambridge University Press.
Law, K., Stuart, A., and Zygalakis, K. (2015). Data Assimilation: A Mathematical Introduction. Springer.
Rogers, S. and Girolami, M. (2016). A First Course in Machine Learning, 2nd Edition. CRC.
Sullivan, T. J. (2015). Introduction to Uncertainty Quantification. Springer.
Please refer to Form & conduct of the examinations.
Last modified: 08/02/2021 14:34