Undergraduate Teaching 2017-18

Engineering Tripos Part IIB, 4F10: Deep Learning & Structured Data, 2017-18

Engineering Tripos Part IIB, 4F10: Deep Learning & Structured Data, 2017-18

Not logged in. More information may be available... Login via Raven / direct.

PDF versionPDF version


Prof M Gales


Prof M Gales, Dr J M Hernandez-Lobato

Timing and Structure

Michaelmas term. 14 lectures + 2 examples classes. Assessment: 100% exam


Part IIA Modules 3F1 and 3F3 advisable


The aims of the course are to:

  • This module aims to teach the basic concepts of deep learning and forms of structure that can be used for generative and discriminative models. In ad-dition the use of models for classifying structured data, such as speech and language, will be discussed


As specific objectives, by the end of the course students should be able to:

  • Understand the basic principles of pattern classification and deep learning;
  • Understand generative and discriminative models for structured data;
  • Understand the application of deep-learning to structured data;
  • Be able to apply pattern processing techniques to practical applications.


Introduction (1L)

Links with 3F8 and 4F13. General machine learning, examples of struc- tured data, DNA, vision, speech and language processing.

Graphical Models and Conditional Indpendence (1L)

Graphical models and Bayesian networks. Simple inference examples. Dynamic Bayesian networks. Hidden Markov Models reminder.


Restricted Boltzman Machines (1L)

Structure of restricted Boltzman machines, contrastive divergence.


Deep Learning (2L)

Generative and discriminative deep models. Forms of network and acti- vation functions. Convolutional neural networks, mixture-density neural networks. Optimisation approaches (first/second order methods, adaptive learning rates) and initialisation.


Deep Learning for Sequences (1L)

Recurrent neural networks, and long-short-term memory models. Variants of RNN including bidirectional RNNs. Use in generative and discrimina- tive models.


Alternate Deep Network Architectures (1L)

Auto-encoders and variational extension, student-teacher training (possi- bly other examples: adversarial networks, siamese networks).


Discriminative Sequence Models (1L)

Conditional random fields and log-linear models (discuss maximum en- tropy models).

Support Vector Machines (2L)

Maximum margin classifiers, handling non-separable data, training SVMs, non-linear SVMs, kernel functions. Links with other kernel methods Gaus- sian Processes, Relevance Vector Machines. Multi-class SVMs and struc- tured SVMs.

Kernels over Structured Data (1L)

Tree kernels, graph kernels, Fisher kernels. Relationship to RNNs.


Traditional and Bayesian Non-Parametric Techniques(2L)

  • Classification and regression trees, parzen windows, K-nearest neighbours, nearest neighbour rule.  Ensemble methods:  random forests, bagging, boosting and model combination.


Application: Speech Processing (1L)

Example application of deep-learning to speech processing.



Please see the Booklist for Group F Courses for references for this module.

Examination Guidelines

Please refer to Form & conduct of the examinations.


The UK Standard for Professional Engineering Competence (UK-SPEC) describes the requirements that have to be met in order to become a Chartered Engineer, and gives examples of ways of doing this.

UK-SPEC is published by the Engineering Council on behalf of the UK engineering profession. The standard has been developed, and is regularly updated, by panels representing professional engineering institutions, employers and engineering educators. Of particular relevance here is the 'Accreditation of Higher Education Programmes' (AHEP) document which sets out the standard for degree accreditation.

The Output Standards Matrices indicate where each of the Output Criteria as specified in the AHEP 3rd edition document is addressed within the Engineering and Manufacturing Engineering Triposes.

Last modified: 23/06/2017 15:28