Aims

The aims of the course are to:

• This module aims to teach the basic concepts of deep learning and forms of structure that can be used for generative and discriminative models. In addition the use of models for classifying structured data, such as speech and language, will be discussed

Objectives

As specific objectives, by the end of the course students should be able to:

• Understand the basic principles of pattern classification and deep learning;
• Understand generative and discriminative models for structured data;
• Understand the application of deep-learning to structured data;
• Be able to apply pattern processing techniques to practical applications.

Content

Introduction (1L)

Links with 3F8 and 4F13. General machine learning, examples of structured data, DNA, vision, speech and language processing.

Graphical Models and Conditional Indpendence (1L)

Graphical models and Bayesian networks. Simple inference examples. 

Generative and discriminative sequence models (2L)

Hidden Markov models and expectation maximisatiion (EM) - use for training Gaussian mixture models (GMMs) and Factor Analysis. Representation of these models as dynamic Bayesian networks. Conditional Random Fields (CRFs) as an example of a discriminative sequence model.

Deep Learning (3L)

Generative and discriminative deep models. Forms of network and activation functions. Convolutional neural networks, mixture-density neural networks. Optimisation approaches (first/second order methods, adaptive learning rates) and initialisation.

Deep Learning for Sequences (1L)

Recurrent neural networks, and long-short-term memory models. Variants of RNN including bidirectional RNNs. Use in generative and discriminative models.

Alternate Deep Network Architectures (1L)

Variational Auto Encoders (VAE) and variational estimation (possibly other examples: generative adversarial networks, Siamese networks).

Support Vector Machines (2L)

Maximum margin classifiers, handling non-separable data, training SVMs, non-linear SVMs, kernel functions. Links with other kernel methods Gaus- sian Processes, Relevance Vector Machines. Multi-class SVMs and structured SVMs.

Kernels over Structured Data (1L)

Tree kernels, graph kernels, Fisher kernels. Relationship to RNNs.

Traditional and Bayesian Non-Parametric Techniques(2L)

Classification and regression trees, parzen windows, K-nearest neighbours, nearest neighbour rule.  Ensemble methods:  random forests, bagging, boosting and model combination.

This syllabus contributes to the following areas of the UK-SPEC standard:

Toggle display of UK-SPEC areas.

 
Last modified: 20/02/2025 12:12