Engineering Tripos Part IIB, 4F8: Image Processing & Imaging Coding, 2022-23
Module Leader
Lecturers
Timing and Structure
Lent term. 16 lectures (including examples classes). Assessment: 100% exam
Prerequisites
3F1 assumed; 3F3, 3F7 useful
Aims
The aims of the course are to:
- introduce the key tools for performing sophisticated processing of images by digital hardware
Objectives
As specific objectives, by the end of the course students should be able to:
- understand the main elements of 2-dimensional linear system theory.
- design linear spatial filters for a variety of applications (smoothing etc)
- understand techniques for the restoration and enhancement of degraded images.
- show familiarity with the main characteristics of the human visual system with particular reference to subjective criteria for image data compression.
- understand techniques for image coding using transform methods including the Discrete Cosine Transform (as used in the JPEG coding standard) and overlapped transforms.
- understand methods for coding transform coefficients to provide maximum data compression.
Content
Sophisticated processing of images by digital hardware is now fairly common, and ranges from special effects in video games to satellite image enhancement. Three of the main application areas are video data compression, image enhancement, and scene understanding. This module introduces the key tools for performing these tasks, and shows how these tools can be applied. The module will be split into two courses of 8 lectures each: Image Processing, and Image Coding. Lectures are supported by computer demonstrations. There will be one examples sheet for each of the two 8-lecture sections.
Image Processing (8L, Dr J Lasenby)
This course covers the following topics, relevant to most aspects of image processing:
- Two-dimensional linear system theory, as applied to discretely sampled systems:
- The continuous 2D Fourier transform and its properties
- Digitisation, sampling, aliasing and quantisation
- The discrete 2D Fourier transform (DFT)
- 2D Digital Filters and Filter Design
- Zero phase filters
- Ideal 2D filters: rectangular and bandpass
- Filter design: the window method
- Image Deconvolution
- Deconvolution of noiseless images -- the inverse filter
- The Wiener filter (conventional and Bayesian derivations)
- Maximum Entropy deconvolution
- Image Enhancement
- Contrast enhancement
- Histogram equalisation
- Median filtering
Image Coding (8L, Prof N Kingsbury)
This course concentrates on image and video data compression techniques, and covers the following topics:
- Characteristics of the human visual system which are important for data compression:
- Spatial and temporal frequency sensitivities
- Distortion masking phenomena
- Luminance and colour (chrominance) processing
- 2D block transforms and wavelet transforms:
- Discrete cosine transforms
- Bi-orthogonal and orthonormal wavelet transforms
- Energy compaction properties of transforms for typical images
- Optimal quantisation techniques of coding transform coefficients for maximum data compression
- Huffman coding
- Run-length coding
- JPEG 2-dimensional run-size coding
- Video coding techniques
- Motion analysis
- Motion vector coding
- MPEG coding standards
Booklists
Please refer to the Booklist for Part IIB Courses for references to this module, this can be found on the associated Moodle course.
Examination Guidelines
Please refer to Form & conduct of the examinations.
UK-SPEC
This syllabus contributes to the following areas of the UK-SPEC standard:
Toggle display of UK-SPEC areas.
GT1
Develop transferable skills that will be of value in a wide range of situations. These are exemplified by the Qualifications and Curriculum Authority Higher Level Key Skills and include problem solving, communication, and working with others, as well as the effective use of general IT facilities and information retrieval skills. They also include planning self-learning and improving performance, as the foundation for lifelong learning/CPD.
IA1
Apply appropriate quantitative science and engineering tools to the analysis of problems.
IA2
Demonstrate creative and innovative ability in the synthesis of solutions and in formulating designs.
KU1
Demonstrate knowledge and understanding of essential facts, concepts, theories and principles of their engineering discipline, and its underpinning science and mathematics.
KU2
Have an appreciation of the wider multidisciplinary engineering context and its underlying principles.
D1
Wide knowledge and comprehensive understanding of design processes and methodologies and the ability to apply and adapt them in unfamiliar situations.
D4
Ability to generate an innovative design for products, systems, components or processes to fulfil new needs.
E1
Ability to use fundamental knowledge to investigate new and emerging technologies.
E2
Ability to extract data pertinent to an unfamiliar problem, and apply its solution using computer based engineering tools when appropriate.
E3
Ability to apply mathematical and computer based models for solving problems in engineering, and the ability to assess the limitations of particular cases.
E4
Understanding of and ability to apply a systems approach to engineering problems.
P1
A thorough understanding of current practice and its limitations and some appreciation of likely new developments.
P3
Understanding of contexts in which engineering knowledge can be applied (e.g. operations and management, technology, development, etc).
P8
Ability to apply engineering techniques taking account of a range of commercial and industrial constraints.
US1
A comprehensive understanding of the scientific principles of own specialisation and related disciplines.
US2
A comprehensive knowledge and understanding of mathematical and computer models relevant to the engineering discipline, and an appreciation of their limitations.
US4
An awareness of developing technologies related to own specialisation.
Last modified: 24/05/2022 13:12
Engineering Tripos Part IIB, 4F8: Image Processing & Imaging Coding, 2020-21
Module Leader
Lecturers
Timing and Structure
Lent term. 16 lectures (including examples classes). Assessment: 100% exam
Prerequisites
3F1 assumed; 3F3, 3F7 useful
Aims
The aims of the course are to:
- introduce the key tools for performing sophisticated processing of images by digital hardware
Objectives
As specific objectives, by the end of the course students should be able to:
- understand the main elements of 2-dimensional linear system theory.
- design linear spatial filters for a variety of applications (smoothing etc)
- understand techniques for the restoration and enhancement of degraded images.
- show familiarity with the main characteristics of the human visual system with particular reference to subjective criteria for image data compression.
- understand techniques for image coding using transform methods including the Discrete Cosine Transform (as used in the JPEG coding standard) and overlapped transforms.
- understand methods for coding transform coefficients to provide maximum data compression.
Content
Sophisticated processing of images by digital hardware is now fairly common, and ranges from special effects in video games to satellite image enhancement. Three of the main application areas are video data compression, image enhancement, and scene understanding. This module introduces the key tools for performing these tasks, and shows how these tools can be applied. The module will be split into two courses of 8 lectures each: Image Processing, and Image Coding. Lectures are supported by computer demonstrations. There will be one examples sheet for each of the two 8-lecture sections.
Image Processing (8L, Dr J Lasenby)
This course covers the following topics, relevant to most aspects of image processing:
- Two-dimensional linear system theory, as applied to discretely sampled systems:
- The continuous 2D Fourier transform and its properties
- Digitisation, sampling, aliasing and quantisation
- The discrete 2D Fourier transform (DFT)
- 2D Digital Filters and Filter Design
- Zero phase filters
- Ideal 2D filters: rectangular and bandpass
- Filter design: the window method
- Image Deconvolution
- Deconvolution of noiseless images -- the inverse filter
- The Wiener filter (conventional and Bayesian derivations)
- Maximum Entropy deconvolution
- Image Enhancement
- Contrast enhancement
- Histogram equalisation
- Median filtering
Image Coding (8L, Prof N Kingsbury)
This course concentrates on image and video data compression techniques, and covers the following topics:
- Characteristics of the human visual system which are important for data compression:
- Spatial and temporal frequency sensitivities
- Distortion masking phenomena
- Luminance and colour (chrominance) processing
- 2D block transforms and wavelet transforms:
- Discrete cosine transforms
- Bi-orthogonal and orthonormal wavelet transforms
- Energy compaction properties of transforms for typical images
- Optimal quantisation techniques of coding transform coefficients for maximum data compression
- Huffman coding
- Run-length coding
- JPEG 2-dimensional run-size coding
- Video coding techniques
- Motion analysis
- Motion vector coding
- MPEG coding standards
Booklists
Please refer to the Booklist for Part IIB Courses for references to this module, this can be found on the associated Moodle course.
Examination Guidelines
Please refer to Form & conduct of the examinations.
UK-SPEC
This syllabus contributes to the following areas of the UK-SPEC standard:
Toggle display of UK-SPEC areas.
GT1
Develop transferable skills that will be of value in a wide range of situations. These are exemplified by the Qualifications and Curriculum Authority Higher Level Key Skills and include problem solving, communication, and working with others, as well as the effective use of general IT facilities and information retrieval skills. They also include planning self-learning and improving performance, as the foundation for lifelong learning/CPD.
IA1
Apply appropriate quantitative science and engineering tools to the analysis of problems.
IA2
Demonstrate creative and innovative ability in the synthesis of solutions and in formulating designs.
KU1
Demonstrate knowledge and understanding of essential facts, concepts, theories and principles of their engineering discipline, and its underpinning science and mathematics.
KU2
Have an appreciation of the wider multidisciplinary engineering context and its underlying principles.
D1
Wide knowledge and comprehensive understanding of design processes and methodologies and the ability to apply and adapt them in unfamiliar situations.
D4
Ability to generate an innovative design for products, systems, components or processes to fulfil new needs.
E1
Ability to use fundamental knowledge to investigate new and emerging technologies.
E2
Ability to extract data pertinent to an unfamiliar problem, and apply its solution using computer based engineering tools when appropriate.
E3
Ability to apply mathematical and computer based models for solving problems in engineering, and the ability to assess the limitations of particular cases.
E4
Understanding of and ability to apply a systems approach to engineering problems.
P1
A thorough understanding of current practice and its limitations and some appreciation of likely new developments.
P3
Understanding of contexts in which engineering knowledge can be applied (e.g. operations and management, technology, development, etc).
P8
Ability to apply engineering techniques taking account of a range of commercial and industrial constraints.
US1
A comprehensive understanding of the scientific principles of own specialisation and related disciplines.
US2
A comprehensive knowledge and understanding of mathematical and computer models relevant to the engineering discipline, and an appreciation of their limitations.
US4
An awareness of developing technologies related to own specialisation.
Last modified: 01/09/2020 10:38
Engineering Tripos Part IIB, 4F7: Statistical Signal Analysis, 2018-19
Module Leader
Lecturer
Timing and Structure
Lent term. 16 lectures (including examples classes). Assessment: 100% exam
Prerequisites
3F3; Useful 3F1 and 3F8
Aims
The aims of the course are to:
- Continue the study of statistical signal processing techniques from the basics studied in 3F3.
- Introduce time-series models, in particular State-space models and hidden Markov models; understand their role in applications of signal processing.
- Develop techniques for fitting statisical modes to data and estimating hidden signals from noisy observations.
Objectives
As specific objectives, by the end of the course students should be able to:
- Understand state-space models and hidden Markov models including their mathematical characterisation, strengths and limitations.
- Understand how to execute all the necessary computational tasks involved in fitting the models to data, to estimate unobserved quantities and make future predictions.
- Understand the computational methodology employed, their mathematical derivation, their strengths and weaknesses, how to execute them, and their use more generally in Statistical and data-centric engineering problems.
Content
This course is about fitting statistical models to data that arrives sequentially over time. Once an appropriate model has been fit, tasks like predicting future trends or estimating quantities not directly observed can be performed. The statistical modelling and computational methodology covered by this course is widely used in many applied areas. For example, data that arrives sequentially over time is a common occurrence in Signal Processing (Engineering), Finance, Machine Learning, Environmental statistics etc.
The model that most appropriately describes data that arrives sequentially over time is a time-series model, an example of which is the ARMA model (studied in 3F3.) However, this course will look at more versatile models that incorporate hidden or latent state variables as these are able to account for richer behaviour. Also, models that aim describe how many really physical processes evolve over time often necessarily have to incorporate unobserved hidden states that form a Markov process.
- Introduction to state-space models and optimal linear filtering; the Kalman filter; exemplar problems in signal processing.
- Introduction to hidden Markov models: definition; inference/estimation aims; exact computation of the conditional probability distributions.
- Importance sampling: introduction; weight degeneracy; statistical properties.
- Sequential importance sampling and resampling (also known as the particle filter): application to hidden Markov models; filtering; smoothing.
- Calibrating hidden Markov models: maximum likelihood estimation and its implementation.
- Exemplar problems in Signal Processing.
- Examples Papers.
Booklists
Please see the Booklist for Group F Courses for references for this module.
Examination Guidelines
Please refer to Form & conduct of the examinations.
UK-SPEC
This syllabus contributes to the following areas of the UK-SPEC standard:
Toggle display of UK-SPEC areas.
GT1
Develop transferable skills that will be of value in a wide range of situations. These are exemplified by the Qualifications and Curriculum Authority Higher Level Key Skills and include problem solving, communication, and working with others, as well as the effective use of general IT facilities and information retrieval skills. They also include planning self-learning and improving performance, as the foundation for lifelong learning/CPD.
IA1
Apply appropriate quantitative science and engineering tools to the analysis of problems.
IA2
Demonstrate creative and innovative ability in the synthesis of solutions and in formulating designs.
KU1
Demonstrate knowledge and understanding of essential facts, concepts, theories and principles of their engineering discipline, and its underpinning science and mathematics.
KU2
Have an appreciation of the wider multidisciplinary engineering context and its underlying principles.
E1
Ability to use fundamental knowledge to investigate new and emerging technologies.
E2
Ability to extract data pertinent to an unfamiliar problem, and apply its solution using computer based engineering tools when appropriate.
E3
Ability to apply mathematical and computer based models for solving problems in engineering, and the ability to assess the limitations of particular cases.
E4
Understanding of and ability to apply a systems approach to engineering problems.
P1
A thorough understanding of current practice and its limitations and some appreciation of likely new developments.
P3
Understanding of contexts in which engineering knowledge can be applied (e.g. operations and management, technology, development, etc).
P8
Ability to apply engineering techniques taking account of a range of commercial and industrial constraints.
US1
A comprehensive understanding of the scientific principles of own specialisation and related disciplines.
US2
A comprehensive knowledge and understanding of mathematical and computer models relevant to the engineering discipline, and an appreciation of their limitations.
Last modified: 13/09/2018 15:29
Engineering Tripos Part IIB, 4F7: Statistical Signal Analysis, 2021-22
Module Leader
Lecturer
Timing and Structure
Michaelmas term. 16 lectures (including examples classes). Assessment: 100% exam
Prerequisites
3F3; Useful 3F1 and 3F8
Aims
The aims of the course are to:
- Continue the study of statistical signal processing techniques from the basics studied in 3F3.
- Introduce time-series models, in particular State-space models and hidden Markov models; understand their role in applications of signal processing.
- Develop techniques for fitting statisical modes to data and estimating hidden signals from noisy observations.
Objectives
As specific objectives, by the end of the course students should be able to:
- Understand state-space models and hidden Markov models including their mathematical characterisation, strengths and limitations.
- Understand how to execute all the necessary computational tasks involved in fitting the models to data, to estimate unobserved quantities and make future predictions.
- Understand the computational methodology employed, their mathematical derivation, their strengths and weaknesses, how to execute them, and their use more generally in Statistical and data-centric engineering problems.
Content
This course is about fitting statistical models to data that arrives sequentially over time. Once an appropriate model has been fit, tasks like predicting future trends or estimating quantities not directly observed can be performed. The statistical modelling and computational methodology covered by this course is widely used in many applied areas. For example, data that arrives sequentially over time is a common occurrence in Signal Processing (Engineering), Finance, Machine Learning, Environmental statistics etc.
The model that most appropriately describes data that arrives sequentially over time is a time-series model, an example of which is the ARMA model (studied in 3F3.) However, this course will look at more versatile models that incorporate hidden or latent state variables as these are able to account for richer behaviour. Also, models that aim describe how many really physical processes evolve over time often necessarily have to incorporate unobserved hidden states that form a Markov process.
- Introduction to state-space models and optimal linear filtering; the Kalman filter; exemplar problems in signal processing.
- Introduction to hidden Markov models: definition; inference/estimation aims; exact computation of the conditional probability distributions.
- Importance sampling: introduction; weight degeneracy; statistical properties.
- Sequential importance sampling and resampling (also known as the particle filter): application to hidden Markov models; filtering; smoothing.
- Calibrating hidden Markov models: maximum likelihood estimation and its implementation.
- Exemplar problems in Signal Processing.
- Examples Papers.
Booklists
Please refer to the Booklist for Part IIB Courses for references to this module, this can be found on the associated Moodle course.
Examination Guidelines
Please refer to Form & conduct of the examinations.
UK-SPEC
This syllabus contributes to the following areas of the UK-SPEC standard:
Toggle display of UK-SPEC areas.
GT1
Develop transferable skills that will be of value in a wide range of situations. These are exemplified by the Qualifications and Curriculum Authority Higher Level Key Skills and include problem solving, communication, and working with others, as well as the effective use of general IT facilities and information retrieval skills. They also include planning self-learning and improving performance, as the foundation for lifelong learning/CPD.
IA1
Apply appropriate quantitative science and engineering tools to the analysis of problems.
IA2
Demonstrate creative and innovative ability in the synthesis of solutions and in formulating designs.
KU1
Demonstrate knowledge and understanding of essential facts, concepts, theories and principles of their engineering discipline, and its underpinning science and mathematics.
KU2
Have an appreciation of the wider multidisciplinary engineering context and its underlying principles.
E1
Ability to use fundamental knowledge to investigate new and emerging technologies.
E2
Ability to extract data pertinent to an unfamiliar problem, and apply its solution using computer based engineering tools when appropriate.
E3
Ability to apply mathematical and computer based models for solving problems in engineering, and the ability to assess the limitations of particular cases.
E4
Understanding of and ability to apply a systems approach to engineering problems.
P1
A thorough understanding of current practice and its limitations and some appreciation of likely new developments.
P3
Understanding of contexts in which engineering knowledge can be applied (e.g. operations and management, technology, development, etc).
P8
Ability to apply engineering techniques taking account of a range of commercial and industrial constraints.
US1
A comprehensive understanding of the scientific principles of own specialisation and related disciplines.
US2
A comprehensive knowledge and understanding of mathematical and computer models relevant to the engineering discipline, and an appreciation of their limitations.
Last modified: 20/05/2021 07:49
Engineering Tripos Part IIB, 4F7: Statistical signal and network models, 2025-26
Module Leader
lecturers
Timing and Structure
Michaelmas term. 16 lectures (including examples classes). Assessment: 100% exam
Prerequisites
3F1, 3F3, 3F8 recommended. 3M1 useful.
Aims
The aims of the course are to:
- Introduce time-series models, in particular State-space models and hidden Markov models; understand their role in applications of signal processing.
- Develop techniques for fitting statisical models to data and estimating hidden signals from noisy observations.
- Introduce network models, graph algorithms, and techniques to analyse large scale relational data
Objectives
As specific objectives, by the end of the course students should be able to:
- Understand state-space models, hidden Markov models, and network models including their mathematical characterisation, strengths and limitations.
- Execute the necessary computational tasks involved in fitting the models to data, to estimate unobserved quantities and make future predictions.
- Understand the computational methodology employed, their mathematical derivation, their strengths and weaknesses, how to execute them, and their use more generally in Statistical and data-centric engineering problems.
Content
This course is about fitting statistical models to data that arrives sequentially over time and across network structures. Once an appropriate model has been fitted, tasks such as predicting future trends or estimating quantities not directly observed can be performed. The statistical modelling and computational methodology covered by this course is widely used in many applied areas. For example, data that arrives sequentially over time is a common occurrence in Signal Processing (Engineering), Finance, Machine Learning, Environmental statistics etc.
The model that most appropriately describes data that arrives sequentially over time is a time-series model, an example of which is the ARMA model (studied in 3F3.) However, this course will look at more versatile models that incorporate hidden or latent state variables as these are able to account for richer behaviour. Also, models that aim describe how many really physical processes evolve over time often necessarily have to incorporate unobserved hidden states that form a Markov process.
Besides changing over time, many data are distributed over different individuals or sites. For example, users of a social media platform will each have different properties. Networks (graphs) provide a simple formalism to analyse distributed systems composed of small but interrelated parts.
State space models and time series:
The model that most appropriately describes data that arrives sequentially over time is a time-series model, an example of which is the AR model (studied in 3F3). However, this course will look at more versatile stochastic (random) state-space models that incorporate hidden or latent state variables as these are able to account for richer behaviour. Also, models that aim describe how many really physical processes evolve over time often necessarily have to incorporate unobserved hidden states that form a Markov process.
-
Introduction to state-space models and optimal linear filtering; the Kalman filter; exemplar problems in signal processing.
-
Introduction to hidden Markov models: definition; inference/estimation aims; exact computation of the conditional probability distributions.
-
Importance sampling: introduction; weight degeneracy; statistical properties.
-
Sequential importance sampling and resampling (also known as the particle filter): application to hidden Markov models; filtering; smoothing.
-
Calibrating hidden Markov models: maximum likelihood estimation and its implementation.
-
Exemplar problems in Signal Processing.
-
Examples Papers.
Networks modelling:
Besides changing over time, many data are distributed over different individuals or sites. For example, users of a social media platform will each have different properties. Networks (graphs) provide a simple formalism to analyse distributed systems composed of small but interrelated parts. We will cover:
-
· Fundamentals, basic graph theory and algorithms.
-
· Metrics: centrality (e.g. PageRank), assortativity, clustering, diameter ("six degrees of separation").
-
· Models of networks: Erdős–Rényi, small-world and scale free.
-
· Models on networks: Spreading, reliability and percolation.
-
· Community detection and stochastic block models.
-
· Graph spectra and their applications.
Booklists
Please refer to the Booklist for Part IIB Courses for references to this module, this can be found on the associated Moodle course.
Examination Guidelines
Please refer to Form & conduct of the examinations.
UK-SPEC
This syllabus contributes to the following areas of the UK-SPEC standard:
Toggle display of UK-SPEC areas.
GT1
Develop transferable skills that will be of value in a wide range of situations. These are exemplified by the Qualifications and Curriculum Authority Higher Level Key Skills and include problem solving, communication, and working with others, as well as the effective use of general IT facilities and information retrieval skills. They also include planning self-learning and improving performance, as the foundation for lifelong learning/CPD.
IA1
Apply appropriate quantitative science and engineering tools to the analysis of problems.
IA2
Demonstrate creative and innovative ability in the synthesis of solutions and in formulating designs.
KU1
Demonstrate knowledge and understanding of essential facts, concepts, theories and principles of their engineering discipline, and its underpinning science and mathematics.
KU2
Have an appreciation of the wider multidisciplinary engineering context and its underlying principles.
E1
Ability to use fundamental knowledge to investigate new and emerging technologies.
E2
Ability to extract data pertinent to an unfamiliar problem, and apply its solution using computer based engineering tools when appropriate.
E3
Ability to apply mathematical and computer based models for solving problems in engineering, and the ability to assess the limitations of particular cases.
E4
Understanding of and ability to apply a systems approach to engineering problems.
P1
A thorough understanding of current practice and its limitations and some appreciation of likely new developments.
P3
Understanding of contexts in which engineering knowledge can be applied (e.g. operations and management, technology, development, etc).
P8
Ability to apply engineering techniques taking account of a range of commercial and industrial constraints.
US1
A comprehensive understanding of the scientific principles of own specialisation and related disciplines.
US2
A comprehensive knowledge and understanding of mathematical and computer models relevant to the engineering discipline, and an appreciation of their limitations.
Last modified: 04/06/2025 13:30
Engineering Tripos Part IIB, 4F7: Statistical Signal Analysis, 2019-20
Module Leader
Lecturer
Timing and Structure
Michaelmas term. 16 lectures (including examples classes). Assessment: 100% exam
Prerequisites
3F3; Useful 3F1 and 3F8
Aims
The aims of the course are to:
- Continue the study of statistical signal processing techniques from the basics studied in 3F3.
- Introduce time-series models, in particular State-space models and hidden Markov models; understand their role in applications of signal processing.
- Develop techniques for fitting statisical modes to data and estimating hidden signals from noisy observations.
Objectives
As specific objectives, by the end of the course students should be able to:
- Understand state-space models and hidden Markov models including their mathematical characterisation, strengths and limitations.
- Understand how to execute all the necessary computational tasks involved in fitting the models to data, to estimate unobserved quantities and make future predictions.
- Understand the computational methodology employed, their mathematical derivation, their strengths and weaknesses, how to execute them, and their use more generally in Statistical and data-centric engineering problems.
Content
This course is about fitting statistical models to data that arrives sequentially over time. Once an appropriate model has been fit, tasks like predicting future trends or estimating quantities not directly observed can be performed. The statistical modelling and computational methodology covered by this course is widely used in many applied areas. For example, data that arrives sequentially over time is a common occurrence in Signal Processing (Engineering), Finance, Machine Learning, Environmental statistics etc.
The model that most appropriately describes data that arrives sequentially over time is a time-series model, an example of which is the ARMA model (studied in 3F3.) However, this course will look at more versatile models that incorporate hidden or latent state variables as these are able to account for richer behaviour. Also, models that aim describe how many really physical processes evolve over time often necessarily have to incorporate unobserved hidden states that form a Markov process.
- Introduction to state-space models and optimal linear filtering; the Kalman filter; exemplar problems in signal processing.
- Introduction to hidden Markov models: definition; inference/estimation aims; exact computation of the conditional probability distributions.
- Importance sampling: introduction; weight degeneracy; statistical properties.
- Sequential importance sampling and resampling (also known as the particle filter): application to hidden Markov models; filtering; smoothing.
- Calibrating hidden Markov models: maximum likelihood estimation and its implementation.
- Exemplar problems in Signal Processing.
- Examples Papers.
Booklists
Please see the Booklist for Group F Courses for references for this module.
Examination Guidelines
Please refer to Form & conduct of the examinations.
UK-SPEC
This syllabus contributes to the following areas of the UK-SPEC standard:
Toggle display of UK-SPEC areas.
GT1
Develop transferable skills that will be of value in a wide range of situations. These are exemplified by the Qualifications and Curriculum Authority Higher Level Key Skills and include problem solving, communication, and working with others, as well as the effective use of general IT facilities and information retrieval skills. They also include planning self-learning and improving performance, as the foundation for lifelong learning/CPD.
IA1
Apply appropriate quantitative science and engineering tools to the analysis of problems.
IA2
Demonstrate creative and innovative ability in the synthesis of solutions and in formulating designs.
KU1
Demonstrate knowledge and understanding of essential facts, concepts, theories and principles of their engineering discipline, and its underpinning science and mathematics.
KU2
Have an appreciation of the wider multidisciplinary engineering context and its underlying principles.
E1
Ability to use fundamental knowledge to investigate new and emerging technologies.
E2
Ability to extract data pertinent to an unfamiliar problem, and apply its solution using computer based engineering tools when appropriate.
E3
Ability to apply mathematical and computer based models for solving problems in engineering, and the ability to assess the limitations of particular cases.
E4
Understanding of and ability to apply a systems approach to engineering problems.
P1
A thorough understanding of current practice and its limitations and some appreciation of likely new developments.
P3
Understanding of contexts in which engineering knowledge can be applied (e.g. operations and management, technology, development, etc).
P8
Ability to apply engineering techniques taking account of a range of commercial and industrial constraints.
US1
A comprehensive understanding of the scientific principles of own specialisation and related disciplines.
US2
A comprehensive knowledge and understanding of mathematical and computer models relevant to the engineering discipline, and an appreciation of their limitations.
Last modified: 28/05/2019 15:10
Engineering Tripos Part IIB, 4F7: Statistical Signal Analysis, 2022-23
Module Leader
Lecturer
Timing and Structure
Michaelmas term. 16 lectures (including examples classes). Assessment: 100% exam
Prerequisites
3F3; Useful 3F1 and 3F8
Aims
The aims of the course are to:
- Continue the study of statistical signal processing techniques from the basics studied in 3F3.
- Introduce time-series models, in particular State-space models and hidden Markov models; understand their role in applications of signal processing.
- Develop techniques for fitting statisical modes to data and estimating hidden signals from noisy observations.
Objectives
As specific objectives, by the end of the course students should be able to:
- Understand state-space models and hidden Markov models including their mathematical characterisation, strengths and limitations.
- Understand how to execute all the necessary computational tasks involved in fitting the models to data, to estimate unobserved quantities and make future predictions.
- Understand the computational methodology employed, their mathematical derivation, their strengths and weaknesses, how to execute them, and their use more generally in Statistical and data-centric engineering problems.
Content
This course is about fitting statistical models to data that arrives sequentially over time. Once an appropriate model has been fit, tasks like predicting future trends or estimating quantities not directly observed can be performed. The statistical modelling and computational methodology covered by this course is widely used in many applied areas. For example, data that arrives sequentially over time is a common occurrence in Signal Processing (Engineering), Finance, Machine Learning, Environmental statistics etc.
The model that most appropriately describes data that arrives sequentially over time is a time-series model, an example of which is the ARMA model (studied in 3F3.) However, this course will look at more versatile models that incorporate hidden or latent state variables as these are able to account for richer behaviour. Also, models that aim describe how many really physical processes evolve over time often necessarily have to incorporate unobserved hidden states that form a Markov process.
- Introduction to state-space models and optimal linear filtering; the Kalman filter; exemplar problems in signal processing.
- Introduction to hidden Markov models: definition; inference/estimation aims; exact computation of the conditional probability distributions.
- Importance sampling: introduction; weight degeneracy; statistical properties.
- Sequential importance sampling and resampling (also known as the particle filter): application to hidden Markov models; filtering; smoothing.
- Calibrating hidden Markov models: maximum likelihood estimation and its implementation.
- Exemplar problems in Signal Processing.
- Examples Papers.
Booklists
Please refer to the Booklist for Part IIB Courses for references to this module, this can be found on the associated Moodle course.
Examination Guidelines
Please refer to Form & conduct of the examinations.
UK-SPEC
This syllabus contributes to the following areas of the UK-SPEC standard:
Toggle display of UK-SPEC areas.
GT1
Develop transferable skills that will be of value in a wide range of situations. These are exemplified by the Qualifications and Curriculum Authority Higher Level Key Skills and include problem solving, communication, and working with others, as well as the effective use of general IT facilities and information retrieval skills. They also include planning self-learning and improving performance, as the foundation for lifelong learning/CPD.
IA1
Apply appropriate quantitative science and engineering tools to the analysis of problems.
IA2
Demonstrate creative and innovative ability in the synthesis of solutions and in formulating designs.
KU1
Demonstrate knowledge and understanding of essential facts, concepts, theories and principles of their engineering discipline, and its underpinning science and mathematics.
KU2
Have an appreciation of the wider multidisciplinary engineering context and its underlying principles.
E1
Ability to use fundamental knowledge to investigate new and emerging technologies.
E2
Ability to extract data pertinent to an unfamiliar problem, and apply its solution using computer based engineering tools when appropriate.
E3
Ability to apply mathematical and computer based models for solving problems in engineering, and the ability to assess the limitations of particular cases.
E4
Understanding of and ability to apply a systems approach to engineering problems.
P1
A thorough understanding of current practice and its limitations and some appreciation of likely new developments.
P3
Understanding of contexts in which engineering knowledge can be applied (e.g. operations and management, technology, development, etc).
P8
Ability to apply engineering techniques taking account of a range of commercial and industrial constraints.
US1
A comprehensive understanding of the scientific principles of own specialisation and related disciplines.
US2
A comprehensive knowledge and understanding of mathematical and computer models relevant to the engineering discipline, and an appreciation of their limitations.
Last modified: 29/07/2022 09:37
Engineering Tripos Part IIB, 4F7: Statistical Signal Analysis, 2017-18
Module Leader
Lecturer
Timing and Structure
Michaelmas term. 16 lectures (including examples classes). Assessment: 100% exam
Prerequisites
3F3; Useful 3F1 and 3F8
Aims
The aims of the course are to:
- Continue the study of statistical signal processing from the basics studied in 3F3.
- Introduce the fundamental concepts and methods of adaptive filtering.
- Introduce time-series models, in particular Hidden Markov Models; understand their role in applications of signal processing; develop techniques for estimating hidden signals from noisy observations.
- Develop techniques for calibrating statistical time-series models for real data.
Objectives
As specific objectives, by the end of the course students should be able to:
- Understand the theory and objectives of optimal filtering in an adaptive setting.
- Recognise and describe the classes of problem where adaptive filtering might be applied.
- Describe the implementation of the Least Mean Square and its variants, and understand their convergence properties.
- Understand the basic principles of Kalman filtering and filtering for hidden Markov models.
- Understand the principles of Sequential Importance Sampling with Resampling, aslo known as particle filtering, for inference in hidden Markov models.
- Undertand Maximum Likelihood estimation for model calibration and its implementation.
- Formulate signal processing tasks in a model-based framework, and to estimate the model parameters.
Content
The first aim of the course is to introduce the fundamental concepts and methods of adaptive linear filtering, i.e. filters that are linear functions of the data, which attempt to adapt their parameters automatically on-line to the data at hand. Examples of this are echo cancellation in telephony or background noise cancellation. (This part of the course is an extension of the basic filter design material combined with the optimal filtering material from 3F3.) Optimality of these techniques require that the data generating processes satisfy certain stationarity assumptions. Modern filtering theory will then be introduced through state-space models that do not require any stationarity assumptions. State-space mode are thus far more general and more widely applicable to real data settings. An even more general model is the hidden Markov model which will be studied in detail. Inference aims for the hidden Markov model will be defined and exact computation of the probability laws will be addressed. In many applications though exact computation is not possible and the most successful technique to date that addresses this problem is a Monte Carlo method called sequential importance sampling with resampling, also known as particle filtering. The particle filter will be derived and applied to both inference and model calibration for time-series data.
- Optimal linear filtering: the least mean square algorithm and its variants; recursive least squares; exemplar problems in signal processing.
- Introduction to state-space models and the recursive optimal linear filtering; the Kalman filter.
- Introduction to hidden Markov models: definition; inference aims; exact computation of the filter.
- Importance sampling: introduction; weight degeneracy.
- Sequential importance sampling and resampling (also known as the particle filter): application to hidden Markov models; filtering; smoothing.
- Calibrating hidden Markov models: maximum likelihood estimation and its implementation
- Exemplar problems in Signal Processing
- Examples Papers
Booklists
Please see the Booklist for Group F Courses for references for this module.
Examination Guidelines
Please refer to Form & conduct of the examinations.
UK-SPEC
This syllabus contributes to the following areas of the UK-SPEC standard:
Toggle display of UK-SPEC areas.
GT1
Develop transferable skills that will be of value in a wide range of situations. These are exemplified by the Qualifications and Curriculum Authority Higher Level Key Skills and include problem solving, communication, and working with others, as well as the effective use of general IT facilities and information retrieval skills. They also include planning self-learning and improving performance, as the foundation for lifelong learning/CPD.
IA1
Apply appropriate quantitative science and engineering tools to the analysis of problems.
IA2
Demonstrate creative and innovative ability in the synthesis of solutions and in formulating designs.
KU1
Demonstrate knowledge and understanding of essential facts, concepts, theories and principles of their engineering discipline, and its underpinning science and mathematics.
KU2
Have an appreciation of the wider multidisciplinary engineering context and its underlying principles.
E1
Ability to use fundamental knowledge to investigate new and emerging technologies.
E2
Ability to extract data pertinent to an unfamiliar problem, and apply its solution using computer based engineering tools when appropriate.
E3
Ability to apply mathematical and computer based models for solving problems in engineering, and the ability to assess the limitations of particular cases.
E4
Understanding of and ability to apply a systems approach to engineering problems.
P1
A thorough understanding of current practice and its limitations and some appreciation of likely new developments.
P3
Understanding of contexts in which engineering knowledge can be applied (e.g. operations and management, technology, development, etc).
P8
Ability to apply engineering techniques taking account of a range of commercial and industrial constraints.
US1
A comprehensive understanding of the scientific principles of own specialisation and related disciplines.
US2
A comprehensive knowledge and understanding of mathematical and computer models relevant to the engineering discipline, and an appreciation of their limitations.
Last modified: 16/09/2017 11:16
Engineering Tripos Part IIB, 4F7: Statistical signal and network models, 2024-25
Module Leader
lecturers
Timing and Structure
Lent term. 16 lectures (including examples classes). Assessment: 100% exam
Prerequisites
3F1, 3F3, 3F8 recommended. 3M1 useful.
Aims
The aims of the course are to:
- Introduce time-series models, in particular State-space models and hidden Markov models; understand their role in applications of signal processing.
- Develop techniques for fitting statisical models to data and estimating hidden signals from noisy observations.
- Introduce network models, graph algorithms, and techniques to analyse large scale relational data
Objectives
As specific objectives, by the end of the course students should be able to:
- Understand state-space models, hidden Markov models, and network models including their mathematical characterisation, strengths and limitations.
- Execute the necessary computational tasks involved in fitting the models to data, to estimate unobserved quantities and make future predictions.
- Understand the computational methodology employed, their mathematical derivation, their strengths and weaknesses, how to execute them, and their use more generally in Statistical and data-centric engineering problems.
Content
This course is about fitting statistical models to data that arrives sequentially over time and across network structures. Once an appropriate model has been fitted, tasks such as predicting future trends or estimating quantities not directly observed can be performed. The statistical modelling and computational methodology covered by this course is widely used in many applied areas. For example, data that arrives sequentially over time is a common occurrence in Signal Processing (Engineering), Finance, Machine Learning, Environmental statistics etc.
The model that most appropriately describes data that arrives sequentially over time is a time-series model, an example of which is the ARMA model (studied in 3F3.) However, this course will look at more versatile models that incorporate hidden or latent state variables as these are able to account for richer behaviour. Also, models that aim describe how many really physical processes evolve over time often necessarily have to incorporate unobserved hidden states that form a Markov process.
Besides changing over time, many data are distributed over different individuals or sites. For example, users of a social media platform will each have different properties. Networks (graphs) provide a simple formalism to analyse distributed systems composed of small but interrelated parts.
State space models and time series:
The model that most appropriately describes data that arrives sequentially over time is a time-series model, an example of which is the AR model (studied in 3F3). However, this course will look at more versatile stochastic (random) state-space models that incorporate hidden or latent state variables as these are able to account for richer behaviour. Also, models that aim describe how many really physical processes evolve over time often necessarily have to incorporate unobserved hidden states that form a Markov process.
-
Introduction to state-space models and optimal linear filtering; the Kalman filter; exemplar problems in signal processing.
-
Introduction to hidden Markov models: definition; inference/estimation aims; exact computation of the conditional probability distributions.
-
Importance sampling: introduction; weight degeneracy; statistical properties.
-
Sequential importance sampling and resampling (also known as the particle filter): application to hidden Markov models; filtering; smoothing.
-
Calibrating hidden Markov models: maximum likelihood estimation and its implementation.
-
Exemplar problems in Signal Processing.
-
Examples Papers.
Networks modelling:
Besides changing over time, many data are distributed over different individuals or sites. For example, users of a social media platform will each have different properties. Networks (graphs) provide a simple formalism to analyse distributed systems composed of small but interrelated parts. We will cover:
-
· Fundamentals, basic graph theory and algorithms.
-
· Metrics: centrality (e.g. PageRank), assortativity, clustering, diameter ("six degrees of separation").
-
· Models of networks: Erdős–Rényi, small-world and scale free.
-
· Models on networks: Spreading, reliability and percolation.
-
· Community detection and stochastic block models.
-
· Graph spectra and their applications.
Booklists
Please refer to the Booklist for Part IIB Courses for references to this module, this can be found on the associated Moodle course.
Examination Guidelines
Please refer to Form & conduct of the examinations.
UK-SPEC
This syllabus contributes to the following areas of the UK-SPEC standard:
Toggle display of UK-SPEC areas.
GT1
Develop transferable skills that will be of value in a wide range of situations. These are exemplified by the Qualifications and Curriculum Authority Higher Level Key Skills and include problem solving, communication, and working with others, as well as the effective use of general IT facilities and information retrieval skills. They also include planning self-learning and improving performance, as the foundation for lifelong learning/CPD.
IA1
Apply appropriate quantitative science and engineering tools to the analysis of problems.
IA2
Demonstrate creative and innovative ability in the synthesis of solutions and in formulating designs.
KU1
Demonstrate knowledge and understanding of essential facts, concepts, theories and principles of their engineering discipline, and its underpinning science and mathematics.
KU2
Have an appreciation of the wider multidisciplinary engineering context and its underlying principles.
E1
Ability to use fundamental knowledge to investigate new and emerging technologies.
E2
Ability to extract data pertinent to an unfamiliar problem, and apply its solution using computer based engineering tools when appropriate.
E3
Ability to apply mathematical and computer based models for solving problems in engineering, and the ability to assess the limitations of particular cases.
E4
Understanding of and ability to apply a systems approach to engineering problems.
P1
A thorough understanding of current practice and its limitations and some appreciation of likely new developments.
P3
Understanding of contexts in which engineering knowledge can be applied (e.g. operations and management, technology, development, etc).
P8
Ability to apply engineering techniques taking account of a range of commercial and industrial constraints.
US1
A comprehensive understanding of the scientific principles of own specialisation and related disciplines.
US2
A comprehensive knowledge and understanding of mathematical and computer models relevant to the engineering discipline, and an appreciation of their limitations.
Last modified: 18/09/2024 13:21
Engineering Tripos Part IIB, 4F7: Statistical Signal Analysis, 2020-21
Module Leader
Lecturer
Timing and Structure
Michaelmas term. 16 lectures (including examples classes). Assessment: 100% exam
Prerequisites
3F3; Useful 3F1 and 3F8
Aims
The aims of the course are to:
- Continue the study of statistical signal processing techniques from the basics studied in 3F3.
- Introduce time-series models, in particular State-space models and hidden Markov models; understand their role in applications of signal processing.
- Develop techniques for fitting statisical modes to data and estimating hidden signals from noisy observations.
Objectives
As specific objectives, by the end of the course students should be able to:
- Understand state-space models and hidden Markov models including their mathematical characterisation, strengths and limitations.
- Understand how to execute all the necessary computational tasks involved in fitting the models to data, to estimate unobserved quantities and make future predictions.
- Understand the computational methodology employed, their mathematical derivation, their strengths and weaknesses, how to execute them, and their use more generally in Statistical and data-centric engineering problems.
Content
This course is about fitting statistical models to data that arrives sequentially over time. Once an appropriate model has been fit, tasks like predicting future trends or estimating quantities not directly observed can be performed. The statistical modelling and computational methodology covered by this course is widely used in many applied areas. For example, data that arrives sequentially over time is a common occurrence in Signal Processing (Engineering), Finance, Machine Learning, Environmental statistics etc.
The model that most appropriately describes data that arrives sequentially over time is a time-series model, an example of which is the ARMA model (studied in 3F3.) However, this course will look at more versatile models that incorporate hidden or latent state variables as these are able to account for richer behaviour. Also, models that aim describe how many really physical processes evolve over time often necessarily have to incorporate unobserved hidden states that form a Markov process.
- Introduction to state-space models and optimal linear filtering; the Kalman filter; exemplar problems in signal processing.
- Introduction to hidden Markov models: definition; inference/estimation aims; exact computation of the conditional probability distributions.
- Importance sampling: introduction; weight degeneracy; statistical properties.
- Sequential importance sampling and resampling (also known as the particle filter): application to hidden Markov models; filtering; smoothing.
- Calibrating hidden Markov models: maximum likelihood estimation and its implementation.
- Exemplar problems in Signal Processing.
- Examples Papers.
Booklists
Please refer to the Booklist for Part IIB Courses for references to this module, this can be found on the associated Moodle course.
Examination Guidelines
Please refer to Form & conduct of the examinations.
UK-SPEC
This syllabus contributes to the following areas of the UK-SPEC standard:
Toggle display of UK-SPEC areas.
GT1
Develop transferable skills that will be of value in a wide range of situations. These are exemplified by the Qualifications and Curriculum Authority Higher Level Key Skills and include problem solving, communication, and working with others, as well as the effective use of general IT facilities and information retrieval skills. They also include planning self-learning and improving performance, as the foundation for lifelong learning/CPD.
IA1
Apply appropriate quantitative science and engineering tools to the analysis of problems.
IA2
Demonstrate creative and innovative ability in the synthesis of solutions and in formulating designs.
KU1
Demonstrate knowledge and understanding of essential facts, concepts, theories and principles of their engineering discipline, and its underpinning science and mathematics.
KU2
Have an appreciation of the wider multidisciplinary engineering context and its underlying principles.
E1
Ability to use fundamental knowledge to investigate new and emerging technologies.
E2
Ability to extract data pertinent to an unfamiliar problem, and apply its solution using computer based engineering tools when appropriate.
E3
Ability to apply mathematical and computer based models for solving problems in engineering, and the ability to assess the limitations of particular cases.
E4
Understanding of and ability to apply a systems approach to engineering problems.
P1
A thorough understanding of current practice and its limitations and some appreciation of likely new developments.
P3
Understanding of contexts in which engineering knowledge can be applied (e.g. operations and management, technology, development, etc).
P8
Ability to apply engineering techniques taking account of a range of commercial and industrial constraints.
US1
A comprehensive understanding of the scientific principles of own specialisation and related disciplines.
US2
A comprehensive knowledge and understanding of mathematical and computer models relevant to the engineering discipline, and an appreciation of their limitations.
Last modified: 01/09/2020 10:38
Pages
