Engineering Tripos Part IIB, 4F7: Statistical Signal Analysis, 2020-21
Module Leader
Lecturer
Timing and Structure
Michaelmas term. 16 lectures (including examples classes). Assessment: 100% exam
Prerequisites
3F3; Useful 3F1 and 3F8
Aims
The aims of the course are to:
- Continue the study of statistical signal processing techniques from the basics studied in 3F3.
- Introduce time-series models, in particular State-space models and hidden Markov models; understand their role in applications of signal processing.
- Develop techniques for fitting statisical modes to data and estimating hidden signals from noisy observations.
Objectives
As specific objectives, by the end of the course students should be able to:
- Understand state-space models and hidden Markov models including their mathematical characterisation, strengths and limitations.
- Understand how to execute all the necessary computational tasks involved in fitting the models to data, to estimate unobserved quantities and make future predictions.
- Understand the computational methodology employed, their mathematical derivation, their strengths and weaknesses, how to execute them, and their use more generally in Statistical and data-centric engineering problems.
Content
This course is about fitting statistical models to data that arrives sequentially over time. Once an appropriate model has been fit, tasks like predicting future trends or estimating quantities not directly observed can be performed. The statistical modelling and computational methodology covered by this course is widely used in many applied areas. For example, data that arrives sequentially over time is a common occurrence in Signal Processing (Engineering), Finance, Machine Learning, Environmental statistics etc.
The model that most appropriately describes data that arrives sequentially over time is a time-series model, an example of which is the ARMA model (studied in 3F3.) However, this course will look at more versatile models that incorporate hidden or latent state variables as these are able to account for richer behaviour. Also, models that aim describe how many really physical processes evolve over time often necessarily have to incorporate unobserved hidden states that form a Markov process.
- Introduction to state-space models and optimal linear filtering; the Kalman filter; exemplar problems in signal processing.
- Introduction to hidden Markov models: definition; inference/estimation aims; exact computation of the conditional probability distributions.
- Importance sampling: introduction; weight degeneracy; statistical properties.
- Sequential importance sampling and resampling (also known as the particle filter): application to hidden Markov models; filtering; smoothing.
- Calibrating hidden Markov models: maximum likelihood estimation and its implementation.
- Exemplar problems in Signal Processing.
- Examples Papers.
Booklists
Please refer to the Booklist for Part IIB Courses for references to this module, this can be found on the associated Moodle course.
Examination Guidelines
Please refer to Form & conduct of the examinations.
UK-SPEC
This syllabus contributes to the following areas of the UK-SPEC standard:
Toggle display of UK-SPEC areas.
GT1
Develop transferable skills that will be of value in a wide range of situations. These are exemplified by the Qualifications and Curriculum Authority Higher Level Key Skills and include problem solving, communication, and working with others, as well as the effective use of general IT facilities and information retrieval skills. They also include planning self-learning and improving performance, as the foundation for lifelong learning/CPD.
IA1
Apply appropriate quantitative science and engineering tools to the analysis of problems.
IA2
Demonstrate creative and innovative ability in the synthesis of solutions and in formulating designs.
KU1
Demonstrate knowledge and understanding of essential facts, concepts, theories and principles of their engineering discipline, and its underpinning science and mathematics.
KU2
Have an appreciation of the wider multidisciplinary engineering context and its underlying principles.
E1
Ability to use fundamental knowledge to investigate new and emerging technologies.
E2
Ability to extract data pertinent to an unfamiliar problem, and apply its solution using computer based engineering tools when appropriate.
E3
Ability to apply mathematical and computer based models for solving problems in engineering, and the ability to assess the limitations of particular cases.
E4
Understanding of and ability to apply a systems approach to engineering problems.
P1
A thorough understanding of current practice and its limitations and some appreciation of likely new developments.
P3
Understanding of contexts in which engineering knowledge can be applied (e.g. operations and management, technology, development, etc).
P8
Ability to apply engineering techniques taking account of a range of commercial and industrial constraints.
US1
A comprehensive understanding of the scientific principles of own specialisation and related disciplines.
US2
A comprehensive knowledge and understanding of mathematical and computer models relevant to the engineering discipline, and an appreciation of their limitations.
Last modified: 01/09/2020 10:38
Engineering Tripos Part IIB, 4F5: Advanced Information Theory and Coding, 2019-20
Module Leader
Lecturer
Prof I Kontoyiannis and Dr J Sayir
Timing and Structure
Lent term. 16 lectures. Assessment: 100% exam
Prerequisites
3F7 assumed, 3F1, 3F4 useful but not necessary
Aims
The aims of the course are to:
- Learn about applications of information theory to universal data compression, statistics and inference
- Introduce students to the principles of algebraic coding and Reed Solomon coding in particular
- Give students an overview of cryptology with example of techniques that share the same mathematical background as algebraic coding.
Objectives
As specific objectives, by the end of the course students should be able to:
- have gained an appreciation for the connection between information-theoretic concepts and fundamental problems in statistics
- learnt some core information-theoretical tools that can be used in probability and statistics
- have a good understanding of the foundations of the problem of universal data compression
- know and be able to use the basic results in large deviations theory, especially as applied in information theory and communications
- have gained a practical understanding of the algebraic fundamentals that underlie channel coding and cryptology
- understand the properties of linear block codes over finite fields
- be able to implement encoders and decoders for Reed Solomon codes
- have gained an overview of methods and aims in cryptology (including cryptography, crypt- analysis, secrecy, authenticity)
- be familiar with one example each of a block cipher and a stream cipher
- be able to implement public key cryptosystems, in particular the Diffie-Hellman and Rivest- Shamir-Adleman (RSA) systems
Content
-
This course will introduce students to applications of information theory and coding theory in statistics, information storage, and cryptography.
The first part of the course will discuss applications of information theory to universal data compression, statistics, and inference.
The second part of the course will expand linear coding principles acquired in 3F7 to non-binary codes over finite fields. After establish the algebraic fundamentals, we will cover Reed-Solomon coding, a technique used in a wide range of communication and storage systems (hard disks, blu ray discs, QR codes, USB mass storage device class, DNA storage, and others.)
The final part of the course will introduce the discipline of cryptology, which includes cryptography, the essential art of ensuring secrecy and authenticity, and cryptanalysis, the dark art of breaking that secrecy. The course will cover a number of methods to provide secrecy, ranging from mathematically provable secrecy to public key methods through which computationally secure communication links can be established over public channels.
Information theory and statistics (7-9L, Prof. Ioannis Kontoyiannis)
- Source coding, probability of error, error exponents
- Method of types, error rates in data compression and hypothesis testing
- Fundamental limits of estimation and hypothesis testing: The Cram ́er-Rao bound, Chernoff information, Neyman-Pearson tests, Stein’s lemma, strong converses
- Large deviations: Cram ́er’s theorem, Sanov’s theorem, the conditional limit theorem
- Entropy and Poisson approximation
- Universal source coding: The capacity-redundancy theorem, the price of universality, Rissanen’s lower bound
Introduction to practical number theory and algebra (2-3L, Dr Jossy Sayir)
- Elementary number theory
- Groups and fields, extension fields
- 3 equivalent approaches to multiplication in extension fields
- Matrix operations and the Discrete Fourier Transform
Algebraic Coding (3L, Dr Jossy Sayir)
- Linear coding and the Singleton Bound
- Distance profiles and MacWilliams Identities
- Blahut’s theorem
- Reed Solomon (RS) codes
- Encoding and decoding of RS codes
Introduction to Cryptology (2-3L, Dr Jossy Sayir )
- Overview of cryptology
- Stream ciphers, examples
- Block ciphers, examples
- Public key cryptography, basic techniques
Further notes
Examples papers
3 examples papers:
- Information theory & data compression
- Number theory and algebra
- Coding & Cryptology
Coursework
none
Booklists
- Information Theory:
- Elements of Information Theory, T. M. Cover & J. A. Thomas, Wiley-Interscience, 2nd Ed, 2006.
- Information Theory: Coding Theorems for Discrete Memoryless Systems, I. Csiszàr & J. Körner, Cambridge University Press, 2nd Ed. 2011.
- Coding theory:
- The Theory of Error-Correcting Codes, F. J. MacWilliams & N. J. A. Sloane, North Holland.
- Algebraic Codes for Data Transmission, Richard E. Blahut, Cambridge University Press, 2003 (Online 2012)
Please see the Booklist for Group F Courses for library holdings.
Examination Guidelines
Please refer to Form & conduct of the examinations.
UK-SPEC
This syllabus contributes to the following areas of the UK-SPEC standard:
Toggle display of UK-SPEC areas.
GT1
Develop transferable skills that will be of value in a wide range of situations. These are exemplified by the Qualifications and Curriculum Authority Higher Level Key Skills and include problem solving, communication, and working with others, as well as the effective use of general IT facilities and information retrieval skills. They also include planning self-learning and improving performance, as the foundation for lifelong learning/CPD.
IA1
Apply appropriate quantitative science and engineering tools to the analysis of problems.
IA2
Demonstrate creative and innovative ability in the synthesis of solutions and in formulating designs.
KU1
Demonstrate knowledge and understanding of essential facts, concepts, theories and principles of their engineering discipline, and its underpinning science and mathematics.
KU2
Have an appreciation of the wider multidisciplinary engineering context and its underlying principles.
D1
Wide knowledge and comprehensive understanding of design processes and methodologies and the ability to apply and adapt them in unfamiliar situations.
E1
Ability to use fundamental knowledge to investigate new and emerging technologies.
E3
Ability to apply mathematical and computer based models for solving problems in engineering, and the ability to assess the limitations of particular cases.
E4
Understanding of and ability to apply a systems approach to engineering problems.
P1
A thorough understanding of current practice and its limitations and some appreciation of likely new developments.
P3
Understanding of contexts in which engineering knowledge can be applied (e.g. operations and management, technology, development, etc).
US1
A comprehensive understanding of the scientific principles of own specialisation and related disciplines.
US4
An awareness of developing technologies related to own specialisation.
Last modified: 28/05/2019 16:04
Engineering Tripos Part IIB, 4F5: Advanced Information Theory and Coding, 2025-26
Module Leader
Lecturer
Prof A Guillen i Fabregas and Dr Jossy Sayir
Timing and Structure
Lent term. 16 lectures. Assessment: 100% exam
Prerequisites
3F7 assumed, 3F1, 3F4 useful but not necessary
Aims
The aims of the course are to:
- Learn about applications of information theory to hypothesis testing as well as refinements of source and channel coding theorems through error exponents.
- Introduce students to the principles of algebraic coding and Reed Solomon coding in particular
- Give students an overview of cryptology with example of techniques that share the same mathematical background as algebraic coding.
Objectives
As specific objectives, by the end of the course students should be able to:
- have gained an appreciation for the connection between information-theoretic concepts and fundamental problems in statistics
- have a good understanding of the derivations of error exponents for data compression and transmission
- have a good understanding of the fundamental connections between hypothesis testing and information theory
- have gained a practical understanding of the algebraic fundamentals that underlie channel coding and cryptology
- understand the properties of linear block codes over finite fields
- be able to implement encoders and decoders for Reed Solomon codes
- have gained an overview of methods and aims in cryptology (including cryptography, crypt- analysis, secrecy, authenticity)
- be familiar with one example each of a block cipher and a stream cipher
- be able to implement public key cryptosystems, in particular the Diffie-Hellman and Rivest- Shamir-Adleman (RSA) systems
Content
-
This course will introduce students to applications of information theory and coding theory in statistics, information storage, and cryptography.
The first part of the course will discuss applications of information theory to universal data compression, statistics, and inference.
The second part of the course will expand linear coding principles acquired in 3F7 to non-binary codes over finite fields. After establishing the algebraic fundamentals, we will cover Reed-Solomon coding, a technique used in a wide range of communication and storage systems (hard disks, blu ray discs, QR codes, USB mass storage device class, DNA storage, and others.)
The final part of the course will introduce the discipline of cryptology, which includes cryptography, the essential art of ensuring secrecy and authenticity, and cryptanalysis, the dark art of breaking that secrecy. The course will cover a number of methods to provide secrecy, ranging from mathematically provable secrecy to public key methods through which computationally secure communication links can be established over public channels.
Information theory and statistics (7-9L, Prof Albert Guillén i Fàbregas)
- Source coding, optimum fixed-rate coding, error exponents
- Binary hypothesis testing, probability of error, error exponents, Stein's lemma
- M-ary hypothesis testing, probability of error
- Channel coding, connection with hypothesis testing, perfect codes, error exponents
Introduction to practical number theory and algebra (2-3L, Dr Jossy Sayir)
- Elementary number theory
- Groups and fields, extension fields
- 3 equivalent approaches to multiplication in extension fields
- Matrix operations and the Discrete Fourier Transform
Algebraic Coding (3L, Dr Jossy Sayir)
- Linear coding and the Singleton Bound
- Distance profiles and MacWilliams Identities
- Blahut’s theorem
- Reed Solomon (RS) codes
- Encoding and decoding of RS codes
Introduction to Cryptology (2L, Dr Jossy Sayir )
- Overview of cryptology
- Stream ciphers, examples
- Block ciphers, examples
- Public key cryptography, basic techniques
Further notes
Examples papers
Examples papers consist of a recommended list of problems to solve in the lecture notes.
Coursework
none
Booklists
- Information Theory:
- Elements of Information Theory, T. M. Cover & J. A. Thomas, Wiley-Interscience, 2nd Ed, 2006.
- Information Theory: Coding Theorems for Discrete Memoryless Systems, I. Csiszàr & J. Körner, Cambridge University Press, 2nd Ed. 2011.
- Coding theory:
- The Theory of Error-Correcting Codes, F. J. MacWilliams & N. J. A. Sloane, North Holland.
- Algebraic Codes for Data Transmission, Richard E. Blahut, Cambridge University Press, 2003 (Online 2012)
Please refer to the Booklist for Part IIB Courses for references to this module, this can be found on the associated Moodle course.
Examination Guidelines
Please refer to Form & conduct of the examinations.
UK-SPEC
This syllabus contributes to the following areas of the UK-SPEC standard:
Toggle display of UK-SPEC areas.
GT1
Develop transferable skills that will be of value in a wide range of situations. These are exemplified by the Qualifications and Curriculum Authority Higher Level Key Skills and include problem solving, communication, and working with others, as well as the effective use of general IT facilities and information retrieval skills. They also include planning self-learning and improving performance, as the foundation for lifelong learning/CPD.
IA1
Apply appropriate quantitative science and engineering tools to the analysis of problems.
IA2
Demonstrate creative and innovative ability in the synthesis of solutions and in formulating designs.
KU1
Demonstrate knowledge and understanding of essential facts, concepts, theories and principles of their engineering discipline, and its underpinning science and mathematics.
KU2
Have an appreciation of the wider multidisciplinary engineering context and its underlying principles.
D1
Wide knowledge and comprehensive understanding of design processes and methodologies and the ability to apply and adapt them in unfamiliar situations.
E1
Ability to use fundamental knowledge to investigate new and emerging technologies.
E3
Ability to apply mathematical and computer based models for solving problems in engineering, and the ability to assess the limitations of particular cases.
E4
Understanding of and ability to apply a systems approach to engineering problems.
P1
A thorough understanding of current practice and its limitations and some appreciation of likely new developments.
P3
Understanding of contexts in which engineering knowledge can be applied (e.g. operations and management, technology, development, etc).
US1
A comprehensive understanding of the scientific principles of own specialisation and related disciplines.
US4
An awareness of developing technologies related to own specialisation.
Last modified: 04/06/2025 13:30
Engineering Tripos Part IIB, 4F5: Advanced Information Theory and Coding, 2023-24
Module Leader
Lecturer
Prof A Guillen i Fabregas and Dr Jossy Sayir
Timing and Structure
Lent term. 16 lectures. Assessment: 100% exam
Prerequisites
3F7 assumed, 3F1, 3F4 useful but not necessary
Aims
The aims of the course are to:
- Learn about applications of information theory to hypothesis testing as well as refinements of source and channel coding theorems through error exponents.
- Introduce students to the principles of algebraic coding and Reed Solomon coding in particular
- Give students an overview of cryptology with example of techniques that share the same mathematical background as algebraic coding.
Objectives
As specific objectives, by the end of the course students should be able to:
- have gained an appreciation for the connection between information-theoretic concepts and fundamental problems in statistics
- have a good understanding of the derivations of error exponents for data compression and transmission
- have a good understanding of the fundamental connections between hypothesis testing and information theory
- have gained a practical understanding of the algebraic fundamentals that underlie channel coding and cryptology
- understand the properties of linear block codes over finite fields
- be able to implement encoders and decoders for Reed Solomon codes
- have gained an overview of methods and aims in cryptology (including cryptography, crypt- analysis, secrecy, authenticity)
- be familiar with one example each of a block cipher and a stream cipher
- be able to implement public key cryptosystems, in particular the Diffie-Hellman and Rivest- Shamir-Adleman (RSA) systems
Content
-
This course will introduce students to applications of information theory and coding theory in statistics, information storage, and cryptography.
The first part of the course will discuss applications of information theory to universal data compression, statistics, and inference.
The second part of the course will expand linear coding principles acquired in 3F7 to non-binary codes over finite fields. After establishing the algebraic fundamentals, we will cover Reed-Solomon coding, a technique used in a wide range of communication and storage systems (hard disks, blu ray discs, QR codes, USB mass storage device class, DNA storage, and others.)
The final part of the course will introduce the discipline of cryptology, which includes cryptography, the essential art of ensuring secrecy and authenticity, and cryptanalysis, the dark art of breaking that secrecy. The course will cover a number of methods to provide secrecy, ranging from mathematically provable secrecy to public key methods through which computationally secure communication links can be established over public channels.
Information theory and statistics (7-9L, Prof Albert Guillén i Fàbregas)
- Source coding, optimum fixed-rate coding, error exponents
- Binary hypothesis testing, probability of error, error exponents, Stein's lemma
- M-ary hypothesis testing, probability of error
- Channel coding, connection with hypothesis testing, perfect codes, error exponents
Introduction to practical number theory and algebra (2-3L, Dr Jossy Sayir)
- Elementary number theory
- Groups and fields, extension fields
- 3 equivalent approaches to multiplication in extension fields
- Matrix operations and the Discrete Fourier Transform
Algebraic Coding (3L, Dr Jossy Sayir)
- Linear coding and the Singleton Bound
- Distance profiles and MacWilliams Identities
- Blahut’s theorem
- Reed Solomon (RS) codes
- Encoding and decoding of RS codes
Introduction to Cryptology (2L, Dr Jossy Sayir )
- Overview of cryptology
- Stream ciphers, examples
- Block ciphers, examples
- Public key cryptography, basic techniques
Further notes
Examples papers
Examples papers consist of a recommended list of problems to solve in the lecture notes.
Coursework
none
Booklists
- Information Theory:
- Elements of Information Theory, T. M. Cover & J. A. Thomas, Wiley-Interscience, 2nd Ed, 2006.
- Information Theory: Coding Theorems for Discrete Memoryless Systems, I. Csiszàr & J. Körner, Cambridge University Press, 2nd Ed. 2011.
- Coding theory:
- The Theory of Error-Correcting Codes, F. J. MacWilliams & N. J. A. Sloane, North Holland.
- Algebraic Codes for Data Transmission, Richard E. Blahut, Cambridge University Press, 2003 (Online 2012)
Please refer to the Booklist for Part IIB Courses for references to this module, this can be found on the associated Moodle course.
Examination Guidelines
Please refer to Form & conduct of the examinations.
UK-SPEC
This syllabus contributes to the following areas of the UK-SPEC standard:
Toggle display of UK-SPEC areas.
GT1
Develop transferable skills that will be of value in a wide range of situations. These are exemplified by the Qualifications and Curriculum Authority Higher Level Key Skills and include problem solving, communication, and working with others, as well as the effective use of general IT facilities and information retrieval skills. They also include planning self-learning and improving performance, as the foundation for lifelong learning/CPD.
IA1
Apply appropriate quantitative science and engineering tools to the analysis of problems.
IA2
Demonstrate creative and innovative ability in the synthesis of solutions and in formulating designs.
KU1
Demonstrate knowledge and understanding of essential facts, concepts, theories and principles of their engineering discipline, and its underpinning science and mathematics.
KU2
Have an appreciation of the wider multidisciplinary engineering context and its underlying principles.
D1
Wide knowledge and comprehensive understanding of design processes and methodologies and the ability to apply and adapt them in unfamiliar situations.
E1
Ability to use fundamental knowledge to investigate new and emerging technologies.
E3
Ability to apply mathematical and computer based models for solving problems in engineering, and the ability to assess the limitations of particular cases.
E4
Understanding of and ability to apply a systems approach to engineering problems.
P1
A thorough understanding of current practice and its limitations and some appreciation of likely new developments.
P3
Understanding of contexts in which engineering knowledge can be applied (e.g. operations and management, technology, development, etc).
US1
A comprehensive understanding of the scientific principles of own specialisation and related disciplines.
US4
An awareness of developing technologies related to own specialisation.
Last modified: 28/07/2023 11:26
Engineering Tripos Part IIB, 4F5: Advanced Information Theory and Coding, 2022-23
Module Leader
Lecturer
Dr A Guillen i Fabregas and Dr Jossy Sayir
Timing and Structure
Lent term. 16 lectures. Assessment: 100% exam
Prerequisites
3F7 assumed, 3F1, 3F4 useful but not necessary
Aims
The aims of the course are to:
- Learn about applications of information theory to hypothesis testing as well as refinements of source and channel coding theorems through error exponents.
- Introduce students to the principles of algebraic coding and Reed Solomon coding in particular
- Give students an overview of cryptology with example of techniques that share the same mathematical background as algebraic coding.
Objectives
As specific objectives, by the end of the course students should be able to:
- have gained an appreciation for the connection between information-theoretic concepts and fundamental problems in statistics
- have a good understanding of the derivations of error exponents for data compression and transmission
- have a good understanding of the fundamental connections between hypothesis testing and information theory
- have gained a practical understanding of the algebraic fundamentals that underlie channel coding and cryptology
- understand the properties of linear block codes over finite fields
- be able to implement encoders and decoders for Reed Solomon codes
- have gained an overview of methods and aims in cryptology (including cryptography, crypt- analysis, secrecy, authenticity)
- be familiar with one example each of a block cipher and a stream cipher
- be able to implement public key cryptosystems, in particular the Diffie-Hellman and Rivest- Shamir-Adleman (RSA) systems
Content
-
This course will introduce students to applications of information theory and coding theory in statistics, information storage, and cryptography.
The first part of the course will discuss applications of information theory to universal data compression, statistics, and inference.
The second part of the course will expand linear coding principles acquired in 3F7 to non-binary codes over finite fields. After establishing the algebraic fundamentals, we will cover Reed-Solomon coding, a technique used in a wide range of communication and storage systems (hard disks, blu ray discs, QR codes, USB mass storage device class, DNA storage, and others.)
The final part of the course will introduce the discipline of cryptology, which includes cryptography, the essential art of ensuring secrecy and authenticity, and cryptanalysis, the dark art of breaking that secrecy. The course will cover a number of methods to provide secrecy, ranging from mathematically provable secrecy to public key methods through which computationally secure communication links can be established over public channels.
Information theory and statistics (7-9L, Dr Albert Guillén i Fàbregas)
- Source coding, optimum fixed-rate coding, error exponents
- Binary hypothesis testing, probability of error, error exponents, Stein's lemma
- M-ary hypothesis testing, probability of error
- Channel coding, connection with hypothesis testing, perfect codes, error exponents
Introduction to practical number theory and algebra (2-3L, Dr Jossy Sayir)
- Elementary number theory
- Groups and fields, extension fields
- 3 equivalent approaches to multiplication in extension fields
- Matrix operations and the Discrete Fourier Transform
Algebraic Coding (3L, Dr Jossy Sayir)
- Linear coding and the Singleton Bound
- Distance profiles and MacWilliams Identities
- Blahut’s theorem
- Reed Solomon (RS) codes
- Encoding and decoding of RS codes
Introduction to Cryptology (2L, Dr Jossy Sayir )
- Overview of cryptology
- Stream ciphers, examples
- Block ciphers, examples
- Public key cryptography, basic techniques
Further notes
Examples papers
Examples papers consist of a recommended list of problems to solve in the lecture notes.
Coursework
none
Booklists
- Information Theory:
- Elements of Information Theory, T. M. Cover & J. A. Thomas, Wiley-Interscience, 2nd Ed, 2006.
- Information Theory: Coding Theorems for Discrete Memoryless Systems, I. Csiszàr & J. Körner, Cambridge University Press, 2nd Ed. 2011.
- Coding theory:
- The Theory of Error-Correcting Codes, F. J. MacWilliams & N. J. A. Sloane, North Holland.
- Algebraic Codes for Data Transmission, Richard E. Blahut, Cambridge University Press, 2003 (Online 2012)
Please refer to the Booklist for Part IIB Courses for references to this module, this can be found on the associated Moodle course.
Examination Guidelines
Please refer to Form & conduct of the examinations.
UK-SPEC
This syllabus contributes to the following areas of the UK-SPEC standard:
Toggle display of UK-SPEC areas.
GT1
Develop transferable skills that will be of value in a wide range of situations. These are exemplified by the Qualifications and Curriculum Authority Higher Level Key Skills and include problem solving, communication, and working with others, as well as the effective use of general IT facilities and information retrieval skills. They also include planning self-learning and improving performance, as the foundation for lifelong learning/CPD.
IA1
Apply appropriate quantitative science and engineering tools to the analysis of problems.
IA2
Demonstrate creative and innovative ability in the synthesis of solutions and in formulating designs.
KU1
Demonstrate knowledge and understanding of essential facts, concepts, theories and principles of their engineering discipline, and its underpinning science and mathematics.
KU2
Have an appreciation of the wider multidisciplinary engineering context and its underlying principles.
D1
Wide knowledge and comprehensive understanding of design processes and methodologies and the ability to apply and adapt them in unfamiliar situations.
E1
Ability to use fundamental knowledge to investigate new and emerging technologies.
E3
Ability to apply mathematical and computer based models for solving problems in engineering, and the ability to assess the limitations of particular cases.
E4
Understanding of and ability to apply a systems approach to engineering problems.
P1
A thorough understanding of current practice and its limitations and some appreciation of likely new developments.
P3
Understanding of contexts in which engineering knowledge can be applied (e.g. operations and management, technology, development, etc).
US1
A comprehensive understanding of the scientific principles of own specialisation and related disciplines.
US4
An awareness of developing technologies related to own specialisation.
Last modified: 24/05/2022 12:53
Engineering Tripos Part IIB, 4F5: Advanced Information Theory and Coding, 2021-22
Module Leader
Lecturer
Timing and Structure
Lent term. 16 lectures. Assessment: 100% exam
Prerequisites
3F7 assumed, 3F1, 3F4 useful but not necessary
Aims
The aims of the course are to:
- Learn about applications of information theory to hypothesis testing as well as refinements of source and channel coding theorems through error exponents.
- Introduce students to the principles of algebraic coding and Reed Solomon coding in particular
- Give students an overview of cryptology with example of techniques that share the same mathematical background as algebraic coding.
Objectives
As specific objectives, by the end of the course students should be able to:
- have gained an appreciation for the connection between information-theoretic concepts and fundamental problems in statistics
- have a good understanding of the derivations of error exponents for data compression and transmission
- have a good understanding of the fundamental connections between hypothesis testing and information theory
- have gained a practical understanding of the algebraic fundamentals that underlie channel coding and cryptology
- understand the properties of linear block codes over finite fields
- be able to implement encoders and decoders for Reed Solomon codes
- have gained an overview of methods and aims in cryptology (including cryptography, crypt- analysis, secrecy, authenticity)
- be familiar with one example each of a block cipher and a stream cipher
- be able to implement public key cryptosystems, in particular the Diffie-Hellman and Rivest- Shamir-Adleman (RSA) systems
Content
-
This course will introduce students to applications of information theory and coding theory in statistics, information storage, and cryptography.
The first part of the course will discuss applications of information theory to universal data compression, statistics, and inference.
The second part of the course will expand linear coding principles acquired in 3F7 to non-binary codes over finite fields. After establishing the algebraic fundamentals, we will cover Reed-Solomon coding, a technique used in a wide range of communication and storage systems (hard disks, blu ray discs, QR codes, USB mass storage device class, DNA storage, and others.)
The final part of the course will introduce the discipline of cryptology, which includes cryptography, the essential art of ensuring secrecy and authenticity, and cryptanalysis, the dark art of breaking that secrecy. The course will cover a number of methods to provide secrecy, ranging from mathematically provable secrecy to public key methods through which computationally secure communication links can be established over public channels.
Information theory and statistics (7-9L, Dr Albert Guillén i Fàbregas)
- Source coding, optimum fixed-rate coding, error exponents
- Binary hypothesis testing, probability of error, error exponents, Stein's lemma
- M-ary hypothesis testing, probability of error
- Channel coding, connection with hypothesis testing, perfect codes, error exponents
Introduction to practical number theory and algebra (2-3L, Dr Jossy Sayir)
- Elementary number theory
- Groups and fields, extension fields
- 3 equivalent approaches to multiplication in extension fields
- Matrix operations and the Discrete Fourier Transform
Algebraic Coding (3L, Dr Jossy Sayir)
- Linear coding and the Singleton Bound
- Distance profiles and MacWilliams Identities
- Blahut’s theorem
- Reed Solomon (RS) codes
- Encoding and decoding of RS codes
Introduction to Cryptology (2L, Dr Jossy Sayir )
- Overview of cryptology
- Stream ciphers, examples
- Block ciphers, examples
- Public key cryptography, basic techniques
Further notes
Examples papers
Examples papers consist of a recommended list of problems to solve in the lecture notes.
Coursework
none
Booklists
- Information Theory:
- Elements of Information Theory, T. M. Cover & J. A. Thomas, Wiley-Interscience, 2nd Ed, 2006.
- Information Theory: Coding Theorems for Discrete Memoryless Systems, I. Csiszàr & J. Körner, Cambridge University Press, 2nd Ed. 2011.
- Coding theory:
- The Theory of Error-Correcting Codes, F. J. MacWilliams & N. J. A. Sloane, North Holland.
- Algebraic Codes for Data Transmission, Richard E. Blahut, Cambridge University Press, 2003 (Online 2012)
Please refer to the Booklist for Part IIB Courses for references to this module, this can be found on the associated Moodle course.
Examination Guidelines
Please refer to Form & conduct of the examinations.
UK-SPEC
This syllabus contributes to the following areas of the UK-SPEC standard:
Toggle display of UK-SPEC areas.
GT1
Develop transferable skills that will be of value in a wide range of situations. These are exemplified by the Qualifications and Curriculum Authority Higher Level Key Skills and include problem solving, communication, and working with others, as well as the effective use of general IT facilities and information retrieval skills. They also include planning self-learning and improving performance, as the foundation for lifelong learning/CPD.
IA1
Apply appropriate quantitative science and engineering tools to the analysis of problems.
IA2
Demonstrate creative and innovative ability in the synthesis of solutions and in formulating designs.
KU1
Demonstrate knowledge and understanding of essential facts, concepts, theories and principles of their engineering discipline, and its underpinning science and mathematics.
KU2
Have an appreciation of the wider multidisciplinary engineering context and its underlying principles.
D1
Wide knowledge and comprehensive understanding of design processes and methodologies and the ability to apply and adapt them in unfamiliar situations.
E1
Ability to use fundamental knowledge to investigate new and emerging technologies.
E3
Ability to apply mathematical and computer based models for solving problems in engineering, and the ability to assess the limitations of particular cases.
E4
Understanding of and ability to apply a systems approach to engineering problems.
P1
A thorough understanding of current practice and its limitations and some appreciation of likely new developments.
P3
Understanding of contexts in which engineering knowledge can be applied (e.g. operations and management, technology, development, etc).
US1
A comprehensive understanding of the scientific principles of own specialisation and related disciplines.
US4
An awareness of developing technologies related to own specialisation.
Last modified: 09/09/2021 11:05
Engineering Tripos Part IIB, 4F5: Advanced Information Theory and Coding, 2018-19
Leader
Lecturer
Dr J Sayir, Prof I Kontoyiannis
Timing and Structure
Lent term. 16 lectures. Assessment: 100% exam
Prerequisites
3F7 assumed, 3F1, 3F4 useful but not necessary
Aims
The aims of the course are to:
- Learn about applications of information theory to universal data compression, statistics and inference
- Introduce students to the principles of algebraic coding and Reed Solomon coding in particular
- Give students an overview of cryptology with example of techniques that share the same mathematical background as algebraic coding.
Objectives
As specific objectives, by the end of the course students should be able to:
- have gained an appreciation for the connection between information-theoretic concepts and fundamental problems in statistics
- learnt some core information-theoretical tools that can be used in probability and statistics
- have a good understanding of the foundations of the problem of universal data compression
- know and be able to use the basic results in large deviations theory, especially as applied in information theory and communications
- have gained a practical understanding of the algebraic fundamentals that underlie channel coding and cryptology
- understand the properties of linear block codes over finite fields
- be able to implement encoders and decoders for Reed Solomon codes
- have gained an overview of methods and aims in cryptology (including cryptography, crypt- analysis, secrecy, authenticity)
- be familiar with one example each of a block cipher and a stream cipher
- be able to implement public key cryptosystems, in particular the Diffie-Hellman and Rivest- Shamir-Adleman (RSA) systems
Content
-
This course will introduce students to applications of information theory and coding theory in statistics, information storage, and cryptography.
The first part of the course will discuss applications of information theory to universal data compression, statistics, and inference.
The second part of the course will expand linear coding principles acquired in 3F7 to non-binary codes over finite fields. After establish the algebraic fundamentals, we will cover Reed-Solomon coding, a technique used in a wide range of communication and storage systems (hard disks, blu ray discs, QR codes, USB mass storage device class, DNA storage, and others.)
The final part of the course will introduce the discipline of cryptology, which includes cryptography, the essential art of ensuring secrecy and authenticity, and cryptanalysis, the dark art of breaking that secrecy. The course will cover a number of methods to provide secrecy, ranging from mathematically provable secrecy to public key methods through which computationally secure communication links can be established over public channels.
Information theory and statistics (7-9L, Prof. Ioannis Kontoyiannis)
- Source coding, probability of error, error exponents
- Method of types, error rates in data compression and hypothesis testing
- Fundamental limits of estimation and hypothesis testing: The Cram ́er-Rao bound, Chernoff information, Neyman-Pearson tests, Stein’s lemma, strong converses
- Large deviations: Cram ́er’s theorem, Sanov’s theorem, the conditional limit theorem
- Entropy and Poisson approximation
- Universal source coding: The capacity-redundancy theorem, the price of universality, Rissanen’s lower bound
Introduction to practical number theory and algebra (2-3L, Dr Jossy Sayir)
- Elementary number theory
- Groups and fields, extension fields
- 3 equivalent approaches to multiplication in extension fields
- Matrix operations and the Discrete Fourier Transform
Algebraic Coding (3L, Dr Jossy Sayir)
- Linear coding and the Singleton Bound
- Distance profiles and MacWilliams Identities
- Blahut’s theorem
- Reed Solomon (RS) codes
- Encoding and decoding of RS codes
Introduction to Cryptology (2-3L, Dr Jossy Sayir )
- Overview of cryptology
- Stream ciphers, examples
- Block ciphers, examples
- Public key cryptography, basic techniques
Further notes
Examples papers
3 examples papers:
- Information theory & data compression
- Number theory and algebra
- Coding & Cryptology
Coursework
none
Booklists
- Information Theory:
- Elements of Information Theory, T. M. Cover & J. A. Thomas, Wiley-Interscience, 2nd Ed, 2006.
- Information Theory: Coding Theorems for Discrete Memoryless Systems, I. Csiszàr & J. Körner, Cambridge University Press, 2nd Ed. 2011.
- Coding theory:
- The Theory of Error-Correcting Codes, F. J. MacWilliams & N. J. A. Sloane, North Holland.
- Algebraic Codes for Data Transmission, Richard E. Blahut, Cambridge University Press, 2003 (Online 2012)
Please see the Booklist for Group F Courses for library holdings.
Examination Guidelines
Please refer to Form & conduct of the examinations.
UK-SPEC
This syllabus contributes to the following areas of the UK-SPEC standard:
Toggle display of UK-SPEC areas.
GT1
Develop transferable skills that will be of value in a wide range of situations. These are exemplified by the Qualifications and Curriculum Authority Higher Level Key Skills and include problem solving, communication, and working with others, as well as the effective use of general IT facilities and information retrieval skills. They also include planning self-learning and improving performance, as the foundation for lifelong learning/CPD.
IA1
Apply appropriate quantitative science and engineering tools to the analysis of problems.
IA2
Demonstrate creative and innovative ability in the synthesis of solutions and in formulating designs.
KU1
Demonstrate knowledge and understanding of essential facts, concepts, theories and principles of their engineering discipline, and its underpinning science and mathematics.
KU2
Have an appreciation of the wider multidisciplinary engineering context and its underlying principles.
D1
Wide knowledge and comprehensive understanding of design processes and methodologies and the ability to apply and adapt them in unfamiliar situations.
E1
Ability to use fundamental knowledge to investigate new and emerging technologies.
E3
Ability to apply mathematical and computer based models for solving problems in engineering, and the ability to assess the limitations of particular cases.
E4
Understanding of and ability to apply a systems approach to engineering problems.
P1
A thorough understanding of current practice and its limitations and some appreciation of likely new developments.
P3
Understanding of contexts in which engineering knowledge can be applied (e.g. operations and management, technology, development, etc).
US1
A comprehensive understanding of the scientific principles of own specialisation and related disciplines.
US4
An awareness of developing technologies related to own specialisation.
Last modified: 02/06/2018 00:47
Engineering Tripos Part IIB, 4F5: Advanced Communications & Coding, 2017-18
Leader
Lecturer
Dr J Sayir
Lecturer
Dr R Venkataramanan
Timing and Structure
Lent term. 16 lectures. Assessment: 100% exam
Prerequisites
The main pre-requisite is a good background in probability and information theory. 3F1, 3F4 and 3F7 useful.
Aims
The aims of the course are to:
- Introduce students to the principles of algebraic coding and Reed Solomon coding in particular
- Give students an overview of cryptology with example of techniques that share the same mathematical background as algebraic coding.
- Give students an understanding of the challenges inherent in wireless communcation, and the tools to design modulation schemes that address these challenges
Objectives
As specific objectives, by the end of the course students should be able to:
- Introduction to applied abstract algebra
- Basic definitions of linear codes and the Reed Solomon code
- Encoding and decoding of Reed Solomon codes for error and erasure channels
- Overview of Crypylogy and some algebraic cryptographic techniques
- Be familiar with standard modulation techniques, and be able to analyse their performance in the presence of noise
- Understand the concept of fading in wireless channels and how diversty techniques can be used to combat fading
Content
- The first part of the course will give an introduction to abstract algebra with an eye to practical applications. In particular, we will study arithmetic over groups and finite fields to a point where students should have the knowledge to implement a practical finite field calculator
- In the second part of the course, we will introduce the basic concepts of algebraic linear coding and give a spectral presentation of Reed Solomon codes, one of the most commonly used codes in applications as wide as data storage, cellular wireless communications, QR codes and many others.
- The spectral presentation will lead to an easily implementable encoder and decoder structure for both error corrections or erasure recovery.
- In the third part of the course, we will give an overview of the field of Cryptology, or the science of secret and authentic communication. We will then present a number of cryptographic techniques that share the same algebraic fundamentals as linear algebraic coding.
- The final part of the course will cover modulation techniques and wireless communication. We will discuss the phenomenon of fading, a key concept in wireless communication, and look at how to combat fading by using diversity in time/frequency/space.
All the topics will be presented in the context of an integrated end-to-end communication system.
Introduction to practical number theory and algebra (4L)
- Elementary number theory
- Groups and fields
- Extension fields
- 3 equivalent approaches to multiplication in extension fields
- matrix operations and the Discrete Fourier Transform
Algebraic Coding (3L)
- Linear coding and the Singleton Bound
- Blahut's theorem
- Reed Solomon (RS) codes
- Encoding and decoding of RS codes
- Erasure channel decoding
Introduction to Cryptology (3L)
- Overview of Cryptology
- Stream ciphers, examples
- Block ciphers, examples
- Public key cryptography, basic techniques
Modulation Techniques and Wireless Communication (6L)
- Modulation techniques and their performance over additive Gaussian noise channels
- Modelling a wireless channel: the concept of fading
- Combating fading with diversity in time/frequency/space
Further notes
Booklists
Useful References
Coding Theory
- Modern Coding Theory, T. Richardson & R. Urbanke, Cambridge Univ. Press. (this books covers LDPC codes)
- The Theory of Error-Correcting Codes, F. J. MacWilliams & N. J. A. Sloane, North Holland. (covers classical coding theory)
Wireless Communication
- Fundamentals of Wireless Communication, D. Tse & P.Viswanath, Cambridge Univ. Press 2005. (Available free online)
- Wireless Communications, A. Goldsmith, Cambridge Univ. Press 2005.
Please see the Booklist for Group F Courses for library holdings.
Examination Guidelines
Please refer to Form & conduct of the examinations.
UK-SPEC
This syllabus contributes to the following areas of the UK-SPEC standard:
Toggle display of UK-SPEC areas.
GT1
Develop transferable skills that will be of value in a wide range of situations. These are exemplified by the Qualifications and Curriculum Authority Higher Level Key Skills and include problem solving, communication, and working with others, as well as the effective use of general IT facilities and information retrieval skills. They also include planning self-learning and improving performance, as the foundation for lifelong learning/CPD.
IA1
Apply appropriate quantitative science and engineering tools to the analysis of problems.
IA2
Demonstrate creative and innovative ability in the synthesis of solutions and in formulating designs.
KU1
Demonstrate knowledge and understanding of essential facts, concepts, theories and principles of their engineering discipline, and its underpinning science and mathematics.
KU2
Have an appreciation of the wider multidisciplinary engineering context and its underlying principles.
D1
Wide knowledge and comprehensive understanding of design processes and methodologies and the ability to apply and adapt them in unfamiliar situations.
E1
Ability to use fundamental knowledge to investigate new and emerging technologies.
E3
Ability to apply mathematical and computer based models for solving problems in engineering, and the ability to assess the limitations of particular cases.
E4
Understanding of and ability to apply a systems approach to engineering problems.
P1
A thorough understanding of current practice and its limitations and some appreciation of likely new developments.
P3
Understanding of contexts in which engineering knowledge can be applied (e.g. operations and management, technology, development, etc).
US1
A comprehensive understanding of the scientific principles of own specialisation and related disciplines.
US4
An awareness of developing technologies related to own specialisation.
Last modified: 24/01/2018 07:38
Engineering Tripos Part IIB, 4F5: Advanced Information Theory and Coding, 2024-25
Module Leader
Lecturer
Prof A Guillen i Fabregas and Dr Jossy Sayir
Timing and Structure
Michaelmas term. 16 lectures. Assessment: 100% exam
Prerequisites
3F7 assumed, 3F1, 3F4 useful but not necessary
Aims
The aims of the course are to:
- Learn about applications of information theory to hypothesis testing as well as refinements of source and channel coding theorems through error exponents.
- Introduce students to the principles of algebraic coding and Reed Solomon coding in particular
- Give students an overview of cryptology with example of techniques that share the same mathematical background as algebraic coding.
Objectives
As specific objectives, by the end of the course students should be able to:
- have gained an appreciation for the connection between information-theoretic concepts and fundamental problems in statistics
- have a good understanding of the derivations of error exponents for data compression and transmission
- have a good understanding of the fundamental connections between hypothesis testing and information theory
- have gained a practical understanding of the algebraic fundamentals that underlie channel coding and cryptology
- understand the properties of linear block codes over finite fields
- be able to implement encoders and decoders for Reed Solomon codes
- have gained an overview of methods and aims in cryptology (including cryptography, crypt- analysis, secrecy, authenticity)
- be familiar with one example each of a block cipher and a stream cipher
- be able to implement public key cryptosystems, in particular the Diffie-Hellman and Rivest- Shamir-Adleman (RSA) systems
Content
-
This course will introduce students to applications of information theory and coding theory in statistics, information storage, and cryptography.
The first part of the course will discuss applications of information theory to universal data compression, statistics, and inference.
The second part of the course will expand linear coding principles acquired in 3F7 to non-binary codes over finite fields. After establishing the algebraic fundamentals, we will cover Reed-Solomon coding, a technique used in a wide range of communication and storage systems (hard disks, blu ray discs, QR codes, USB mass storage device class, DNA storage, and others.)
The final part of the course will introduce the discipline of cryptology, which includes cryptography, the essential art of ensuring secrecy and authenticity, and cryptanalysis, the dark art of breaking that secrecy. The course will cover a number of methods to provide secrecy, ranging from mathematically provable secrecy to public key methods through which computationally secure communication links can be established over public channels.
Information theory and statistics (7-9L, Prof Albert Guillén i Fàbregas)
- Source coding, optimum fixed-rate coding, error exponents
- Binary hypothesis testing, probability of error, error exponents, Stein's lemma
- M-ary hypothesis testing, probability of error
- Channel coding, connection with hypothesis testing, perfect codes, error exponents
Introduction to practical number theory and algebra (2-3L, Dr Jossy Sayir)
- Elementary number theory
- Groups and fields, extension fields
- 3 equivalent approaches to multiplication in extension fields
- Matrix operations and the Discrete Fourier Transform
Algebraic Coding (3L, Dr Jossy Sayir)
- Linear coding and the Singleton Bound
- Distance profiles and MacWilliams Identities
- Blahut’s theorem
- Reed Solomon (RS) codes
- Encoding and decoding of RS codes
Introduction to Cryptology (2L, Dr Jossy Sayir )
- Overview of cryptology
- Stream ciphers, examples
- Block ciphers, examples
- Public key cryptography, basic techniques
Further notes
Examples papers
Examples papers consist of a recommended list of problems to solve in the lecture notes.
Coursework
none
Booklists
- Information Theory:
- Elements of Information Theory, T. M. Cover & J. A. Thomas, Wiley-Interscience, 2nd Ed, 2006.
- Information Theory: Coding Theorems for Discrete Memoryless Systems, I. Csiszàr & J. Körner, Cambridge University Press, 2nd Ed. 2011.
- Coding theory:
- The Theory of Error-Correcting Codes, F. J. MacWilliams & N. J. A. Sloane, North Holland.
- Algebraic Codes for Data Transmission, Richard E. Blahut, Cambridge University Press, 2003 (Online 2012)
Please refer to the Booklist for Part IIB Courses for references to this module, this can be found on the associated Moodle course.
Examination Guidelines
Please refer to Form & conduct of the examinations.
UK-SPEC
This syllabus contributes to the following areas of the UK-SPEC standard:
Toggle display of UK-SPEC areas.
GT1
Develop transferable skills that will be of value in a wide range of situations. These are exemplified by the Qualifications and Curriculum Authority Higher Level Key Skills and include problem solving, communication, and working with others, as well as the effective use of general IT facilities and information retrieval skills. They also include planning self-learning and improving performance, as the foundation for lifelong learning/CPD.
IA1
Apply appropriate quantitative science and engineering tools to the analysis of problems.
IA2
Demonstrate creative and innovative ability in the synthesis of solutions and in formulating designs.
KU1
Demonstrate knowledge and understanding of essential facts, concepts, theories and principles of their engineering discipline, and its underpinning science and mathematics.
KU2
Have an appreciation of the wider multidisciplinary engineering context and its underlying principles.
D1
Wide knowledge and comprehensive understanding of design processes and methodologies and the ability to apply and adapt them in unfamiliar situations.
E1
Ability to use fundamental knowledge to investigate new and emerging technologies.
E3
Ability to apply mathematical and computer based models for solving problems in engineering, and the ability to assess the limitations of particular cases.
E4
Understanding of and ability to apply a systems approach to engineering problems.
P1
A thorough understanding of current practice and its limitations and some appreciation of likely new developments.
P3
Understanding of contexts in which engineering knowledge can be applied (e.g. operations and management, technology, development, etc).
US1
A comprehensive understanding of the scientific principles of own specialisation and related disciplines.
US4
An awareness of developing technologies related to own specialisation.
Last modified: 23/08/2024 18:37
Engineering Tripos Part IIB, 4F5: Advanced Information Theory and Coding, 2020-21
Module Leader
Lecturer
Lecturer
Timing and Structure
Lent term. 16 lectures. Assessment: 100% exam
Prerequisites
3F7 assumed, 3F1, 3F4 useful but not necessary
Aims
The aims of the course are to:
- Learn about applications of information theory to hypothesis testing as well as refinements of source and channel coding theorems through error exponents.
- Introduce students to the principles of algebraic coding and Reed Solomon coding in particular
- Give students an overview of cryptology with example of techniques that share the same mathematical background as algebraic coding.
Objectives
As specific objectives, by the end of the course students should be able to:
- have gained an appreciation for the connection between information-theoretic concepts and fundamental problems in statistics
- have a good understanding of the derivations of error exponents for data compression and transmission
- have a good understanding of the fundamental connections between hypothesis testing and information theory
- have gained a practical understanding of the algebraic fundamentals that underlie channel coding and cryptology
- understand the properties of linear block codes over finite fields
- be able to implement encoders and decoders for Reed Solomon codes
- have gained an overview of methods and aims in cryptology (including cryptography, crypt- analysis, secrecy, authenticity)
- be familiar with one example each of a block cipher and a stream cipher
- be able to implement public key cryptosystems, in particular the Diffie-Hellman and Rivest- Shamir-Adleman (RSA) systems
Content
-
This course will introduce students to applications of information theory and coding theory in statistics, information storage, and cryptography.
The first part of the course will discuss applications of information theory to universal data compression, statistics, and inference.
The second part of the course will expand linear coding principles acquired in 3F7 to non-binary codes over finite fields. After establishing the algebraic fundamentals, we will cover Reed-Solomon coding, a technique used in a wide range of communication and storage systems (hard disks, blu ray discs, QR codes, USB mass storage device class, DNA storage, and others.)
The final part of the course will introduce the discipline of cryptology, which includes cryptography, the essential art of ensuring secrecy and authenticity, and cryptanalysis, the dark art of breaking that secrecy. The course will cover a number of methods to provide secrecy, ranging from mathematically provable secrecy to public key methods through which computationally secure communication links can be established over public channels.
Information theory and statistics (7-9L, Dr Albert Guillén i Fàbregas)
- Binary hypothesis testing, probability of error, error exponents, Stein's lemma
- M-ary hypothesis testing, probability of error
- Source coding, optimum fixed-rate coding, error exponents
- Method of types and duality
- Channel coding, connection with hypothesis testing, perfect codes, error exponents
Introduction to practical number theory and algebra (2-3L, Dr Jossy Sayir)
- Elementary number theory
- Groups and fields, extension fields
- 3 equivalent approaches to multiplication in extension fields
- Matrix operations and the Discrete Fourier Transform
Algebraic Coding (3L, Dr Jossy Sayir)
- Linear coding and the Singleton Bound
- Distance profiles and MacWilliams Identities
- Blahut’s theorem
- Reed Solomon (RS) codes
- Encoding and decoding of RS codes
Introduction to Cryptology (2L, Dr Jossy Sayir )
- Overview of cryptology
- Stream ciphers, examples
- Block ciphers, examples
- Public key cryptography, basic techniques
Further notes
Examples papers
Examples papers consist of a recommended list of problems to solve in the lecture notes.
Coursework
none
Booklists
- Information Theory:
- Elements of Information Theory, T. M. Cover & J. A. Thomas, Wiley-Interscience, 2nd Ed, 2006.
- Information Theory: Coding Theorems for Discrete Memoryless Systems, I. Csiszàr & J. Körner, Cambridge University Press, 2nd Ed. 2011.
- Coding theory:
- The Theory of Error-Correcting Codes, F. J. MacWilliams & N. J. A. Sloane, North Holland.
- Algebraic Codes for Data Transmission, Richard E. Blahut, Cambridge University Press, 2003 (Online 2012)
Please refer to the Booklist for Part IIB Courses for references to this module, this can be found on the associated Moodle course.
Examination Guidelines
Please refer to Form & conduct of the examinations.
UK-SPEC
This syllabus contributes to the following areas of the UK-SPEC standard:
Toggle display of UK-SPEC areas.
GT1
Develop transferable skills that will be of value in a wide range of situations. These are exemplified by the Qualifications and Curriculum Authority Higher Level Key Skills and include problem solving, communication, and working with others, as well as the effective use of general IT facilities and information retrieval skills. They also include planning self-learning and improving performance, as the foundation for lifelong learning/CPD.
IA1
Apply appropriate quantitative science and engineering tools to the analysis of problems.
IA2
Demonstrate creative and innovative ability in the synthesis of solutions and in formulating designs.
KU1
Demonstrate knowledge and understanding of essential facts, concepts, theories and principles of their engineering discipline, and its underpinning science and mathematics.
KU2
Have an appreciation of the wider multidisciplinary engineering context and its underlying principles.
D1
Wide knowledge and comprehensive understanding of design processes and methodologies and the ability to apply and adapt them in unfamiliar situations.
E1
Ability to use fundamental knowledge to investigate new and emerging technologies.
E3
Ability to apply mathematical and computer based models for solving problems in engineering, and the ability to assess the limitations of particular cases.
E4
Understanding of and ability to apply a systems approach to engineering problems.
P1
A thorough understanding of current practice and its limitations and some appreciation of likely new developments.
P3
Understanding of contexts in which engineering knowledge can be applied (e.g. operations and management, technology, development, etc).
US1
A comprehensive understanding of the scientific principles of own specialisation and related disciplines.
US4
An awareness of developing technologies related to own specialisation.
Last modified: 01/09/2020 10:37
Pages
