Undergraduate Teaching 2025-26

Not logged in. More information may be available... Login via Raven / direct.

Engineering Tripos Part IIA Project, GB3: RISC-V Processor, 2021-22

Leader

Assistant Professor Qixiang Cheng

Timing and Structure

Fridays 9-11am plus afternoons, and Tuesdays 11-1pm

Prerequisites

3B2 (essential). Prior familiarity with Unix command line tools (e.g., basic shell scripts, creating Makefiles, and so on) and with git and GitHub is desirable.

Aims

The aims of the course are to:

  • To become familiar with Verilog HDL and with the Lattice iCE40, a state-of-the-art low-power miniature FPGA used in many commercial embedded sensing and wearable computing applications.
  • To obtain experience working with FPGA synthesis tools for embedded sensing and computing applications.
  • To gain experience with the RISC-V (pronounced "RISC-five") architecture, an exciting, new, and forward-looking reduced instruction set computer (RISC) architecture.
  • To carry out the implementation and evaluation of a minimal subset of the RISC-V architecture on the iCE40 FPGA.

Content

Students will work in groups of three for this project.

Students modify an unoptimized RV32I RISC-V processor running on an iCE40 FPGA in a tiny wafer-scale 2.15x2.50 mm WLCSP package, using a completely open-source toolchain. The hardware used in the project has isolated power rails for the FPGA core, I/O, and PLL. Because the hardware has built-in shunt resistors, students can measure power usage with a laboratory multimeter. Students work in groups of three to evaluate the performance, power dissipation, and resource usage of their modifications and the project culminates in a competition between teams to achieve designs on the Pareto frontier. (One team last year got active power dissipation, while running a provided benchmark, down to 214 microwatts.)

The RISC-V architecture is a new open reduced instruction set computer (RISC) architecture that has many advantages over legacy architectures such as ARM. Because it was designed from the ground up for efficiency, RISC-V enables more efficient hardware implementations than many existing commercial architectures. One variant of the RISC-V is small enough to fit within the Lattice iCE40, a low-power miniature FPGA (in a 2.15x2.55mm package) targeted at embedded sensing systems. This project will provide students with the opportunity to gain experience with the RISC-V architecture, an exciting, new, and forward-looking reduced instruction set computer (RISC) architecture and to implement and evaluate a minimal subset of the RISC-V architecture on the iCE40 FPGA.

Week 1

Complete the warm-up exercise mapping the provided pre-implemented RISC-V subset processor core on the iCE40 and become familiar with the open-source FPGA synthesis tools, with the RISC-V processor emulator, and with Verilog. Watch the overview fascicle videos on the toolchain and on a Bayesian view of measurements, measurement uncertainty, sensors, and computing on sensor data.

Week 2

Evaluate design options to improve performance, FPGA resource usage, and power dissipation of the baseline RISC-V processor core.

Week 3

Keeping in mind the goal of Pareto-optimal designs, the three team members implement improvements to the performance, power dissipation, and resource usage of the baseline RISC-V processor.

Week 4

Evaluate the improved design in terms of performance, power dissipation, and FPGA resource usage for the measurement data uncertainty propagation and particle filter benchmark applications.

Coursework

Coursework Due date Marks

Interim report 1 (individual credit)

Report on your characterization of the performance, power dissipation, and FPGA resource usage of the provided baseline RISC-V processor implementation.

4pm Friday 20th May 2022

 

20

Interim report 2 (individual credit)

Describe progress improving the implementation of the provided RISC-V processor core in terms of performance, FPGA resource usage, and power dissipation when running on the iCE40 FPGA.

4pm Friday 27th May 2022

 

30

 

Final report (10 points for group credit for demo, individual credit for report)

Demonstration and report on modified RISC-V processor core implementation.

4pm Thursday 9th June 2022

 

30

 

 

Examination Guidelines

Please refer to Form & conduct of the examinations.

 
Last modified: 22/04/2022 10:27

Engineering Tripos Part IIA Project, GB3: RISC-V Processor, 2020-21

Leader

Dr P Stanley-Marbell

Timing and Structure

Fridays 9-11am plus afternoons, and Tuesdays 11-1pm

Prerequisites

3B2 (essential). Prior familiarity with Unix command line tools (e.g., basic shell scripts, creating Makefiles, and so on) and with git and GitHub is desirable.

Aims

The aims of the course are to:

  • To become familiar with Verilog HDL and with the Lattice iCE40, a state-of-the-art low-power miniature FPGA used in many commercial embedded sensing and wearable computing applications.
  • Obtain hands-on experience with implementing ideas relating to a Bayesian view of measurements, measurement uncertainty, sensors, and computing on sensor data on state-of-the-art FPGAs
  • To obtain experience working with FPGA synthesis tools for Internet of Things (i.e., embedded sensing and computing) applications.
  • To gain experience with the RISC-V (pronounced "RISC-five") architecture, an exciting, new, and forward-looking reduced instruction set computer (RISC) architecture.
  • To carry out the implementation and evaluation of a minimal subset of the RISC-V architecture on the iCE40 FPGA.

Content

Students will work in groups of three for this project.

Students modify an unoptimized RV32I RISC-V processor running on an iCE40 FPGA in a tiny wafer-scale 2.15x2.50 mm WLCSP package, using a completely open-source toolchain. The hardware used in the project has isolated power rails for the FPGA core, I/O, and PLL. Because the hardware has built-in shunt resistors, students can measure power usage with a laboratory multimeter. Students work in groups of three to evaluate the performance, power dissipation, and resource usage of their modifications and the project culminates in a competition between teams to achieve designs on the Pareto frontier. (One team last year got active power dissipation, while running a provided benchmark, down to 214 microwatts.)

The RISC-V architecture is a new open reduced instruction set computer (RISC) architecture that has many advantages over legacy architectures such as ARM. Because it was designed from the ground up for efficiency, RISC-V enables more efficient hardware implementations than many existing commercial architectures. One variant of the RISC-V is small enough to fit within the Lattice iCE40, a low-power miniature FPGA (in a 2.15x2.55mm package) targeted at embedded sensing / Internet of Things systems. This project will provide students with the opportunity to gain experience with the RISC-V architecture, an exciting, new, and forward-looking reduced instruction set computer (RISC) architecture and to implement and evaluate a minimal subset of the RISC-V architecture on the iCE40 FPGA. Through the project's choice of application examples, this IIA project will also provide hand-on experience with implementing ideas relating to a Bayesian view of measurements, measurement uncertainty, sensors, and computing on sensor data that is synergistic with other research and teaching in the Department of Engineering at the University of Cambridge.

Week 1

Complete the warm-up exercise mapping the provided pre-implemented RISC-V subset processor core on the iCE40 and become familiar with the open-source FPGA synthesis tools, with the RISC-V processor emulator, and with Verilog. Watch the overview fascicle videos on the toolchain and on a Bayesian view of measurements, measurement uncertainty, sensors, and computing on sensor data.

Week 2

Evaluate design options to improve performance, FPGA resource usage, and power dissipation of the baseline RISC-V processor core.

Week 3

Keeping in mind the goal of Pareto-optimal designs, the three team members implement improvements to the performance, power dissipation, and resource usage of the baseline RISC-V processor.

Week 4

Evaluate the improved design in terms of performance, power dissipation, and FPGA resource usage for the measurement data uncertainty propagation and particle filter benchmark applications.

Coursework

Coursework Due date Marks

Interim report 1 (individual credit)

Report on your characterization of the performance, power dissipation, and FPGA resource usage of the provided baseline RISC-V processor implementation.

4pm Friday 21st May 2021

(week two)

20

Interim report 2 (individual credit)

Describe progress improving the implementation of the provided RISC-V processor core in terms of performance, FPGA resource usage, and power dissipation when running on the iCE40 FPGA.

4pm Friday 28th May 2021

(week three)

30

 

Final report (10 points for group credit for demo, individual credit for report)

Demonstration and report on modified RISC-V processor core implementation.

4pm Thursday 10th June 2021

(week five)

30

 

 

Examination Guidelines

Please refer to Form & conduct of the examinations.

 
Last modified: 30/04/2021 12:13

Engineering Tripos Part IIA Project, GB3: RISC-V Processor, 2019-20

Leader

Dr P Stanley-Marbell

Timing and Structure

Fridays 9-11am plus afternoons, and Tuesdays 11-1pm

Prerequisites

3B2 (essential). Prior familiarity with Unix command line tools (e.g., basic shell scripts, creating Makefiles, and so on) and with git and GitHub is desirable.

Aims

The aims of the course are to:

  • To become familiar with Verilog HDL and with the Lattice iCE40, a state-of-the-art low-power miniature FPGA used in many commercial embedded sensing and wearable computing applications.
  • Obtain hands-on experience with implementing ideas relating to a Bayesian view of measurements, measurement uncertainty, sensors, and computing on sensor data on state-of-the-art FPGAs
  • To obtain experience working with FPGA synthesis tools for Internet of Things (i.e., embedded sensing and computing) applications.
  • To gain experience with the RISC-V (pronounced "RISC-five") architecture, an exciting, new, and forward-looking reduced instruction set computer (RISC) architecture.
  • To carry out the implementation and evaluation of a minimal subset of the RISC-V architecture on the iCE40 FPGA.

Content

Students will work in groups of three for this project.

Students modify an unoptimized RV32I RISC-V processor running on an iCE40 FPGA in a tiny wafer-scale 2.15x2.50 mm WLCSP package, using a completely open-source toolchain. The hardware used in the project has isolated power rails for the FPGA core, I/O, and PLL. Because the hardware has built-in shunt resistors, students can measure power usage with a laboratory multimeter. Students work in groups of three to evaluate the performance, power dissipation, and resource usage of their modifications and the project culminates in a competition between teams to achieve designs on the Pareto frontier. (One team last year got active power dissipation, while running a provided benchmark, down to 214 microwatts.)

The RISC-V architecture is a new open reduced instruction set computer (RISC) architecture that has many advantages over legacy architectures such as ARM. Because it was designed from the ground up for efficiency, RISC-V enables more efficient hardware implementations than many existing commercial architectures. One variant of the RISC-V is small enough to fit within the Lattice iCE40, a low-power miniature FPGA (in a 2.15x2.55mm package) targeted at embedded sensing / Internet of Things systems. This project will provide students with the opportunity to gain experience with the RISC-V architecture, an exciting, new, and forward-looking reduced instruction set computer (RISC) architecture and to implement and evaluate a minimal subset of the RISC-V architecture on the iCE40 FPGA. Through the project's choice of application examples, this IIA project will also provide hand-on experience with implementing ideas relating to a Bayesian view of measurements, measurement uncertainty, sensors, and computing on sensor data that is synergistic with other research and teaching in the Department of Engineering at the University of Cambridge.

Week 1

Complete the warm-up exercise mapping the provided pre-implemented RISC-V subset processor core on the iCE40 and become familiar with the open-source FPGA synthesis tools, with the RISC-V processor emulator, and with Verilog. Watch the overview fascicle videos on the toolchain and on a Bayesian view of measurements, measurement uncertainty, sensors, and computing on sensor data.

Week 2

Evaluate design options to improve performance, FPGA resource usage, and power dissipation of the baseline RISC-V processor core.

Week 3

Keeping in mind the goal of Pareto-optimal designs, the three team members implement improvements to the performance, power dissipation, and resource usage of the baseline RISC-V processor.

Week 4

Evaluate the improved design in terms of performance, power dissipation, and FPGA resource usage for the measurement data uncertainty propagation and particle filter benchmark applications.

Coursework

Coursework Due date Marks

Interim report 1 (individual credit)

Report on your characterization of the performance, power dissipation, and FPGA resource usage of the provided baseline RISC-V processor implementation.

4pm Friday 15th May 2020

(end of week one)

20

Interim report 2 (individual credit)

Describe progress improving the implementation of the provided RISC-V processor core in terms of performance, FPGA resource usage, and power dissipation when running on the iCE40 FPGA.

4pm Friday 29th May 2020

(end of week three)

30

 

Final report (10 points for group credit for demo, individual credit for report)

Demonstration and report on modified RISC-V processor core implementation.

4pm Friday 5th June 2020

(end of week four)

30

 

 

Examination Guidelines

Please refer to Form & conduct of the examinations.

 
Last modified: 12/01/2020 20:04

Engineering Tripos Part IIA Project, SF3: Machine Learning, 2025-26

Leader

Dr C Micou

Timing and Structure

Students work to their own schedule. A staffed "surgery" runs during scheduled lab sessions on Tuesdays and Fridays to give help, advice and feedback.

Prerequisites

Part I computing; Either of 3F3 or 3F8

Aims

The aims of the course are to:

  • expose students to machine learning approaches to non-linear regression
  • to gain practical experience necessary to use these techniques successfully (e.g the use of training and test sets for evaluation, optimisation etc.)
  • to understand the robustness of these approaches to challenging real world phenomena including noise and non-linearities

Content

 

In this project, students will consider the inverted pendulum system receiving a software simulator of a cart with a pendulum attached written in Python.
 
The goal will be to learn a controller that balances the pendulum in a data-driven way. The students will initially learn how to operate the simulator and explore the different types of behaviour that the system can exhibit. Next, they will collect training data from the simulator and use this to train non-linear regression models, including linear regression with non-linear basis functions. The trained models will be assessed on test data from the simulator. Once accurate models are learned these will be used to learn controllers that can balance the pendulum in the upright position and keep it there. Finally, the controllers and the models will will be stress tested in various ways to test their robustness. 
 
Students work individually for this project. 
 

Week 1

Explore the cart-pendulum system using the simulator. Understand the state space and the governing differential equations and fit data with simple linear models.
 

Week 2

Further explore training and test data from the simulator for building models of the system and validating them. Introduce non-linear models and assess their quality compared to simple linear models.
 

Week 3

Define a function that maps from the system's state to control actions (the "policy"), optimise the policy to keep pendulum upright. 
 

Week 4

Extend the project by investigating robustness to noise and disturbances, or investigate more advanced policies for control.
 

Coursework

Coursework Due date Marks

Interim report

29 May 2026

20

Final report

12 June 2026

60

All reports are submitted via Moodle. The deadline for submission is 4pm on the due date.

Examination Guidelines

Please refer to Form & conduct of the examinations.

 
Last modified: 28/04/2026 16:29

Engineering Tripos Part IIA Project, SF3: Machine Learning, 2024-25

Leader

Prof J M H Lobato

Leader

Prof C E Rasmussen

Timing and Structure

Students work to their own schedule. A staffed "surgery" runs every weekday 10-11am to give help, advice and feedback.

Prerequisites

Part I computing; Either of 3F3 or 3F8

Aims

The aims of the course are to:

  • expose students to machine learning approaches to non-linear regression
  • to gain practical experience necessary to use these techniques successfully (e.g the use of training and test sets for evaluation, optimisation etc.)
  • to understand the robustness of these approaches to challenging real world phenomena including noise and non-linearities

Content

 

In this project, students will consider the inverted pendulum system receiving a software simulator of a cart with a pendulum attached written in Python.
 
The goal will be to learn a controller that balances the pendulum in a data-driven way. The students will initially learn how to operate the simulator and explore the different types of behaviour that the system can exhibit. Next, they will collect training data from the simulator and use this to train non-linear regression models, including linear regression with non-linear basis functions. The trained models will be assessed on test data from the simulator. Once accurate models are learned these will be used to learn controllers that can balance the pendulum in the upright position and keep it there. Finally, the controllers and the models will will be stress tested in various ways to test their robustness. 
 
Students work individually for this project. 
 

Week 1

Explore the cart-pendulum system using the simulator. Understand the state space and the governing differential equations.
 

Week 2

Gather training and test data from the simulator for building models of the system and validating them. Fit various models and assess their quality.
 

Week 3

Define a function that maps from the system's state to control actions (the "policy"), optimise the policy to keep pendulum upright. 
 

Week 4

Stress-test control and learning systems in various ways. 
 

Coursework

Coursework Due date Marks

Interim report

TBC

20

Final report

TBC

60

 

Examination Guidelines

Please refer to Form & conduct of the examinations.

 
Last modified: 29/11/2024 15:20

Engineering Tripos Part IIA Project, SF3: Machine Learning, 2022-23

Leader

Prof G Csanyi

Timing and Structure

Students work to their own schedule. A staffed "surgery" runs every weekday 10-11am to give help, advice and feedback.

Prerequisites

Part I computing; Either of 3F3 or 3F8

Aims

The aims of the course are to:

  • expose students to machine learning approaches to non-linear regression
  • to gain practical experience necessary to use these techniques successfully (e.g the use of training and test sets for evaluation, optimisation etc.)
  • to understand the robustness of these approaches to challenging real world phenomena including noise and non-linearities

Content

 

In this project, students will consider the inverted pendulum system receiving a software simulator of a cart with a pendulum attached written in Python.
 
The goal will be to learn a controller that balances the pendulum in a data-driven way. The students will initially learn how to operate the simulator and explore the different types of behaviour that the system can exhibit. Next, they will collect training data from the simulator and use this to train non-linear regression models, including linear regression with non-linear basis functions. The trained models will be assessed on test data from the simulator. Once accurate models are learned these will be used to learn controllers that can balance the pendulum in the upright position and keep it there. Finally, the controllers and the models will will be stress tested in various ways to test their robustness. 
 
Students work individually for this project. 
 

Week 1

Explore the cart-pendulum system using the simulator. Understand the state space and the governing differential equations.
 

Week 2

Gather training and test data from the simulator for building models of the system and validating them. Fit various models and assess their quality.
 

Week 3

Define a function that maps from the system's state to control actions (the "policy"), optimise the policy to keep pendulum upright. 
 

Week 4

Stress-test control and learning systems in various ways. 
 

Coursework

Coursework Due date Marks

Interim report

TBC

20

Final report

TBC

60

 

Examination Guidelines

Please refer to Form & conduct of the examinations.

 
Last modified: 28/11/2022 10:35

Engineering Tripos Part IIA Project, SF3: Machine Learning, 2021-22

Leader

Prof G Csanyi

Timing and Structure

Fridays 11-1pm and Tuesdays 9-11am plus afternoons

Prerequisites

Part I computing; Either of 3F3 or 3F8

Aims

The aims of the course are to:

  • expose students to machine learning approaches to non-linear regression and model-based reinforcement learning
  • to gain practical experience necessary to use these techniques successfully (e.g the use of training and test sets for evaluation, automatic differentiation for optimisation etc.)
  • to understand the robustness of these approaches to challenging real world phenomena including noise and non-linearities

Content

Note: This is a new project this year; some of the details below may evolve as the project content is developed further during the Michaelmas and Lent Terms.

 

In this project, students will consider the inverted pendulum system receiving a software simulator of a cart with a pendulum attached written in Python.
 
The goal will be to learn a controller that balances the pendulum in a data-driven way. The students will initially learn how to operate the simulator and explore the different types of behaviour that the system can exhibit. Next, they will collect training data from the simulator and use this to train non-linear regression models, including linear regression with non-linear basis functions. The trained models will be assessed on test data from the simulator. Once accurate models are learned these will be used to learn controllers that can balance the pendulum in the upright position and keep it there. Data-efficient model-based reinforcement learning techniques will be used for this stage. Finally, the controllers and the models will will be stress tested in various ways to test their robustness. 
 
Students work individually for this project. 
 

Week 1

Explore the cart-pendulum system using the simulator. Understand the state space and the governing differential equations.
 

Week 2

Gather training and test data from the simulator for building models of the system and validating them. Fit various models and assess their quality.
 

Week 3

Define a function that maps from the system's state to control actions (the "policy"), optimise the policy to keep pendulum upright. 
 

Week 4

Stress-test control and learning systems in various ways. 
 

Coursework

Coursework Due date Marks

Interim report

4pm Sunday 17 May 2020

30

Final report

4pm Thursday 4 June 2020

50

 

Examination Guidelines

Please refer to Form & conduct of the examinations.

 
Last modified: 02/12/2021 12:52

Engineering Tripos Part IIA Project, SF3: Machine Learning, 2020-21

Leader

Prof G Csanyi

Timing and Structure

Fridays 11-1pm and Tuesdays 9-11am plus afternoons

Prerequisites

Part I computing; Either of 3F3 or 3F8

Aims

The aims of the course are to:

  • expose students to machine learning approaches to non-linear regression and model-based reinforcement learning
  • to gain practical experience necessary to use these techniques successfully (e.g the use of training and test sets for evaluation, automatic differentiation for optimisation etc.)
  • to understand the robustness of these approaches to challenging real world phenomena including noise and non-linearities

Content

Note: This is a new project this year; some of the details below may evolve as the project content is developed further during the Michaelmas and Lent Terms.

 

In this project, students will consider the inverted pendulum system receiving a software simulator of a cart with a pendulum attached written in Python.
 
The goal will be to learn a controller that balances the pendulum in a data-driven way. The students will initially learn how to operate the simulator and explore the different types of behaviour that the system can exhibit. Next, they will collect training data from the simulator and use this to train non-linear regression models, including linear regression with non-linear basis functions. The trained models will be assessed on test data from the simulator. Once accurate models are learned these will be used to learn controllers that can balance the pendulum in the upright position and keep it there. Data-efficient model-based reinforcement learning techniques will be used for this stage. Finally, the controllers and the models will will be stress tested in various ways to test their robustness. 
 
Students work individually for this project. 
 

Week 1

Explore the cart-pendulum system using the simulator. Understand the state space and the governing differential equations.
 

Week 2

Gather training and test data from the simulator for building models of the system and validating them. Fit various models and assess their quality.
 

Week 3

Define a function that maps from the system's state to control actions (the "policy"), optimise the policy to keep pendulum upright. 
 

Week 4

Stress-test control and learning systems in various ways. 
 

Coursework

Coursework Due date Marks

Interim report

4pm Sunday 17 May 2020

30

Final report

4pm Thursday 4 June 2020

50

 

Examination Guidelines

Please refer to Form & conduct of the examinations.

 
Last modified: 30/11/2020 09:05

Engineering Tripos Part IIA Project, SF3: Machine Learning, 2019-20

Leader

Prof G Csanyi

Academic

Dr Richard Turner

Academic

Dr José Miguel Hernández-Lobato

Timing and Structure

Fridays 11-1pm and Tuesdays 9-11am plus afternoons

Prerequisites

Part I computing; Either of 3F3 or 3F8

Aims

The aims of the course are to:

  • expose students to machine learning approaches to non-linear regression and model-based reinforcement learning
  • to gain practical experience necessary to use these techniques successfully (e.g the use of training and test sets for evaluation, automatic differentiation for optimisation etc.)
  • to understand the robustness of these approaches to challenging real world phenomena including noise and non-linearities

Content

Note: This is a new project this year; some of the details below may evolve as the project content is developed further during the Michaelmas and Lent Terms.

 

In this project, students will consider the inverted pendulum system receiving a software simulator of a cart with a pendulum attached written in Python.
 
The goal will be to learn a controller that balances the pendulum in a data-driven way. The students will initially learn how to operate the simulator and explore the different types of behaviour that the system can exhibit. Next, they will collect training data from the simulator and use this to train non-linear regression models, including linear regression with non-linear basis functions. The trained models will be assessed on test data from the simulator. Once accurate models are learned these will be used to learn controllers that can balance the pendulum in the upright position and keep it there. Data-efficient model-based reinforcement learning techniques will be used for this stage. Finally, the controllers and the models will will be stress tested in various ways to test their robustness. 
 
Students work individually for this project. 
 

Week 1

Explore the cart-pendulum system using the simulator. Understand the state space and the governing differential equations.
 

Week 2

Gather training and test data from the simulator for building models of the system and validating them. Fit various models and assess their quality.
 

Week 3

Define a function that maps from the system's state to control actions (the "policy"), optimise the policy to keep pendulum upright. 
 

Week 4

Stress-test control and learning systems in various ways. 
 

Coursework

Coursework Due date Marks

Interim report

4pm Sunday 17 May 2020

30

Final report

4pm Thursday 4 June 2020

50

 

Examination Guidelines

Please refer to Form & conduct of the examinations.

 
Last modified: 04/10/2019 14:01

Engineering Tripos Part IIA Project, SF3: Machine Learning, 2018-19

Leader

Prof G Csanyi

Academic

Dr Richard Turner

Academic

Dr José Miguel Hernández-Lobato

Timing and Structure

Fridays 11-1pm and Tuesdays 9-11am plus afternoons

Prerequisites

Part I computing; Either of 3F3 or 3F8

Aims

The aims of the course are to:

  • expose students to machine learning approaches to non-linear regression and model-based reinforcement learning
  • to gain practical experience necessary to use these techniques successfully (e.g the use of training and test sets for evaluation, automatic differentiation for optimisation etc.)
  • to understand the robustness of these approaches to challenging real world phenomena including noise and non-linearities

Content

Note: This is a new project this year; some of the details below may evolve as the project content is developed further during the Michaelmas and Lent Terms.

 

In this project, students will consider the inverted pendulum system receiving a software simulator of a cart with a pendulum attached written in Python.
 
The goal will be to learn a controller that balances the pendulum in a data-driven way. The students will initially learn how to operate the simulator and explore the different types of behaviour that the system can exhibit. Next, they will collect training data from the simulator and use this to train non-linear regression models, including linear regression with non-linear basis functions. The trained models will be assessed on test data from the simulator. Once accurate models are learned these will be used to learn controllers that can balance the pendulum in the upright position and keep it there. Data-efficient model-based reinforcement learning techniques will be used for this stage. Finally, the controllers and the models will will be stress tested in various ways to test their robustness. 
 
Students work individually for this project. 
 

Week 1

Explore the cart-pendulum system using the simulator. Understand the state space and the governing differential equations.
 

Week 2

Gather training and test data from the simulator for building models of the system and validating them. Fit various models and assess their quality.
 

Week 3

Define a function that maps from the system's state to control actions (the "policy"), optimise the policy to keep pendulum upright. 
 

Week 4

Stress-test control and learning systems in various ways. 
 

Coursework

Coursework Due date Marks

Interim report

4pm Sunday 19 May 2019

30

Final report

4pm Thursday 6 June 2019

50

 

Examination Guidelines

Please refer to Form & conduct of the examinations.

 
Last modified: 28/09/2018 20:36

Pages

Subscribe to CUED undergraduate teaching site RSS