Undergraduate Teaching

Engineering Tripos Part IIA Project, SF3: Machine Learning, 2019-20

Engineering Tripos Part IIA Project, SF3: Machine Learning, 2019-20

Not logged in. More information may be available... Login via Raven / direct.

PDF versionPDF version

Leader

Prof G Csanyi

Academic

Dr Richard Turner

Academic

Dr José Miguel Hernández-Lobato

Timing and Structure

Fridays 11-1pm and Tuesdays 9-11am plus afternoons

Prerequisites

Part I computing; Either of 3F3 or 3F8

Aims

The aims of the course are to:

  • expose students to machine learning approaches to non-linear regression and model-based reinforcement learning
  • to gain practical experience necessary to use these techniques successfully (e.g the use of training and test sets for evaluation, automatic differentiation for optimisation etc.)
  • to understand the robustness of these approaches to challenging real world phenomena including noise and non-linearities

Content

Note: This is a new project this year; some of the details below may evolve as the project content is developed further during the Michaelmas and Lent Terms.

 

In this project, students will consider the inverted pendulum system receiving a software simulator of a cart with a pendulum attached written in Python.
 
The goal will be to learn a controller that balances the pendulum in a data-driven way. The students will initially learn how to operate the simulator and explore the different types of behaviour that the system can exhibit. Next, they will collect training data from the simulator and use this to train non-linear regression models, including linear regression with non-linear basis functions. The trained models will be assessed on test data from the simulator. Once accurate models are learned these will be used to learn controllers that can balance the pendulum in the upright position and keep it there. Data-efficient model-based reinforcement learning techniques will be used for this stage. Finally, the controllers and the models will will be stress tested in various ways to test their robustness. 
 
Students work individually for this project. 
 

Week 1

Explore the cart-pendulum system using the simulator. Understand the state space and the governing differential equations.
 

Week 2

Gather training and test data from the simulator for building models of the system and validating them. Fit various models and assess their quality.
 

Week 3

Define a function that maps from the system's state to control actions (the "policy"), optimise the policy to keep pendulum upright. 
 

Week 4

Stress-test control and learning systems in various ways. 
 

Coursework

Coursework Due date Marks

Interim report

4pm Sunday 17 May 2020

30

Final report

4pm Thursday 4 June 2020

50

 

Examination Guidelines

Please refer to Form & conduct of the examinations.

Last modified: 04/10/2019 14:01