In this project, students will consider the inverted pendulum system receiving a software simulator of a cart with a pendulum attached written in Python.

 

The goal will be to learn a controller that balances the pendulum in a data-driven way. The students will initially learn how to operate the simulator and explore the different types of behaviour that the system can exhibit. Next, they will collect training data from the simulator and use this to train non-linear regression models, including linear regression with non-linear basis functions. The trained models will be assessed on test data from the simulator. Once accurate models are learned these will be used to learn controllers that can balance the pendulum in the upright position and keep it there. Finally, the controllers and the models will will be stress tested in various ways to test their robustness. 

 

Students work individually for this project. 

 

Explore the cart-pendulum system using the simulator. Understand the state space and the governing differential equations.

 

Gather training and test data from the simulator for building models of the system and validating them. Fit various models and assess their quality.

 

Define a function that maps from the system's state to control actions (the "policy"), optimise the policy to keep pendulum upright. 

 

Stress-test control and learning systems in various ways.