Tutorials

EE5521: Optimal control

• Linear Quadratic Regulator control of a self balancing robot in ROS/Gazebo
  Solve the discrete-time ARE and apply LQR to a linearised self balancing robot in ROS/Gazebo.

EE5531: Reinforcement learning based control (interactive demos)

• Q-learning on a five-tile corridor
  Step through choose, move, reward and update, and watch the Q-table fill in.
• DQN v0.1: information flow
  Follow one transition through the Q-network, the policy, the loss and the weight update.
• DQN v0.2: experience replay
  See transitions stored in a replay buffer and learned from in random mini-batches.
• DQN v1.0: experience replay and fixed targets
  Add a frozen target network, copied from the evaluation network every k steps.
• DDPG: how the pieces fit together
  Actor, critic, target copies, OU exploration noise and soft updates, one time step at a time.
• DDPG training demos
  • Pendulum swing-up
  • Inverted pendulum (MuJoCo)
  • Reacher (MuJoCo)

---