K-Armed Bandit Problem

• Sequential Decision Making with Evaluative Feedback
• Learning Action Values
• Estimating Action Values Incrementally
• Optimistic initial values
• UCB Action Selection
• Contextual Bandits for Real World RL

Hands-on Lab: Understand expected values from different gambling games

Markov Decision Processes

• Examples of MDPs
• The Reward Hypothesis
• Continuing Tasks
• Episodic and Continuing Tasks

Hands-on Lab: Understand Markov Decision Processes by creating a classifier for thrash random writing

Value Functions and Bellman Equations

• Specifying Policies
• Value Functions
• Bellman Equation Derivation
• Optimal Policies
• Optimal Value Functions
• Using Optimal Value Functions to get Optimal Policies

Dynamic Programming

• Iterative Policy Evaluation
• Policy Iteration
• Efficiency of Dynamic Programming

Hands-on Lab: Implement the Bellman equations and Dynamic Programming for a grid world game

Monte Carlo for Prediction and Control

• What is Monte Carlo?
• Prediction
• Action Values
• Blackjack example
• Epsilon-soft policies
• Off-policy learning

Hands-on Lab:

• Implement the off-policy learning methods and compare them with the single policy ones for a grid world game
• Understand Markov Decision Processes by creating a classifier for thrash random writing

On-policy Prediction with Approximation

• Parameterized Functions
• Generalization and Discrimination
• Value Error Objective
• Gradient Descent

Hands-on Lab: Implement your own Flappy Bird AI that learn to play only from the environment