1 Introduction  
An Example Domain

2 Technical Background  
2.1 Multi-Armed Bandits  
2.2 Markov Decision Processes  
2.3 Partially Observable Markov Decision Processes  
2.4 Reinforcement Learning  
2.5 Bayesian Learning

3 Bayesian Bandits  
3.1 Classical Results  
3.2 Bayes-UCB  
3.3 Thompson Sampling

4 Model-based Bayesian Reinforcement Learning  
4.1 Models and Representations  
4.2 Exploration/Exploitation Dilemma  
4.3 Offline Value Approximation  
4.4 Online Near-myopic Value Approximation  
4.5 Online Tree Search Approximation  
4.6 Methods with Exploration Bonus to Achieve PAC Guarantees  
4.7 Extensions to Unknown Rewards  
4.8 Extensions to Continuous MDPs  
4.9 Extensions to Partially Observable MDPs  
4.10 Extensions to Other Priors and Structured MDPs

5 Model-free Bayesian Reinforcement Learning  
5.1 Value Function Algorithms  
5.2 Bayesian Policy Gradient  
5.3 Bayesian Actor-Critic

6 Risk-aware Bayesian Reinforcement Learning

7 BRL Extensions  
7.1 PAC-Bayes Model Selection  
7.2 Bayesian Inverse Reinforcement Learning  
7.3 Bayesian Multi-agent Reinforcement Learning  
7.4 Bayesian Multi-Task Reinforcement Learning

8 Outlook