1. Introduction  
2. Problem Formulation  
2.1 Policy Search for Multi-Armed Bandits  
2.2 Policy Search for Contextual Bandits  
3. PAC-Bayes Bounds for Bandits  
3.1 PAC-Bayes and Martingales  
3.2 A Unified PAC-Bayes Bound  
3.3 Offline Bandit Example  
3.4 Relation to Existing Methods  
4. PAC-Bayes Reward Bounds  
4.1 Importance Sampling  
4.2 Clipped Importance Sampling  
4.3 Weighted Importance Sampling  
5. PAC-Bayes Regret Bounds  
6. Optimising PAC-Bayes Bandit Bounds  
6.1 The Choice of Prior  
6.2 Optimising Bound Parameters  
7. Experimental Comparison  
7.1 Benchmarks  
7.2 Regret Bounds  
7.3 Reward Bounds  
8. Conclusion