1 Introduction  
2 Challenges in Reinforcement Learning  
2.1 Restricted Expressiveness in Offline Learning  
2.2 Data Scarcity in Experience Replay  
2.3 Compounding Error in Model-based Planning  
2.4 Generalization in Multitask Learning  
3 Foundations of Diffusion Models  
4 The Roles of Diffusion Models in RL  
5 Applications of Diffusion Models  
6 Summary and Future Prospects