1 Introduction
2 Related Surveys
3 Background
3.1 Multi-agent Environment
3.2 Stochastic Game
3.3 Game Theory
3.4 Machine Learning
4 Learning in a Multi-agent Environment
4.1 Benefits of MARL
4.2 Challenges of MARL
5 Prospects of MARL
5.1 Simulating MARL Tasks
5.2 MARL Training Schemes
5.3 Agent Awareness
5.4 Multi-agent Credit Assignment
5.5 Communication
5.6 Modeling Other Agents
5.7 Ad-Hoc Team-Play
5.8 Social Learning
6 Concluding Remarks