1 Introduction
1.1 Why theory is important?
1.2 Roadmap of the paper
1.3 Existing surveys on deep learning theory

2 Approximation theory-based statistical guarantees
2.1 Expressive power of fully-connected networks
2.2 Statistical guarantees for regression tasks

3 Training dynamics-based statistical guarantees

4 Statistical Guarantees of Generative Models

5 Conclusions & Future Topics