INTRODUCTION
LEARNING OBJECTIVES
2.1 Knowledge Distillation
2.2 Knowledge Expansion
2.3 Knowledge Adaptation
2.4 Multi-Task Learning
KNOWLEDGE FORMULATION
3.1 Knowledge Construction
3.2 Knowledge Optimization
TEACHER-STUDENT ARCHITECTURES
4.1 Single Teacher
4.2 Multiple Teachers
LEARNING SCHEMES
5.1 Online Learning
5.2 Self-Learning
APPLICATIONS
OPPORTUNITIES AND FUTURE WORKS
7.1 Teacher-Student Architecture Design
7.2 Quality of Knowledge
7.3 Theoretical Understanding of Regression-Based Knowledge Learning
CONCLUSIONS