1 INTRODUCTION  
1.1 Motivations and Contributions  
1.2 PAC Framework of This Survey  
1.3 Organization of The Survey  
2 FEW-SHOT LEARNING VS. ACTIVE LEARNING  
2.1 Few-shot Learning  
2.2 Active Learning  
3 APPROXIMATELY GENERALIZATION ANALYSIS  
3.1 A More General Concept  
3.2 Learning on Small Data  
4 A GEOMETRIC PERSPECTIVE  
4.1 Fréchet Mean  
4.2 Euclidean Mean  
4.3 Non-Euclidean Mean  
4.4 Kernel Mean  
5 OPTIMIZATION SOLVERS  
5.1 Euclidean Gradients  
5.2 Non-Euclidean Gradients  
5.3 Stein Gradient  
6 LEARNING ON SMALL DATA REPRESENTATION  
6.1 Transfer Learning on Small Data  
6.2 Reinforcement Learning on Small Data  
6.3 Contrastive Learning on Small Data  
6.4 Graph Representation Learning on Small Data  
7 PARAMETER OPTIMIZATION WITH SMALL DATA: A META FASHION  
7.1 Bilevel Fashion  
7.2 Task-Distribution Fashion  
7.3 Feed-Forward Fashion  
8 CHALLENGES  
8.1 Scenarios  
8.2 Data  
9 CONCLUSION