1 INTRODUCTION

2 BACKGROUND
2.1 Training Paradigms for Visual Recognition
2.2 Development of VLMs for Visual Recognition
2.3 Relevant Surveys

3 VLM FOUNDATIONS
3.1 Network Architectures
3.2 VLM Pre-training Objectives
3.3 VLM Pre-training Frameworks
3.4 Evaluation Setups and Downstream Tasks

4 DATASETS
4.1 Datasets for Pre-training VLMs
4.2 Datasets for VLM Evaluation

5 VISION-LANGUAGE MODEL PRE-TRAINING
5.1 VLM Pre-Training with Contrastive Objectives
5.2 VLM Pre-training with Generative Objectives
5.3 VLM Pre-training with Alignment Objectives
5.4 Summary and Discussion

6 VLM TRANSFER LEARNING
6.1 Motivation of Transfer Learning
6.2 Common Setup of Transfer Learning
6.3 Common Transfer Learning Methods
6.4 Summary and Discussion

7 VLM KNOWLEDGE DISTILLATION
7.1 Motivation of Distilling Knowledge from VLMs
7.2 Common Knowledge Distillation Methods
7.3 Summary and Discussion

8 PERFORMANCE COMPARISON
8.1 Performance of VLM Pre-training
8.2 Performance of VLM Transfer Learning
8.3 Performance of VLM Knowledge Distillation
8.4 Summary

9 FUTURE DIRECTIONS

10 CONCLUSION