1 Introduction
2 Pre-training Corpora
2.1 General Pre-training Corpora
2.2 Domain-specific Pre-training Corpora
2.3 Distribution Statistics of Pre-training Corpora
2.4 Preprocessing of Pre-training Data
3 Instruction Fine-tuning Datasets
3.1 Instruction Category
3.2 General Instruction Fine-tuning Datasets
3.3 Domain-specific Instruction Fine-tuning Datasets
3.4 Distribution Statistics of Instruction Fine-tuning Datasets
4 Preference Datasets
4.1 Preference Evaluation Methods
4.2 Distribution Statistics of Preference Datasets
5 Evaluation Datasets
5.1 Evaluation Domains
5.2 Evaluation Methods
5.3 Distribution Statistics of Evaluation Datasets
6 Traditional NLP Datasets
6.1 Question Answering
6.2 Recognizing Textual Entailment
6.3 Math
6.4 Coreference Resolution
6.5 Sentiment Analysis
6.6 Semantic Matching
6.7 Text Generation
6.8 Text Translation
6.9 Text Summarization
6.10 Text Classification
6.11 Text Quality Evaluation
6.12 Text-to-Code
6.13 Named Entity Recognition
6.14 Relation Extraction
6.15 Multitask
7 Challenges and Future Directions
7.1 Pre-training Corpora
7.2 Instruction Fine-tuning Datasets
7.3 Preference Datasets
7.4 Evaluation Datasets
8 Conclusion