1 Introduction
2 Alignment Data Collection
2.1 Instructions from Humans
2.2 Instructions From Strong LLMs
2.3 Instruction Data Management
3 Alignment Training
3.1 Online Human Preference Training
3.2 Offline Human Preference Training
3.3 Parameter-Effective Training
4 Alignment Evaluation
4.1 Evaluation Benchmarks
4.2 Evaluation Paradigm
5 Challenges and Future Directions
6 Conclusion