Contents  
1 Introduction  
1.1 The Motivation for Alignment  
1.2 The Scope of Alignment  
2 Learning from Feedback  
2.1 Feedback Types  
2.2 Preference Modeling  
2.3 Policy Learning  
2.4 Scalable Oversight  
2.5 Weak-to-Strong Generalization  
3 Learning under Distribution Shift  
3.1 The Distribution Shift Challenge  
3.2 Algorithmic Interventions  
3.3 Data Distribution Interventions  
4 Assurance  
4.1 Safety Evaluations  
4.2 Interpretability  
4.3 Human Values Verification  
5 Governance  
5.1 The Role of AI Governance  
5.2 The Multi-Stakeholder Approach  
5.3 Open Problems  
5.4 Rethinking AI Alignment from a Socio-Technical Perspective  
6 Conclusion  
6.1 Key Challenges in the Alignment Cycle  
6.2 Key Traits and Future Directions in Alignment Research