1 Introduction  
2 Training Paradigms of LLMs  
2.1 Traditional Fine-Tuning Paradigm  
2.2 Prompting Paradigm  
3 Explanation for Traditional Fine-Tuning Paradigm  
3.1 Local Explanation  
3.2 Global Explanation  
3.3 Making Use of Explanations  
4 Explanation for Prompting Paradigm  
4.1 Base Model Explanation  
4.2 Assistant Model Explanation  
4.3 Making Use of Explanations  
5 Explanation Evaluation  
5.1 Explanation Evaluations in Traditional Fine-tuning Paradigms  
5.2 Evaluation of Explanations in Prompting Paradigms  
6 Research Challenges  
6.1 Explanation without Ground Truths  
6.2 Sources of Emergent Abilities  
6.3 Comparing Two Paradigms  
6.4 Shortcut Learning of LLMs  
6.5 Attention Redundancy  
6.6 Shifting from Snapshot Explainability to Temporal Analysis  
6.7 Safety and Ethics  
7 Conclusions