1 Introduction
2 Preliminaries
2.1 Differential Privacy
2.2 Deep Learning with Differential Privacy
3 Privacy Attacks
3.1 Training Data Extraction
3.2 Membership Inference Attacks (MIA)
3.3 Model Inversion
4 Solutions for Preserving Privacy in LLMs: Data
4.1 Anonymization
5 Solutions for Preserving Privacy in LLMs: Model
5.1 Training Large Language Models with Differential Privacy
5.2 Inference with Differential Privacy
5.3 Federated Learning with Differential Privacy
5.4 Machine Unlearning
6 Tools & Frameworks
7 Discussion and Conclusion