1 Introduction

2 Low-Rank Adaptation (LoRA)
2.1 LoRA
2.2 Theoretical Analysis
2.3 Efficiency in Practice
2.4 Beyond Fine-tuning

3 Downstream Adaptation Improving
3.1 Breaking the Low-rank Bottleneck
3.2 Dynamic Rank Allocation
3.3 Optimizing the Learning Procedure
3.4 Combining with Other Learning Paradigms

4 Cross-task Generalization
4.1 Mixture with Manually Designed Weights
4.2 Mixture with Learnt Weights
4.3 Mixture of LoRA Experts

5 Efficiency Improving
5.1 Parameter Reduction
5.2 Parameter Quantization
5.3 Parallel LoRA Computing Frameworks

6 LoRA for Federated Learning
6.1 Data Heterogeneity
6.2 Device Heterogeneity
6.3 Model Heterogeneity
6.4 Parameter Privacy

7 Applications of LoRA
7.1 Language Tasks
7.2 Vision Tasks
7.3 Multimodal Tasks

8 Conclusion and Future Direction
8.1 LoRA for GaaS
8.2 LoRA for Continued Pre-training
8.3 LoRA for Autonomous Agents