1. Introduction
2. Background
2.1. General Concepts
2.2. Neural Language Models
2.3. Pre-trained Language Models
3. Multilingual
3.1. Importance of the Multilingual Tasks
3.2. Multilingual Models Performance
3.3. Technology Evolution
3.4. Architecture and Hardware Requirements
3.5. Multilingual Datasets
3.6. Categorization of Multilingual Tasks
4. Multilingual Applications and Tasks
4.1. Translation
4.2. Speech Recognition
4.3. Sentiment Analysis
4.4. Intent Detection and Slot Filling
4.5. Dependency Parsing
4.6. NER
4.7. Question Answering
5. Challenges and Outlook
5.1. Existing Challenges
5.2. Future Directions
6. Conclusion