1. Introduction
2. Backgrounds
2.1. Evolutionary Algorithms
2.2. Reinforcement Learning
3. The Taxonomy of RL-EA
4. Integration Methods for RL and EA
4.1. Direct Integration
4.2. Indirect Integration
5. RL Assistance Strategies Used in RL-EA
5.1. Generating Solution
5.2. Learnable Objective Function
5.3. Algorithm Selection
5.4. Operator Selection
5.5. Sub-population Selection
5.6. Parameter Adaptation
5.7. Other Strategies
5.8. Discussion of DRL-assisted Strategies
6. Attribute Settings of RL in RL-EA
6.1. Setting of State
6.2. Setting of Reward Function
6.3. Principle of Action Setting
7. Application of RL-EA
7.1. Continuous Optimization Problems
7.2. Combinatorial Optimization Problems
7.3. Open Datasets in Main Application Domains
8. Future Research Directions
9. Conclusion