I. INTRODUCTION  
II. FUNDAMENTALS  
A. Reinforcement Learning  
B. Evolution Strategies  
C. Comparison  
III. FUNDAMENTAL ALGORITHMS  
C. Comparison  
IV. DEEP REINFORCEMENT LEARNING VERSUS EVOLUTION STRATEGIES  
A. Parallelism  
B. Exploration  
C. Non-Markov Settings  
D. Learning in Multiagent Settings  
V. HYBRID DEEP REINFORCEMENT LEARNING AND EVOLUTION STRATEGIES ALGORITHMS  
A. Comparison  
VI. APPLICATIONS  
A. Deep Reinforcement Learning Applications  
B. Evolution Strategy Applications  
C. Comparison  
VII. CHALLENGES AND FUTURE RESEARCH DIRECTIONS  
VIII. CONCLUSION