I. Introduction
A. Aim
B. Contributions of this work
C. Organization
II. Activation Functions: A Historical Perspective
III. Biological vs Artificial Neurons
IV. Desired Characteristics of the Activation Functions
V. Taxonomy of Activation Functions
A. Characterization based taxonomy
B. Application based taxonomy
VII. Complex-Valued Activation Functions
VIII. Quantized Activation Functions
IX. A Quest Towards an Optimal Activation Function
A. Parametric Activation Functions
B. Stochastic/probabilistic adaptive activation functions
D. Ensemble adaptive activation functions
C. Fractional adaptive activation functions
X. Performance of Some Fixed and Adaptive Activation Functions for Classification Tasks
XI. Activation Functions for Physics-Informed Machine Learning
A. Differentiability
B. Range
XII. Summary
A. Some future directions