I. INTRODUCTION
II. BACKGROUND ON DEEP NEURAL NETWORKS (DNNs)
A. LAYERS
B. TRAINING AND INFERENCE
C. DNN MODELS
D. SPIKING NEURAL NETWORKS (SNNs)
III. HARDWARE SOLUTIONS AND CO-DESIGN
B. TEMPORAL ARCHITECTURES AND SOFTWARE OPTIMIZATIONS
C. SPATIAL ARCHITECTURES AND DATAFLOW PROCESSING
E. HARDWARE-AWARE NEURAL ARCHITECTURE SEARCH
F. FULL PRECISION VS QUANTIZED IMPLEMENTATIONS
G. METHODS FOR MODEL COMPRESSION
H. ACTIVATIONS AND WEIGHTS SPARSITY: STRATEGIES AND ENCODING
I. APPROXIMATE COMPUTING FOR DEEP LEARNING AND THEIR RESILIENCE
J. EMBEDDED VS CLOUD COMPUTING
K. SNNs HARDWARE ACCELERATORS
IV. MEMORY HIERARCHY
V. DEEP LEARNING SECURITY
A. ADVERSARIAL ATTACKS
B. ADVERSARIAL DEFENSES
VI. BENCHMARKING
A. FRAMEWORKS
B. DATASETS
C. NEURAL NETWORKS MODEL METRICS
D. HARDWARE ACCELERATOR METRICS
VII. CHALLENGES AND THE ROAD AHEAD
VIII. DISTINCTION FROM OTHER SURVEYS
IX. CONCLUSION