In-Memory Computing Elements for Energy-Efficiency Optimization in Deep Neural Network Inference: A Literature Review

Deep neural networks are facing emerging challenges in energy efficiency, primarily driven by two trends: the development of extremely large-scale models, such as large language models with hundreds of billions of parameters, and the deployment of AI workloads on edge computing devices. As an emerging computing paradigm, in-memory computing offers a promising solution to overcome the resource overhead associated with data movement in the von Neumann architecture, which separates memory and computing units. It significantly enhances both energy efficiency and computational throughput. While substantial research has focused on optimizing novel device materials or algorithms, there remains a relative lack of systematic review regarding macro-level in-memory computing components that bridge device-level circuit elements and system-level computing-on-chip architectures. This survey provides a review of in-memory computing macros for energy-efficient deep neural network inference. The research findings are categorized into three representative directions: (1) mixed floating-point and integer architectures, (2) outlier-aware quantization schemes, and (3) support for sparse computations. These correspond respectively to optimization strategies for computational precision, numerical representation, and energy efficiency in computational scale. Through a comparative analysis of the trade-offs among energy efficiency, precision, and functional adaptability across different designs, this study aims to offer insights for the development of in-memory computing macros and outline prospective directions for future macro designs.