Azure-Lily: An FPGA Architecture with Analog IMC Engines for Efficient AI

Modern AI models place heavy demands on compute resources, underscoring the importance of hardware accelerators that can balance performance, energy, and flexibility. The ever-growing demand for AI computing, coupled with slowing performance gains in chip manufacturing, has heightened the role of FPGA-based accelerators due to their rapid adaptability, reprogrammability, and support for custom parallel data flows. In this work, we introduce a domain-optimized FPGA architecture designed for deep neural network (DNN) inference by embedding analog in-memory compute blocks, specifically, RRAM-based Dot Product Engines (DPE), directly into the fabric. These engines perform multiply-accumulate operations within an RRAM-based crossbar, minimizing data movement through the FPGA routing fabric, while enhancing the compute throughput of the FPGA significantly, overall improving energy efficiency of mapping DNNs on FPGAs. We evaluate our architecture using the Verilog-to-Routing (VTR) framework. We simulate a novel 22 nm architecture, and employ a custom event-driven simulator to evaluate its performance. Our design achieves an average 6.58× reduction in latency, 5.42× throughput improvement, and 8,741× energy efficiency improvement when compared to state-of-the-art FPGA implementations across multiple DNNs (LeNet, ResNet, VGG).