Memory Semantics over UCIe: A Scalable Architecture for High-Bandwidth and Power-Efficient On-Package Memory

Emerging computing workloads such as Artificial Intelligence (AI) and data-intensive applications face increasing pressure from the memory wall, as existing on-package memory solutions are unable to deliver the power-efficient bandwidth demands. We propose extending the Universal Chiplet Interconnect Express (UCIe)TM standard with optimized memory semantics to meet the bandwidth demands. Our approaches enable cost-effective, power-efficient, modular, and scalable on-package memory solutions across the entire compute continuum. <p xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Our architecture leverages existing DRAM technologies (LPDDR and HBM) through a logic die. We also propose a novel approach for the long-term where the DRAM dies natively implement UCIe instead of traditional parallel memory buses. Compared to conventional LPDDR and HBM-based solutions, our UCIe-based designs achieve up to 10x higher bandwidth density, 3x lower latency, and 3x power reduction.