2.1 AMD Instinct MI350 Series GPUs: CDNA 4-Based 3D-Stacked 3nm XCDs and 6nm IODs for AI Applications

The AMD Instinct MI350 Series GPU, based on a <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$4^{\text {th }}$</tex>-generation AMD CDNA architecture, features eight stacked accelerator complex dies (XCD), two I/O dies (IOD) interconnected with AMD Infinity Fabric, and interfaces for eight stacks of 12-hi HBM3E memory. The MI350 Series GPU delivers over <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$3 \times$</tex> generational inference performance increase by improving energy efficiency, increasing compute throughput, supporting new lower-precision FP6/FP4 datatypes, and increasing both memory bandwidth and capacity.