Samsung presented its advancements in Processing-in-Memory (PIM) technology at the Hot Chips 2026 conference, focusing on the integration of Multiply-Accumulate (MAC) units within LPDDR5X memory chips. This approach aims to leverage the higher internal bandwidth of memory chips and reduce latency compared to traditional compute cores. The LPDDR5X-PIM chips feature 16 banks, each equipped with a PIM block that can access data without the limitations of the external bus, achieving a bandwidth of 614 GB/s, significantly higher than the 76.8 GB/s of standard DRAM accesses.
The PIM blocks consist of a MAC tree and control logic, allowing for efficient processing of low precision formats. Each block can perform multiple MAC operations per data clock cycle, with an aggregate throughput of 2.4 TOPS for the chip. When multiple LPDDR5X chips are used together, the total throughput can reach 9.6 INT8 TOPS, comparable to Intel’s Meteor Lake NPU, although this configuration would require substantial system memory.
Samsung's PIM technology maintains compatibility with the standard LPDDR5X protocol while introducing additional compute capabilities. Special row addresses are designated for mode control, allowing the chip to switch between single-bank and multi-bank modes, which enables commands to be executed across all banks. The software is expected to load model weights into DRAM, with computations performed in multi-bank mode.
Challenges remain in software implementation, as PIM modes alter the behavior of standard DRAM commands, complicating multitasking and memory access. Samsung suggests isolating PIM regions in memory to mitigate these issues, but this could limit overall memory bandwidth and performance. The need for careful management of memory accesses in a multitasking environment is highlighted, as concurrent threads could lead to unintended computations and incorrect results.
Samsung's PIM technology presents potential for high-performance computing in various devices, but its integration into existing systems may require significant adjustments to both hardware and software paradigms. The company advocates for expanding the DRAM interface to simplify the adoption of in-memory compute commands and enhance the interaction between memory controllers and CPU cores.