Unveiled by d-Matrix, 'Raptor' is a 3D-DRAM architecture that stacks compute units and memory directly, solving the bottleneck of data movement and significantly improving AI inference performance.
Imagine you are trying to find a very important book in a library. However, the book is in a deep basement warehouse, and the reading room is on the top floor. How exhausting would it be to climb the stairs dozens of times every time you want to borrow a book? The generative AI we use today is in a similar situation. The AI’s “brain”—its compute unit—suffers from a “memory bottleneck,” wasting energy and slowing down as it fetches data from the warehouse (memory) where information is stored.
At the recent ‘Hot Chips 2026’ conference, semiconductor company d-Matrix unveiled an exciting technology to solve this dilemma: a 3D-DRAM accelerator called ‘Raptor.’ Now, AI no longer needs to run to a distant warehouse to find data. It is as if a bookshelf is attached right beneath the AI’s brain.
Why Is This Important?
When generative AI generates complex answers, it actually repeats the process of instantly retrieving vast amounts of data from memory and sending it to the compute unit. The movement of data during this process becomes a massive barrier that hinders AI performance. This is professionally referred to as the “Memory Wall.” Source: d-Matrix Raptor 3D-DRAM Architecture at Hot Chips 2026
To make the AI assistant features on our smartphones faster or to make AI services that generate complex videos in real-time more affordable, this wall must be torn down. Raptor was created precisely to break through that technical limitation. Source: d-Matrix’s Raptor puts stacked DRAM to work on inference bandwidth
Easy to Understand
To use a simple analogy, existing AI chips were like “a restaurant where the kitchen and the dining area are far apart.” For a chef (compute unit) to cook, they had to walk a long distance to the ingredient warehouse (memory) to fetch items. In contrast, Raptor is like a “state-of-the-art open kitchen where the chef can retrieve ingredients from right next to them.”
| Raptor uses a method of attaching a compute die (semiconductor fragment) made with TSMC’s 4-nanometer (nm) ultra-fine process directly onto custom DRAM. Source: Hot Chips 2026: d-Matrix stacks AI accelerator directly on custom DRAM for 100 TB/s per card This is called a “3D-DRAM” structure. Because the memory is stacked vertically right beneath the compute unit, the physical distance the data must travel is extremely short. [Source: d-Matrix stacks DRAM directly · Hacker News | Zeli](https://zeli.app/story/49695065) |
Thanks to this, Raptor provides a phenomenal data transfer bandwidth of 100 TB/s (terabytes per second) per card. This dramatically reduces the chronic bottleneck that typical AI chips suffer from. Source: d-Matrix’s Raptor 3D-DRAM Achieves SRAM-Class Bandwidth at 1/10th the HBM Power
Current Situation
According to a presentation at Hot Chips 2026 by Sudeep Bhoja, CTO of d-Matrix, this technology was developed in collaboration with Meta. Source: Hot Chips 2026 - d-Matrix Current high-performance AI chips typically use expensive memory called HBM (High Bandwidth Memory), and a key feature of Raptor is that it can achieve similar or better performance with about 1/10th the power efficiency of HBM. Source: d-Matrix’s Raptor 3D-DRAM Achieves SRAM-Class Bandwidth at 1/10th the HBM Power
However, this technology is not meant to replace all HBM on the market right away. Currently, Raptor is specialized for inference efficiency (the process where a trained AI actually generates answers), and it is expected to serve as a cornerstone of the entire AI infrastructure that handles massive data. Source: d-Matrix’s Raptor Chip Hits 105 TB/s in Real Silicon Tests, But It’s Not an HBM Replacement Yet
What’s Next?
We are moving away from an era of simply pouring in “more power” toward an era of accelerating AI through “more efficient structures.” If 3D-DRAM technologies like Raptor become widespread, the response speed of the AI services you use will increase, while power consumption will decrease. This ultimately means the social cost required to run AI will be lowered. Source: d-Matrix’s Raptor 3D-DRAM Achieves SRAM-Class Bandwidth at 1/10th the HBM Power
It is worth anticipating how much smaller and smarter the semiconductor industry will get in combining memory and compute units, and how far this new roadmap presented by Raptor will change the future of artificial intelligence.
References
- d-Matrix Raptor 3D-DRAM Accelerator for Generative Inference
- d-Matrix Raptor 3D-DRAM Architecture at Hot Chips 2026
- Hot Chips 2026 - d-Matrix
- Hot Chips 2026: d-Matrix stacks AI accelerator directly on custom DRAM for 100 TB/s per card
- Hot Chips 2026: d-Matrix Raptor 3D-DRAM - lukebest.github.io
-
[d-Matrix stacks DRAM directly · Hacker News Zeli](https://zeli.app/story/49695065) - D-Matrix Raptor 3D-DRAM Accelerator for Generative Inference at Hot Chips 2026
- Hot Chips 2026 - Hot Chips 2026
- d-Matrix Raptor: 3D-DRAM with 100 TB/s for AI inference
- d-Matrix’s Raptor 3D DRAM Achieves SRAM-Class Bandwidth at 1/10th the HBM Power
-
[Jobs and Employment at d-Matrix Simplify Jobs](https://simplify.jobs/c/d-Matrix) - d-Matrix’s Raptor 3D DRAM performance - Overview
- d-Matrix’s Raptor puts stacked DRAM to work on inference bandwidth
- SRAM laid out horizontally
- Compute units and DRAM stacked vertically
- Distributed cloud servers
- Higher power consumption
- About 1/10th the energy efficiency
- Reduced data transfer speed
- The 'memory bottleneck' caused during data movement
- Internet connection speed issues
- Lack of AI training data