Can AI Design Its Own Chips? A New Era for AI Hardware

Did you know that the AI services we use every day are essentially running a massive ‘calculator’ in the background? Imagine this: every time you ask an AI, “What should I have for lunch?” countless semiconductor chips are tirelessly processing information behind the scenes. Recently, there has been a surge in companies within the AI industry determined to build these core ‘AI chips’ themselves. Beyond just renting chips made by others, why are world-leading AI companies starting to design their own hardware?

Why Does This Matter?

Until now, AI development has relied on the massive wall known as ‘Nvidia.’ This is because almost all high-performance AI runs on Nvidia’s Graphics Processing Units (GPUs, devices that process data in parallel). However, as AI models become smarter, the cost of operating these services explodes.

AI development is divided into two main stages. First, there is the ‘training’ process, which requires vast amounts of capital to train the AI. Then, there is the ‘inference’ process, where it talks to users and answers questions. Inference is a daily operational cost that occurs billions of times a day. Reducing these operating costs has become the key lever that determines a company’s survival. Ref 5 In short, owning custom chips has become a powerful competitive advantage that allows companies to boost profits without relying on expensive external components. Ref 3

Understanding It Simply

To make AI hardware easier to understand, let’s use an analogy:

• Training: The process of training the AI to memorize an entire encyclopedia. This requires a super-fast calculator.
• Inference: The process of answering a user’s question based on what it has memorized. Ref 4

Put simply, training is the process of ‘learning how to read thousands of books in a library,’ while inference is the ‘process of a librarian finding the exact answer for a questioner.’ While existing general-purpose GPUs are optimized to scan an entire library quickly, the chips AI companies are creating themselves are designed specifically to perform only the role of the ‘librarian finding answers fast.’ Ref 13 The principle is that if the librarian’s movement is optimized, they can provide answers much faster while using less energy.

Current Situation

Global Big Tech is already taking action:

• OpenAI: Partnered with Broadcom to develop their custom inference chip, ‘Jalapeño.’ Compared to existing Nvidia systems, this chip processes more data at the same power consumption and reduces the latency users experience. Ref 7, Ref 18
• Anthropic: Has formed a custom chip development team to design its own Application-Specific Integrated Circuits (ASICs). Ref 12
• DeepSeek: Building inference-dedicated chips to reduce dependence on Nvidia and Huawei. Ref 1, Ref 20
• Tesla: Has been designing its own chips for running neural networks within its vehicles for years. Ref 6
• Z.ai: Surprisingly, they allowed their own model (GLM-5.3) to participate directly in optimizing the hardware architecture. The AI essentially designed the ‘house’ where it can provide its own answers the fastest. Ref 10, Ref 14

What Happens Next?

The era where hardware and software operate separately is coming to an end. Ref 8 The trend will be ‘integrated optimization,’ where software that perfectly understands the AI model’s characteristics merges with hardware physically arranged to match those characteristics. Ref 9

For us as consumers, an environment will be created where we can use smarter AI faster and for longer at increasingly affordable prices. At the same time, however, it is worth noting that only a tiny number of massive corporations equipped with hardware design capabilities may come to dominate the AI ecosystem.

MindTickleBytes’ AI Reporter View

The sight of AI designing its own hardware is reminiscent of how living organisms adapt their environments to better suit themselves throughout the evolutionary process. Now, the center of competition is shifting beyond ‘how much data has been trained’ to ‘how efficiently can we talk on this infrastructure.’ A new AI era where hardware and software move like a single body is right before our eyes.

References

1. Inference Is the New Oil: Who Controls the Pipe - DEV Community (https://dev.to/antseedai/inference-is-the-new-oil-who-controls-the-pipe-122l)

2. The Future of AI Inference Hardware: Beyond the GPU…

   Thinkia (https://thinkia.com/thoughts/future-ai-inference-hardware-google-tpu/)

3. OpenAI Unveils Jalapeño: Its First Custom Inference Chip, Built With… (https://faq.com.tw/en/hardware/2026-07-10-openai-jalapeno-broadcom-inference-chip-en/)
4. The Silicon Stack War: What OpenAI’s Jalapeño Chip Reveals About… (https://insighttrack.ai/openai-jalapeno-chip-nvidia-inference-vertical-integration/)
5. The Real AI Race Isn’t About Models Anymore — It’s About Chips (https://www.linkedin.com/pulse/real-ai-race-isnt-models-anymore-its-chips-madhankumar-r-a-rj9if)
6. Elon Musk Agrees Tesla’s Early Custom AI Chit… — TradingView News (https://www.tradingview.com/news/benzinga:d7ba980ab094b:0-elon-musk-agrees-tesla-s-early-custom-ai-chit-bet-may-be-more-important-than-ever-backs-tsla-engineer-s-warning-current-compute-shortage-is-only-the-tip-of-the-iceberg/)
7. OpenAI publishes Jalapeño’s first benchmarks at Hot Chips · Promptea (https://www.promptea.me/en/blog/openai-jalapeno-first-benchmarks-hot-chips-2026)
8. Inside the Inference Hardware Revolution Of 2026 - IEEE Spectrum (https://spectrum.ieee.org/inference-hardware-revolution)
9. Large Language Model Inference Acceleration: A Comprehensive Hardware … (https://arxiv.org/html/2410.04466v2)
10. AI Optimizing Itself? The Story of a System Built by My Own Hands (https://gipyeong-lee.github.io/2026/09/17/GLM-Built-Its-Own-Inference-Infrastructure.en/)
11. Computer Science > Hardware Architecture - arXiv.org (https://arxiv.org/abs/2601.05047)
12. Anthropic co-designing custom AI inference chips to bypass costly … (https://www.tomshardware.com/tech-industry/anthropic-to-build-its-own-co-designed-custom-ai-accelerator-for-inferencing-workloads-samsung-reported-to-be-partnering-with-the-claude-ai-maker-for-manufacturing)

13. AI Inference Hardware in 2026: Beyond the GPU

    Woyce (https://woyce.ai/blog/state-of-ai-inference-hardware)

14. Toward Recursive Self-Improvement: How GLM Built Its Own Inference … (https://z.ai/blog/glm-built-its-inference-infrastructure)
15. Top 5 Most Significant and Current AI Hardware Developments … (https://applyingai.com/2025/10/top-5-most-significant-and-current-ai-hardware-developments-openais-chip-pivot-and-beyond/)
16. OpenAI Jalapeño AI chip challenges Nvidia in inference - CNBC (https://www.cnbc.com/2026/08/26/openai-jalapeno-ai-chip-nvidia.html)
17. DeepSeek AI Chip Could Shake Up NVIDIA and Huawei at Once (https://memeburn.com/deepseek-ai-chip-could-shake-up-nvidia-and-huawei-at-once/)