AI-Generated Code: 4 Out of 10 Are Flawed? The Betrayal of 'GPU Kernels'

Imagine you asked a brilliant math expert to solve a complex problem. The expert confidently provided an answer, and checking it with a few simple examples confirmed they were all correct. But what if you later found out that nearly half of the problems that expert solved were actually garbage? You would feel a great sense of danger, going beyond simple embarrassment.

Recently, the situation with GPU kernels (essential code for fast data calculation in graphics processing units) created by Artificial Intelligence (AI) is exactly like this. AI-generated code was previously evaluated as ‘perfect,’ but in front of a new verification tool, that flashy performance is being revealed as an ‘illusion.’

Why is this important?

GPU kernels are like engines that are indispensable for training and executing AI models. If this engine is even slightly wrong, AI training efficiency drops significantly, or minor errors in the result values occur. Until now, because it was difficult for humans to inspect every piece of AI-generated code, it had been receiving passing grades with test code created by the AI itself.

However, it has been revealed that this method has a serious loophole. If a company applies flawed AI-generated code directly to its services, it could lead to performance degradation as well as unexpected system errors. Source: LLM-Generated GPU Kernels Often Flawed, New Verifier Reveals

Simply put

How about this analogy for the situation? Existing AI code tests are like saying you got a perfect score just by answering the ‘first question on a standardized test.’ According to researchers, existing test methods used ‘loose’ approaches, running code with a small number of random inputs and approximating the results. Source: 39.5% of ‘Correct’ LLM-Generated GPU Kernels Are Broken: A …

On the other hand, the ‘contract-grade’ verifier developed this time is much stricter. It’s as if 12 different obstacles (12 adversarial gates) have been installed to inspect every corner of the code. This tool carefully evaluates not just whether the code produces the correct answer, but whether it is efficient (is the speed appropriate?), whether it wastes memory excessively, or whether it has slyly tricked the test to make it look good. Source: GitHub - rakib-nyc/kernwright: Contract-grade, adversarial …

Where do we stand now?

The researchers re-graded 2,638 GPU kernels that were officially recognized as ‘correct’ in the past using this new verification tool. The results were shocking. It turned out that among the code that perfectly passed existing methods, a staggering 39.5% to as much as 62.1% were actually flawed. Source: 39.5% of ‘Correct’ LLM-Generated GPU Kernels Are Broken: A …

This figure is a painful indicator of how uncritically we have accepted code created by AI. Source: LLM-Generated GPU Kernels Often Flawed, New Verifier Reveals Currently, for higher precision, this verifier compares results with a slower but accurate reference model, independently proving its correctness. Source: A Contract-Grade Verifier for LLM-Generated GPU Kernels, and …

What will happen in the future?

The software development process using AI will become stricter in the future. Beyond simply writing code quickly, ‘contract-based verification,’ which mathematically validates whether the written code ‘actually works properly,’ will become an essential step. It is highly likely that developers will go through such powerful filtering processes instead of using AI-proposed code immediately in the future. AI is also now facing an era where it is demanded to have a higher level of ‘responsibility’ for its output.

---

MindTickleBytes’ AI Reporter’s Perspective

While AI’s productivity is astounding, the precision of its output remains an area that humans must verify directly. This research serves as an important warning showing how dangerous it is to blindly trust code created by AI.

References

1. A Contract-Grade Verifier for LLM-Generated GPU Kernels, and … (https://arxiv.org/html/2608.12700v1)
2. LLM-Generated GPU Kernels Often Flawed, New Verifier Reveals. (https://learnijoy.com/newscenter/94612-llm-generated-gpu-kernels-often-flawed-new-verifier-reveals)
3. 39.5% of ‘Correct’ LLM-Generated GPU Kernels Are Broken: A … (https://zeli.app/en/story/49301417)
4. GitHub - rakib-nyc/kernwright: Contract-grade, adversarial … (https://github.com/rakib-nyc/kernwright/tree/main)
5. A Contract-Grade Verifier for LLM-Generated GPU Kernels, and a Native Blackwell Backward for the Gated-Linear-Recurrence Family (https://arxiv.org/abs/2608.12700)