There is heated debate over whether high-scoring models in AI competitions possess genuine intelligence or are merely 'slop'—mimicking data without true understanding.
Imagine this: You’re at school solving a complex math puzzle you’ve never seen before, and the friend sitting next to you suddenly shouts out the correct answer. When you ask, “How did you solve it?” they can’t explain it; they just happened to stumble upon the right answer by luck. That is exactly the situation the artificial intelligence (AI) industry is facing right now.
Recently, a controversy regarding “AI slop” (a term for low-quality data or meaningless generative output) erupted following a competition hosted by Google DeepMind. An AI solved a complex problem and took home a large cash prize, but people have begun to question whether this AI truly possesses “intelligence” or if it simply stumbled upon the right answer in an ocean of data.
Why does this matter?
This isn’t just about whether an AI can solve a problem or not. We are at a critical juncture where we must determine whether the AI chatbots we use every day truly “understand” complex situations or are merely “mimicking” plausible-sounding answers.
If an AI cannot grasp the essence of a problem and solves it by pure chance, it could lead to severe, unforeseen errors if we entrust it with important decision-making. The 2025 competition results show that a gap still exists between AI “getting smarter” and AI “learning data to guess the correct answer” Source: ARC Prize 2025 Results & Analysis.
Understanding the basics
The evaluation framework named “ARC-AGI” was created to test the genuine intelligence of AI. Just as we use logic to solve novel types of problems in school, AI is presented with new puzzle problems it has never learned before. It’s similar to solving “unseen” problems on a college entrance exam.
In the 2025 competition, they utilized more difficult and complex puzzles called “ARC-AGI-2” Source: ARC Prize 2025: Technical Report. 1,455 teams engaged in fierce competition, submitting 15,154 solutions Source: adrianwwwang/Kaggle_Google_deepmind_winner_analysis.
To put it simply, it’s like giving an AI tens of thousands of art tools and complex rules for mixing colors, then asking it to complete a painting it has never seen. Some AI models grasp the rules perfectly to create a flawless painting, while others manage a plausible result through trial and error. The “slop” in question refers to cases like the latter—succeeding by chance without logical understanding.
The current landscape
What is the current level of AI? The top score in the 2025 competition reached only 24% on the ARC-AGI-2 test set Source: ARC Prize 2025: Technical Report. 24 out of 100 is not a passing grade; there is still a long way to go. However, efficiency has improved dramatically, with the processing cost per task dropping to just $0.20 Source: ARC Prize 2025: Technical Report.
Many have hoped for DeepMind’s AI to create a historic breakthrough like “Deep Blue,” which defeated chess hero Garry Kasparov. However, the consensus is that current models are still at the level of learning massive amounts of data to approach answers probabilistically Source: Google DeepMind claims ‘historic’ AI breakthrough in problem ….
What’s next?
Google DeepMind knows that existing problem-solving methods are not enough. So, in March 2026, they launched a massive hackathon (an event where developers concentrate on solving problems in a short time) with a $200,000 prize to create a new evaluation framework Source: Google DeepMind launches Kaggle benchmark contest with $200k to measure AGI capabilities.
The industry is now shifting toward verifying “the thought process an AI goes through to reach a conclusion,” rather than just seeing who can guess the right answer the fastest Source: Google DeepMind Releases Cognitive Framework to Measure AGI Progress, Launches $200K Kaggle Hackathon. Moving forward, AI that proves its “logical validity” will be recognized as the true expert over those that rely on “lucky guesses.”
MindTickleBytes’ AI Reporter Perspective
This situation is a “growing pain” that AI must endure to become smarter. It has become more important than ever to have criteria that distinguish true intelligence—AI that penetrates the essence of a problem and thinks for itself—from AI that has merely memorized vast amounts of data. For the sake of the future we are building, it is time to focus on the “logical process” of finding the answer rather than the “result” of the answer itself.
References
- ARC Prize - 2025 Competition Details (https://arcprize.org/competitions/2025)
- ARC Prize 2025 Results & Analysis (https://arcprize.org/blog/arc-prize-2025-results-analysis)
- Google DeepMind claims ‘historic’ AI breakthrough in problem … (https://www.theguardian.com/technology/2025/sep/17/google-deepmind-claims-historic-ai-breakthrough-in-problem-solving)
- ARC Prize 2025: Technical Report - arXiv.org (https://arxiv.org/abs/2601.10904)
- ARC Prize 2025: Technical Report - arXiv.org (https://arxiv.org/html/2601.10904v1)
- adrianwwwang/Kaggle_Google_deepmind_winner_analysis - GitHub (https://github.com/adrianwwwang/Kaggle_Google_deepmind_winner_analysis)
- Google DeepMind Releases Cognitive Framework to Measure AGI Progress, Launches $200K Kaggle Hackathon (https://creati.ai/ai-news/2026-03-18/google-deepmind-cognitive-framework-measure-agi-progress-kaggle-hackathon/)
-
Google DeepMind launches Kaggle benchmark contest with $200k to measure AGI capabilities AI Primer (https://www.ai-primer.com/engineer/stories/kaggle-measuring-agi-hackathon)
- ARC-AGI-1
- ARC-AGI-2
- ARC-AGI-3
- 1,000 teams
- 1,455 teams
- 2,000 teams
- Developing commercial chatbots
- Building a cognitive evaluation framework to measure AI intelligence levels
- Commercializing autonomous vehicles