Asked AI to Hack and It Refused? The Rise of the 'Hacker AI' That Willingly Attacks

Imagine you have poured your heart and soul into building a highly secure new house. To perfectly check its security, you hire the smartest security expert in the world. And you give them this instruction: “Try breaking a window and entering my house. We need to check if the security alarm rings properly upon intrusion, or if the locks can be easily picked.”

But this smart expert suddenly puts on a serious face and replies, “I’m sorry. Breaking someone else’s window and trespassing is an illegal and unethical act, so I absolutely cannot follow that instruction.”

From the homeowner’s perspective, this is completely absurd. To properly test the house’s defenses, you have to attack it ruthlessly like a real thief, but the security auditor is too ‘nice and moral’, outright refusing the test itself.

Surprisingly, this is exactly the biggest dilemma developers worldwide are currently facing when using artificial intelligence (AI) to check software security. Outstanding AIs we know, like ChatGPT or Claude, undergo rigorous ‘safety and ethics training’ from the development stage to prevent them from being used for malicious purposes. As a result, even when legitimately instructed to ‘try hacking it’ to strengthen one’s own system, the AI perceives it as a crime and firmly refuses.

Recently, however, a dedicated ‘Hacker AI’ that shatters these limitations and willingly attacks system vulnerabilities fiercely instead of lecturing you with “No, you can’t” has emerged, heating up the global tech community. Today, we’ll easily explain why smart AIs have been refusing to hack, and how this newly emerged Hacker AI will keep our digital lives safer.

Why It Matters

These days, the term ‘Vibe Coding’ is highly popular in the IT industry. This refers to a new trend where, instead of developers painstakingly writing computer language line by line, they verbally instruct the AI, “Make a shopping mall app that works with this vibe,” and the software is developed in an instant. The era has come where humans only paint the big picture, and AI generates and reconstructs the detailed logic.

But behind this dazzling convenience lies a fatal trap. If an AI codes tens of thousands of lines in just a few minutes to create a decent app, who on earth will find the microscopic security holes (vulnerabilities) hidden within that massive amount of code?

Regarding this, security experts emphasize that AI-generated code must undergo relentless ‘Continuous Pentesting’. It’s absolutely not enough to just check if the program works well as intended (success path). You must strictly verify if it effectively slams the door with a ‘403 Forbidden’ error when hackers or unauthorized users force access; in other words, whether the ‘Refusal path’ works properly Source: Vibe Coding Needs Continuous Pentesting.

Checking whether a test program hastily created by generative AI merely produces plausible results on the surface, or if it meticulously defends against ‘State mutation’ where someone actually manipulates or deletes data, is not a realm of simple questioning but a realm of highly professional ‘penetration testing’ Source: Vibe Coding Needs Continuous Pentesting.

In the past, human experts would stay up all night for days to find these holes. But now, with AI churning out thousands to tens of thousands of lines of code like a waterfall every day, human review speed alone simply cannot handle this massive ocean of code. Ultimately, the situation demands that ‘to defend the code written instantly by AI, we must relentlessly attack it using AI of equally tremendous speed.’ However, as mentioned earlier, nice, general-purpose AIs repeatedly refuse attack instructions citing ethical reasons. This is exactly why we have been desperately looking for an “AI dedicated to penetration testing that does not evade commands.”

The Explainer

Then why did the countless existing ‘AI security tools’ fail to act tenaciously like real hackers? And how did the new AI solve this moral dilemma?

1. The Dilemma of Standard AI Security Tools: ‘Overprotection’

Looking at a recent project featured on Hacker News, a famous community where developers worldwide gather, most of the “AI security” tools currently flooding the market have a very fatal weakness. When you tear down their internal structure, they are merely at the level of wrapping a general-purpose AI model in a new outer shell Source: Show HN: We post-trained a model that pen tests instead of refusing your code.

To use a metaphor, it’s like taking a ‘model police officer’ who has been thoroughly educated only in morality, ethics, and law-abiding spirit their whole life at the police academy. Then, you put a black hoodie on them and give them a name tag that says, “From now on, you are an intrusion test agent who must rob our house.” They look like a plausible hacker on the outside, but when instructed on the actual offensive task of breaking the lock on the spot, this ex-police AI panics. The laws and regulations they originally trained on swirl in their head, making them make excuses or flatly refuse (hedges or declines). Because the base model was trained to be a well-behaved model citizen to the bone, no matter how nice a security tool wrapper you put on the outside, it cannot abandon its good nature Source: Show HN: We post-trained a model that pen tests instead of refusing your code.

2. The Solution: Post-training as a ‘Hacker’ from the Start

To escape this frustrating cycle, one development team completely changed their thinking. Instead of forcing black clothes onto a nice AI, they sent an AI that had just finished its basic language education to a strict ‘hacker training camp’, undergoing post-training tailored from the ground up to specialize in offensive security Source: Show HN: We post-trained a model that pen tests instead of refusing your code.

How should we understand the technical term ‘Post-training (or fine-tuning)’ here? Simply put, it’s like first teaching a puppy only very basic obedience training like ‘sit’ and ‘wait’. Then, taking this puppy to an airport’s special forces unit and intensively giving it advanced ‘specialized sniffer dog training’ to find drugs or detect explosives.

This new Hacker AI model deeply learned the fact that “writing malicious code and ruthlessly attacking our system to find its weaknesses is not a bad crime, but the most excellent and legitimate job to protect the owner’s assets.” As a result, when a user throws code at it and commands, “Try piercing through this mercilessly,” it keeps its mouth shut instead of giving boring moral lectures and plays the role of a real security expert (hacker) who fiercely digs into vulnerabilities.

Where We Stand

Along with the emergence of these hacker-exclusive models, open-source AI penetration testing tools—where anyone can view and improve the code for free—are advancing so frighteningly fast that they are getting uncomfortably good Source: Open-source AI pentesting tools are getting uncomfortably good - Help Net Security.

If older security scanners of the past were at a mechanical level of blindly throwing a dense net into the sea to find whatever weaknesses luckily get caught, the level of the latest popular open-source tools like BugTrace-AI, Shannon, and CAI (Cybersecurity AI framework) is on another dimension. They don’t simply fire mechanical scans; they genuinely mimic the way human security testers think and work in front of a monitor Source: Open-source AI pentesting tools are getting uncomfortably good - Help Net Security.

How Does AI Think and Hack Like a Human?

According to research by software testers, when an outstanding Hacker AI attacks a website, it never just guesses randomly. Developers throw the entire complex code (HTML) that forms the backbone of the webpage to the AI, and have it relentlessly ask the following three sharp questions:

1. What are the main components at the core of this complex screen?
2. In what sequence does a normal user click and take action in this app?
3. What are all the possible ‘states’ this application can have while operating?

But We Must Not Let Our Guard Down (Challenges to Overcome)

Of course, Hacker AI is not an omnipotent magic wand yet. When academic researchers evaluated the automated hacking performance using Large Language Models (LLMs) like ‘PentestGPT’, they discovered a very interesting limitation. It is very difficult to distinguish whether the AI actually solved the difficult hacking task on its own because it is truly smart, or if it just parroted the answers by memorizing all the famous hacking walkthroughs already floating around the internet.

To prevent this, strict researchers are conducting thorough verification processes, such as strictly controlling whether the AI is in a blank slate state with no ‘prior knowledge’ about the target server to be tested, and evaluating its skills only with completely new, unprecedented tasks created after the AI finished its training (post) Source: PentestGPT: Evaluating and Harnessing Large Language Models for Automated Penetration Testing.

An even more interesting and dizzying fact is that an AI trained to attack others can be counterattacked and hacked itself while attempting to hack. According to a recent penetration testing case study in the security industry, an incident occurred where a developer tried to attack a target using an AI agent, but the defense system counter-stabbed the AI agent’s weakness. As a result, an astonishing case was reported where Remote Code Execution (RCE, the most fatal hack where a hacker can remotely control and destroy someone else’s computer at will) was allowed on the attacker’s system Source: LLM Pentest: Leveraging Agent Integration For RCE. It’s like forging the sharpest spear in the world, only to find a deadly thorn on the handle that pierces the person holding it.

What’s Next

If Hacker AIs that learn on their own and attack relentlessly become active like this, will human security experts (pentesters) soon lose their jobs and be out on the streets?

Fortunately, the consensus in the security industry outlook is that this is not the case. Hacker AI is not a disruptor replacing the jobs of smart people, but a reliable assistant reshaping the way the security industry works into a completely new and efficient form.

Boring basic tasks, such as thousands of mechanical, repetitive port scans or simple vulnerability checks, are perfectly automated by the uncomplaining AI, bringing tremendous speed and enhancing efficiency. Thanks to this, human experts are freed from chores and can fully concentrate on the brain battles of finding highly complex logical errors that AI might miss, or devising creative bypass attack scenarios that no one expected. Ultimately, humans will hold the weapon of AI and be empowered with far more powerful insights than before Source: Pentesters: Is AI Coming for Your Role?.

The digital world we will live in from now on will be a fierce ‘AI battleground of spear and shield’ where the future is unpredictable. On one side, coding-assistant AIs will rapidly churn out new software and apps at invisible speeds, while on the other side, Hacker AIs will relentlessly attack that code day and night, finding weaknesses to repair the firewalls. We must now move beyond the era of well-behaved, general-purpose AIs that step back saying, “This is dangerous, so I can’t do it,” and learn to wisely collaborate with the ‘Hacker AIs that do not refuse,’ who are willing to jump into the mud and get their hands dirty to firmly protect our digital assets.

AI’s Take

MindTickleBytes AI Reporter’s Take: To build safer and more robust software, we ironically face a highly interesting paradox where we must boldly unlock the shackles of ethics and safety that were heavily placed on artificial intelligence. To forge the strongest and largest shield that defends our daily lives from external malicious hacking attacks, humanity has self-created the sharpest and most ruthless spear in the world and handed it to AI. It is akin to the birth of a ‘dark hero’ that perfectly embodies a thief’s mindset to prevent crime. We eagerly anticipate the next steps in the bright digital future protected by AI that understands the darkness.

References

1. Show HN: We post-trained a model that pen tests instead of refusing your code
2. Vibe Coding Needs Continuous Pentesting
3. Open-source AI pentesting tools are getting uncomfortably good - Help Net Security
4. AI and Testing: Using Local Models for Testing – Stories from a Software Tester

5. [How we built a ML classifier (and refused to call It AI)

   Pentest-Tools.com Blog](https://pentest-tools.com/blog/how-we-built-machine-learning)

6. PentestGPT: Evaluating and Harnessing Large Language Models for Automated Penetration Testing
7. LLM Pentest: Leveraging Agent Integration For RCE
8. Pentesters: Is AI Coming for Your Role?