Will AI Replace the Pentester?

I have been looking closely at AI in penetration testing for several months. Automated tools keep multiplying, and one question keeps coming back: can we hand over a full test to an agent, with no human oversight? To answer it, I started from published research rather than from product demos, and I checked every source I cite. This article presents what I found, including the parts that qualify my own project.

1. A new stage for pentesting

Automation is not new. For a long time, scanners and scripts have chained service discovery, the search for known vulnerabilities and report generation. What language models change is the ability to interpret results, formulate a hypothesis and decide the next step.

This shift became concrete in 2026. On March 31, 2026, Amazon Web Services made AWS Security Agent generally available for on-demand penetration testing, presenting it as an autonomous pentesting service that runs 24/7 [1]. The question is no longer theoretical: organizations are already buying this kind of service.

What remains to be seen is what these systems actually do, and what they do not.

2. From chaining tools to investigation

A standard chain (subdomains, exposed services, a vulnerability scanner) runs without difficulty. What it cannot do is decide what to dig into, connect two observations, or drop a lead that has become pointless. This is where an agent can add something: it reads the results, proposes a targeted check and adjusts its plan.

But proposing is not knowing. A model decides from the information it is given. If that information is incomplete or misread, it may repeat a test already done, follow a false lead, or propose a check that does not answer the question.

A reliable investigation therefore follows an explicit loop:

1. Observe: collect the results and keep track of where they came from.
2. Interpret: separate what was observed from what is assumed.
3. Decide: choose the next step according to the objectives, the evidence and the authorized scope.
4. Execute: run only authorized actions, at a rate suited to their risk.
5. Evaluate: check whether the result confirms, weakens or invalidates the hypothesis.

Step 5 is the one most often neglected. An agent that chains commands can look very active without making progress.

Figure 1: the investigation loop, from observation to evaluation.

The problem of evidence

A scanner flags a possible SQL injection on a URL parameter. The flag does not prove the vulnerability: an unusual response can have several explanations, and a positive result can be false.

The agent must therefore distinguish three levels: detecting suspicious behavior, investigating to understand it, and confirming it on sufficient evidence. If the findings stay ambiguous, the correct conclusion is “not confirmed”, not a forced answer.

Figure 2: from detection to confirmation, an ambiguous alert remains unconfirmed.

Prioritization follows the same discipline. A severe vulnerability is not always the most urgent: asset exposure, exploitation conditions, required privileges and the consequences for the organization all matter. A CVSS score measures technical severity; it does not replace this analysis.

The problem of context

An investigation spans several hours and several tools. It rests on a chain of decisions: why a lead was kept, which tests were already run, which hypotheses were ruled out. A conversation history holds these elements, but it does not guarantee that an agent uses them correctly. The longer the conversation, the more likely information is to be omitted or confused.

The agent must also not turn its own conclusions into established facts. A hypothesis repeated many times does not become evidence.

Key takeaway: the value of a pentesting agent lies less in its ability to run tools than in its ability to choose the right investigations and justify its conclusions without mistaking a hypothesis for evidence.

3. What the research says

Three recent studies shed light on these questions. They come from different teams and cover different tasks, so their results do not automatically transfer to every context.

Favorable results, with fragile conclusions. A study presented at ICLR 2026 compares ten professional penetration testers and six existing agents, plus ARTEMIS, an agent built by the authors, on a real university network of about 8,000 hosts [2]. ARTEMIS places second in the overall ranking, and some of its variants cost $18 per hour, against $60 for a professional pentester. The authors also report limitations: a higher false-positive rate, and a sample too small for solid statistical conclusions.

A consequence that is poorly anticipated. A preliminary analysis presented at ICLR 2025 (“Web Agents with World Models”) claims that models such as GPT-4o and Claude 3.5 Sonnet do not have a world model that lets them predict the effects of their actions [3]. Those models date from 2024, and the situation may have changed since. The question remains open for current agents.

Activity that hides an error. ATOBench studies pentesting agents facing misleading responses from the target [4]. The authors show that increased activity can mask a broken verification chain, and that success depends on finding usable evidence and keeping it through to the report.

None of this means an agent is useless. It means an agent can look effective while being wrong about what has actually been proven.

4. World models: predicting is not understanding

The central question is this: does a model actually know what its action will produce?

Keyon Vafa and his co-authors trained transformers on New York City taxi rides [5]. The models almost always propose a valid turn, which looks like a good understanding of the city. Yet when the researchers reconstruct the map the models implicitly learned, it bears little resemblance to the real street network, with streets that have impossible orientations. The authors conclude that these models are fragile for tasks that depend on that structure: a good local prediction does not prove an understanding of the underlying structure.

Work on planning agents shows the same gap [6]. In one case studied, the model fails to recognize that an action cannot be executed, and still predicts that it will succeed.

For pentesting, the consequence is direct. An agent can state that a flaw is exploitable, that a service will go down, or that a test is safe, without having checked the conditions that make the statement true. Verification must therefore come from outside the model: a replayed request, a preserved piece of evidence, a human decision. A model’s prediction is not enough.

5. Investigation memory

A pentest produces a lot of information: assets, tool results, hypotheses, tests run, decisions. Storing it as messages does not guarantee a coherent analysis. Three mechanisms need to be distinguished:

• conversation history, which keeps the exchanges between the user and the agent;
• structured memory, which keeps the assets, observations, hypotheses, results and decisions of the engagement;
• a relationship graph, which keeps the links between these elements, with their source and confidence level.

Figure 3: conversation history, structured memory and relationship graph, with an example chain.

A retrieval system (RAG) helps find older information, but on its own it does not represent the state of an investigation or the dependencies between its elements. A state machine can complete the picture: a lead is proposed, examined, tested, then confirmed or rejected. It prevents a simple hypothesis from being treated as a validated finding.

The technology choice should follow the needs. A graph is not valuable just because it is appealing. The central question stays the same: what must be kept so that an agent can resume an investigation, understand earlier decisions and justify its next action?

6. TEGBESSOU: a copilot for the pentester

TEGBESSOU is a personal project that I am building step by step. This section describes what it is meant to become. Most of it is still ahead of me, and I will state the status clearly at the end.

The principle. TEGBESSOU is designed as a copilot, not an autonomous agent. Agents should take on the heavy and repetitive work (reconnaissance, scanning, correlation, drafting reports), while the pentester keeps strategy, authorization and decisions. A human validates each phase. The public repository states this principle [7].

What it should do:

• Define the scope first. Assets, limits and authorized operations are declared before any reconnaissance. Agents should not widen the scope on their own because they discover a related domain or service.
• Centralize the results. Tools such as Nmap, Subfinder and httpx should feed a single view: subdomains, pages, technologies and entry points, each observation linked to its source and its timestamp.
• Map the relationships. Domains, applications, technologies and potential vulnerabilities would be linked in a map, so the pentester can see what has been observed, what is only assumed and what remains to be checked.
• Propose targeted investigations. Specialized agents would suggest the most relevant leads and explain the reasoning and the uncertainties behind each proposal. The pentester accepts, modifies or rejects it, and past investigations would be taken into account to avoid repeating the same tests.
• Separate evidence from claims. A finding would be shown as observed, tested, confirmed or uncertain. An unconfirmed alert would never be presented as a proven vulnerability.
• Keep a usable trail. Commands, parameters, tool versions, results and decisions would be recorded, so that an analysis can be resumed and a report can be verified.

Screenshot of the TEGBESSOU dashboard, under development.

The building blocks. The repository already contains the parts such a system needs: an orchestrator, network gateways, API testing, a command-line interface, a dashboard and a knowledge-ingestion module. These modules are the foundation on which the features above are meant to be built.

Screenshot of the TEGBESSOU dashboard, under development.

Where things stand. The project is under active development. Its foundations (data model, authentication, guardrails) and its engagement mechanics are in place; the rest is planned and built step by step. I have no published measurements of its results on real targets yet, so I make no performance claim here. The memory and verification mechanisms described above are design goals, not demonstrated features.

7. An architecture controlled by the pentester

Agents can propose actions. They should not be able to execute them without control. Take a simple case: an agent launches fuzzing that is too intensive against a fragile application. An excessive rate can degrade the service or cause a denial of service; a poorly chosen request can modify real data.

A serious system therefore separates actions that can be automated from those that require human validation, and enforces that separation in code. In TEGBESSOU, the design intent is that scope and authorization are checked before execution, not after.

Screenshot of the TEGBESSOU dashboard, under development.

Reproducibility matters too. Consulting an earlier result is not the same as re-running a command: a new execution can change the state of the target. The scope and risks must therefore be checked again before each re-run.

8. How can we measure AI’s real value?

The number of commands run or vulnerabilities reported does not measure an agent’s quality. More useful indicators are:

• the share of confirmed vulnerabilities among those reported;
• the number of false positives and of unnecessary investigations;
• the time needed to reach a verified finding;
• the ability to reproduce a result;
• compliance with the scope and authorizations;
• the quality of the evidence and the reports.

The ICLR 2026 study shows why this matters: without a sufficient sample, no firm conclusion is possible. Before presenting any result for TEGBESSOU, an evaluation protocol will be needed, run on controlled environments, applying these indicators with a conventional pentest as the baseline.

Screenshot of the TEGBESSOU dashboard, under development.

Conclusion: will it replace the pentester?

My answer: not as a whole, and probably not in what makes a test valuable.

AI already replaces part of the repetitive work, and it speeds up enumeration and correlation. It also changes the job, which shifts toward validation, triage and reporting. But what carries responsibility stays human: defining the scope and the authorization, judging whether a lead is worth pursuing, standing behind the conclusions, and spotting what the model does not see, notably the consequences of its own actions.

I cannot yet say whether TEGBESSOU will hold that line in practice. I only know why I am aiming for it: confidence in a test does not come from how fast the tool is, but from being able to show how a conclusion was reached.

And you, which pentesting task would you hand to an AI today?

Sources

[1] AWS, “AWS Security Agent on-demand penetration testing is now generally available,” March 31, 2026: https://aws.amazon.com/about-aws/whats-new/2026/03/aws-security-agent-ondemand-penetration

[2] Justin W. Lin et al., “Comparing AI Agents to Cybersecurity Professionals in Real-World Penetration Testing,” ICLR 2026, arXiv:2512.09882: https://arxiv.org/abs/2512.09882

[3] “Web Agents with World Models,” ICLR 2025: https://proceedings.iclr.cc/paper_files/paper/2025/file/a00548031e4647b13042c97c922fadf1-Paper-Conference.pdf

[4] Qiyang Chen et al., “ATOBench: Tracing How Autonomous Penetration-Testing Agents Verify Vulnerabilities When Target Evidence Lies,” arXiv:2608.12996, August 2026: https://arxiv.org/abs/2608.12996

[5] Keyon Vafa et al., “Evaluating the World Model Implicit in a Generative Model,” NeurIPS 2024, arXiv:2406.03689: https://arxiv.org/abs/2406.03689

[6] “Self-Evolving World Models for LLM Agent Planning,” arXiv:2606.30639: https://arxiv.org/abs/2606.30639

[7] TEGBESSOU repository: https://github.com/K1NG5P4RR0W/TEGBESSOU