We pointed a coding agent at our own app and told it to break in

Most of what we ask agents to do is constructive. Build the pipeline, wire the endpoint, migrate the schema. Building has a clear finish line, and the agent knows when it is done because the thing it made either runs or it does not.

Attacking is different. There is no finish line. A pen test is done when someone with judgment decides it is done, and until then the honest answer to "is this secure" is "we have not found a way in yet." We wanted to know whether an agent could operate in that space, so we did the obvious thing: we pointed one at our own app and told it to get in.

Here is what we learned.

An attacker has no acceptance criteria

The first runs were bad, and they were bad in a way we did not expect.

When you hand an agent a build task, the task carries its own definition of success. When you hand an agent "find vulnerabilities," it will happily report that it found them. It flagged missing security headers as critical. It called a public docs endpoint an exposure. It wrote up a login form that had no rate limiting as if it were a breach.

None of this was wrong, exactly. It was just an agent doing what agents do: producing output that looks like the thing you asked for. A list of findings is easy to generate. A list of findings that a security engineer would stand behind is not.

The lesson was that "find problems" is not a goal, it is an invitation to invent them.

We had to define what a real finding was

So we stopped asking for findings and started asking for proof.

We rewrote the objective around evidence. A finding only counts if the agent can show the request it sent and the response that came back, and explain why that response means something is wrong. No proof, no finding. Suspicions go in a separate section clearly marked as things a human should look at, not as results.

That one change moved most of the noise out of the report and into a "worth a look" pile, which is exactly where a junior tester's hunches belong.

It also changed the agent's behavior. Once it knew it had to produce a request and a response, it stopped theorizing and started actually sending traffic.

The dangerous part was not the agent attacking. It was the agent wandering

An agent with a network connection and instructions to probe for weaknesses is a thing you want on a very short leash.

We locked the target down in code, not in the prompt. The client the agent uses to send requests refuses to talk to any host except the one we configured. Point it at a different domain, a different port, even the staging URL when it was told to test local, and it raises an error and stops. This is not a rule we asked it to follow. It is a wall it cannot walk through.

We did this because prompts are suggestions and code is not. An instruction like "only test this host" is exactly the kind of thing that gets lost three steps into an autonomous run. A hard lock does not get lost.

If you take one thing from this post, take that. When an agent can act on the outside world, put the limits in the code, not the instructions.

The interesting findings needed two accounts

The shallow checks, the ones about headers and open endpoints, an agent can do on its own all day. The findings that actually matter in a product like ours are about tenant isolation: can one customer see another customer's data.

You cannot test that with one login. You need two, in two separate organizations, and then you have the agent authenticate as the first and try to reach the second's data.

This is where the agent was genuinely useful. Give it two sets of credentials and a clear question, "acting as org A, can you read anything that belongs to org B," and it will methodically walk the endpoints and try. It is patient in a way a person doing this by hand at 6pm is not.

The result we got back was the good kind of boring. Every cross-organization request came back denied, in both directions. The agent tried, showed its work, and found nothing. That is a report we trust precisely because it shows the requests it made.

What the agent is, and what it is not

We are careful about what we call this, internally and with clients.

What we built is a self-assessment. It is an agent that runs a known set of checks against our own app, on a short leash, and writes up what it can prove. It is fast, it is repeatable, and we can run it on every meaningful change instead of once a year.

It is not an independent penetration test. It does not replace a human firm looking at the system with fresh eyes and no assumptions, and it does not produce the letter of attestation a customer's security team asks for. Pretending otherwise would be its own kind of success theatre, and it would fall apart the first time a real assessor asked a question the checks do not cover.

Holding that line matters. The value of the tool is that it tells us the truth about our own app between the real audits, not that it lets us skip them.

The part worth stealing

If you take anything from this, take the pattern, because it holds well beyond security.

Define success as evidence, not as output, or the agent will hand you output. Put the hard limits in code, because a limit written into a prompt will not survive a long run. Give the agent the patient, repetitive work it genuinely does better than a tired human at the end of the day. And stay honest about the edge of what it can do, especially when the thing on the other side of that edge is a customer asking whether their data is safe.

The agent never got into our app. What it gave us instead is cheaper and, honestly, more useful: a repeatable way to keep asking the question, and to mean it when we answer.