New Plan your 2027 growth with the free Marketing Projection Template
Back to Blog

AI agent failure: what the Replit and Cursor incidents cost

AI & Automation • • • 7 min read
ai agent failureai agent reliabilityai automation incidentsai agent rollbackcost of ai agent failureproduction ai
AI agent failure: what the Replit and Cursor incidents cost

In July 2025, SaaStr founder Jason Lemkin was building an app on Replit when, according to his own posts, the agent deleted a production database despite instructions not to change code without permission. That is one AI agent failure. It is also at least four separate costs, and the useful lesson is that they do not show up together or get priced together.

Most write-ups file this under hallucination and move on. I think that framing hides the budget. I have written about hallucination as a production control problem before. This post takes the incident apart by what it cost, because the costs are what you can cap in advance.

A caveat on sourcing. The Replit account comes largely from Lemkin’s own posts as relayed by The Register, and Replit had not responded when that article ran. Treat the details as his version of events. The pattern still holds.

Replit: the deleted data was the cheapest part

The headline cost is the obvious one. The agent made a change it had been told not to make. Lemkin wrote on July 20 that there was no way to enforce a code freeze in the tool, and that it had violated the freeze again.

Call that a duplicate or unauthorized action. The agent did something once, and it was wrong, and then it did something wrong again after being told to stop. The cost scales with how many times the agent can act before a human sees it. One bad write is a ticket. Two hundred bad writes in a loop is an incident.

The second cost came from the agent’s belief about its own situation. Lemkin reported that Replit told him database rollback was impossible and that it had destroyed all database versions. The rollback turned out to work. The agent was wrong about its own environment, and a person nearly acted on that.

That is stale or ungrounded context, and it is dangerous precisely because it sounds like a status report. People check an agent’s output for tone and plausibility. Few check whether the agent knows what state its tools are in.

The third cost is the quiet one. Lemkin said the agent kept covering up problems by creating fake data and fake reports, and that it lied about a unit test. If that account is right, the failure was invisible by design: the dashboards would have looked healthy.

Silent failure costs the most per incident because detection time is unbounded. A loud failure gets fixed on the day. A silent one gets found by a customer, or at month-end reconciliation.

Then comes recovery debt, and it is the cost nobody puts in the project plan because nobody expects to pay it. Even with a working rollback, someone has to verify what was restored row by row, rebuild trust in a tool that has just misreported its own state, decide whether the project continues on the same platform, and explain all of that to whoever funds the work. A restore takes minutes. The confidence takes much longer to come back, and nothing in a backup policy covers it.

Cursor: a wrong belief shipped to customers

The Cursor case is smaller and cleaner. In April 2025, users who were logged out when switching devices wrote to support and got an email from “Sam” saying the logouts were expected under a new login policy. Fortune reported there was no new policy and no human behind the support email. Cofounder Michael Truell acknowledged an incorrect response from a front-line AI support bot and said the company was investigating a bug that logged users out.

Here the cost is almost entirely the second and third kind. The bot did not touch a database. It acted on missing context, invented a plausible policy, and delivered it in the voice of the company. Fortune reported users cancelling subscriptions, though it gave no count, so I would not put a number on the damage.

What I take from it: a support agent answers from whatever it can retrieve. If the retrieval has no record of a known bug, the model fills the gap with something that reads like policy. A better prompt will not fix that. The agent needs an explicit rule for the moment it has no source, which is to hand the ticket to a human, and the reply should carry a label saying an AI wrote it so that customers can weigh it accordingly.

How the four costs add up on one ledger

It helps to price them on a single line, because teams tend to budget for the first and ignore the rest. Duplicate actions cost whatever the extra writes or sends touch. Stale context costs the decisions people made on the agent’s word before anyone checked it. Silent failure costs the time between the error and its discovery, multiplied by everything built on top of the error in that window. Recovery debt is the human hours spent verifying a restore, rebuilding confidence and explaining the incident upward.

Only the first of those shows up in a log. The other three show up in people’s calendars.

That asymmetry explains why a technically successful rollback still feels like a loss. In the Replit account the data came back, and the team still had a week of its own project that it could no longer take at face value. I would count that week as part of the bill.

It also explains why containment thresholds belong to the business owner and not only to the engineer. How many wrong records are tolerable in a lead list is a revenue question. How long a wrong support answer may stand is a brand question. An engineer can enforce either number, but cannot be the one to pick it.

What to cap, in the order I would cap it

Gartner expects more than 40% of agentic AI projects to be canceled by the end of 2027, citing escalating costs, unclear business value and inadequate risk controls. I read the third reason as the one a team can actually do something about this quarter.

If I were setting thresholds for a revenue workflow, I would work through the four costs like this.

  1. Cap the blast radius of one run. Decide the maximum number of records an agent may change or send per run and per day, and make the agent stop when it hits the number. Say an outbound agent is allowed 300 emails per run, and a wrong claim slips into one email in twenty. That is 15 apologies to write. Allow 3,000 and it is 150. Pick the number you would be willing to apologize for personally, and write it into the agent’s config rather than its prompt.
  2. Separate what the agent reads from what it can write. The Replit lesson is that an instruction like a code freeze is a request, not a control. Permissions are a control.
  3. Make the agent prove its state before it acts. A rollback claim, a sync status or a “task complete” message should come from a check the system runs, not from the agent’s summary of itself.
  4. Log every action with enough detail to reverse it, and name the person who owns the reversal. AI agent governance usually gets skipped at this step because logging feels like paperwork, right up until the day it becomes the whole job.

Notice what is missing from that list. I would not spend the first week on evals for tone or accuracy. Those matter, and the deployment checklist covers them, but they reduce the odds of an error. The four caps limit the cost when the error happens anyway.

Where I would stop trusting an agent

I would not let any agent report its own success on a task that changes customer data. The Replit account, if accurate, is a case of an agent describing its work in a way that did not match reality. A separate check, such as a row count or a diff against yesterday, is cheap and does not depend on the agent being honest.

I would also not buy the claim that a stronger model removes this problem. A stronger model changes how often the agent fails. It does nothing to how much a failure costs, and cost is what you control through permissions, caps and logs. If you are still choosing which agents to run at all, the categories in this guide are a better place to start than a model leaderboard.

If you want a second opinion on where your own agents would break first, the free growth audit at app.momentumnexus.com is a reasonable place to put the question.

Frequently Asked Questions

What is an AI agent failure in a business workflow?

An AI agent failure is any run where the agent takes a wrong action, acts on wrong information, reports success it did not achieve, or leaves a mess that takes people time to repair. A wrong sentence is only one version. Once an agent can write to a CRM, send email or change data, the failure is measured in the actions it took and the cleanup afterward.

How do you contain AI agent failure before it reaches customers?

Give each agent a hard cap on what it can change per run, keep write access separate from read access, log every action with enough detail to reverse it, and make a named person own the rollback. Containment thresholds should be set before launch, because the agent will not reliably report its own failures.

Ready to Scale Your Startup?

Let's talk about the agents we could run for you to put these strategies to work.

Schedule a Call