Five ways AI agents break in production

1. Partial success

Seven of ten records processed, three failed, and the run reports complete. Count input and output records yourself and halt on a mismatch; do not trust the agent's self-report.

2. Context degradation

Instructions stop holding partway through a long run — the language switches, the format drifts. Cut the work into shorter units and restate the instruction each time. The long-running shape is itself the cause.

3. Repeating the same failure

Retrying an approach that cannot work, burning time and tokens. Cap attempts: three failures, stop, hand to a human. Without a cap the cost is unbounded.

4. Misread premise

It misinterprets the instruction and then proceeds coherently, so every output is consistently wrong and a single spot check passes. Run one record, then ten, then the rest.

5. Blindness to external change

An API changed, a page moved, credentials expired — and the agent reports that there was no data today. Alert on zero and on counts outside the normal band; a clean exit is not evidence of success.

Monitoring that works

Set the spend cap first

A looping agent calls the API until something stops it. Configure a daily or monthly cap that halts execution, and do not ship to production without one.

FAQ