An AI agent did the wrong thing: how to trace it from the action log

Find the run first. Search by order number, appointment ID or timestamp, then export every record in sequence, because one failing step rarely explains itself and the steps before it do.

A useful log holds the system prompt version, the goal text, each tool call with its JSON arguments, the tool’s raw response and the model’s reasoning summary. A missing field is itself a finding.

Paste the export into a spreadsheet with one row per step, and mark the first step where the outcome went wrong, not the last one that shows the damage.

Compare every response against the source system. If the order lookup returned two customers named Rao and the agent picked the first, the agent did not hallucinate, because it acted on ambiguous data.

Empty arrays are the sneaky ones. A calendar tool that returns no free slots after a timeout looks identical to a fully booked day, so check the HTTP status and the response time.

Fix the tool when it failed quietly. Return explicit error codes, add record identifiers to each result and stop sending the agent partial data, so a failure looks like a failure.

First cause: wrong facts from a tool. The agent reasoned sensibly on poor input, so the cure lives in the integration code and not in the prompt wording, and a rewrite would only hide it.

Second cause: no limit. It refunded an order because refunds were allowed and nothing capped them, so the cure is a rule outside the model, such as a refund ceiling or human approval.

Third cause: a vague goal. Told to tidy the schedule, it cancelled bookings it judged duplicates, so rewrite the goal with named actions, forbidden actions and a rule to ask when unsure.

Make the failed run a test. Replay the inputs against the changed setup to confirm the outcome differs, and keep it in a folder of past failures rerun after every prompt, tool or model change.

Check the permissions the agent holds. Give it the narrowest API key that does the job, so a mistake cannot delete records or send a payout, and drop OAuth scopes it never used.

Correct the damage too. Reopen the order, restore the slot and tell the customer, because the log lists every record the run touched and nothing should be left half-fixed.

Add a plain line per action. Store logs long enough to cover your complaint window, and restrict access to them, since they contain names, addresses and order details that only a few staff should read.

Alert on patterns, not complaints. Watch for cancellations above a normal daily count, refunds near the ceiling and repeated tool timeouts inside one run, because each warns you before a customer writes in.

Read ten random runs monthly. It takes under an hour, and surprises such as a skipped tool call or an odd refund show up in your own review before customers find them.