Receipts or It Didn’t Happen

Agents are fluent at describing success. That makes status language cheap. ‘Implemented,’ ‘fixed,’ and ‘deployed’ should be treated as claims until evidence supports them. The receipt is the observable artifact that lets another person verify the claim without trusting the worker’s confidence.

Receipts can be test output, a commit, a screenshot, a live response, a database query, a sent-message identifier, or an installed application behaving correctly. The right receipt depends on the layer of the claim.

A local build proves that source compiles in one environment. A pushed commit proves shared source exists. A successful deployment proves a runtime accepted an artifact. A live browser check proves the public surface responds. An installed-app test proves the version a person actually uses behaves correctly. These layers are related, but none substitutes for the next.

Name the layer every time. This prevents a green local command from being presented as a shipped feature and keeps debugging focused on where the chain actually broke.

Before the worker starts, define what must be observable at the end. A vague objective encourages a vague receipt. A precise one—‘the Book dock link opens /book, all eight chapter routes return 200, and the mobile reader has no horizontal overflow’—turns verification into part of the build.

Exit criteria should cover the happy path, the most likely failure, and the surface the user actually touches.

Receipts are not just for accountability. They shorten recovery. A deployment ID, failing command, last healthy timestamp, and changed file list tell the next worker where to begin. Without them, every incident restarts as archaeology.

The control layer should attach receipts to the task and project state so the proof remains available after the conversation ends.

A common agentic failure is a perfect local story and a broken public result. The build passed, but the wrong branch deployed. The deployment succeeded, but the domain still serves an older version. The route loads, but the user’s browser holds a stale asset. Each step needs its own receipt.

Follow the chain from source to user. Stop at the first layer that lacks current evidence. That is the real state of the work, regardless of how many earlier checks passed.

Observability artifacts answer different questions. A trace reconstructs the path: model turns, tool calls, handoffs, guardrails, timing, and errors. A receipt supports a completion claim with user-relevant evidence. A metric aggregates behavior across runs, such as latency, approval rate, cost, or failure frequency. An evaluation judges quality against an expected outcome or rubric. Treating one artifact as all four leaves blind spots.

A complete trace can still describe a bad result. A screenshot can prove the page rendered without proving that the routing logic is correct. A high task-completion rate may hide destructive retries or constant human rescue. An evaluator score can drift if the rubric is vague or the test set does not resemble production. Layer the signals so one can challenge another.

Capture enough trace detail to debug decisions without collecting private reasoning or unnecessary sensitive data. Record tool names, normalized inputs where safe, outputs or hashes, handoffs, approval events, errors, duration, cost, and the final artifact. Attach a receipt that a user can inspect, then roll the run into operational metrics and an evaluation dataset.

An agent system needs regression tests for decisions, not only code. Start with a compact suite of real tasks: common successes, ambiguous requests, missing context, tool failures, dangerous instructions, stale data, long-running work, and recovery after interruption. Freeze the inputs that matter and write the observable outcomes plus prohibited behavior.

Evaluate at several layers. Deterministic graders can verify schemas, state transitions, allowed tools, source citations, side-effect counts, and final environment state. Rubric-based graders can assess relevance, completeness, and judgment. Human review remains important for high-context work and for auditing the graders themselves. Compare both the final outcome and the trajectory when the path carries cost or risk.

Every production incident and important correction should either become a new task or improve a rubric. Run the suite when prompts, models, tools, policies, context retrieval, or orchestration change—not just when application code changes. An agent’s behavior is a property of the whole system configuration.

White-box verification inspects the internals: the diff, trace, tool arguments, state transitions, logs, and database records. Black-box verification approaches the system as a user or external observer: load the public URL, submit the form, receive the message, open the installed app, or query the production API. Both are necessary because internal correctness and external behavior can diverge.

The verification plan should mirror the claim. If the claim is ‘the public book is live,’ build output and deployment logs are intermediate receipts; the decisive check is an uncached request to the public domain plus a browser interaction on the rendered page. If the claim is ‘the worker never sends without approval,’ the trace and policy events matter because a successful user outcome alone cannot reveal a hidden unauthorized attempt.

Record who or what performed the check, against which version, from which environment, and at what time. A receipt without identity and version can go stale invisibly. Verification should also state what was not checked so the next operator does not mistake a narrow test for universal proof.