MyDot Consultancy

Featured image: AI-generated conceptual illustration. Sources checked 26 October 2026.

When an agent produces the wrong result, an operator needs to answer ordinary questions: what request started the work, which records were read, which tools ran, what failed and what changed? A transcript alone may not contain those answers. Observability connects the conversation to the execution that happened around it.

The goal is not to expose a model’s private reasoning or create a dramatic visualization of intelligence. It is to reconstruct observable work with enough precision to diagnose errors, verify controls and support a user. That requires deliberate event design and equally deliberate protection of sensitive information.

Start with an incident you need to explain

Imagine a hypothetical assistant that prepares renewal briefings. A user reports that a briefing names the wrong account owner. The operator needs to determine whether the source record was stale, the wrong account was retrieved, the model misread the result or a later template inserted the wrong value.

A useful trace links the user request to the account lookup, returned record reference, generation step and saved artifact. It records relevant versions and status without necessarily copying every field into a log. The operator can follow the evidence to the layer that failed.

The OpenAI Agents SDK tracing guide describes traces and spans for observing agent execution. These mechanisms can organize events, but the application must decide what business identifiers and outcomes to attach and which sensitive details to exclude.

Give each run a stable identity

Use a run identifier that follows the work across model calls, tool requests, approvals and background jobs. When a task resumes after a pause, preserve the relationship to the original request. If a subtask runs separately, link it to the parent rather than leaving it as an unrelated event.

Keep identifiers free of unnecessary personal information. A customer email address is a poor default trace ID when a random reference can serve the same purpose. Maintain any mapping to business records in an appropriately controlled system.

Record the deployed configuration: model identifier, prompt or instruction version, tool version and policy version where relevant. Without that context, comparing a failed run with a later successful one can be misleading. The system may have changed between attempts.

Log decisions as observable events

A useful event says that a particular tool was requested, approved, rejected, started, completed or failed. It can include safe argument summaries and a reference to the result. Avoid replacing these events with a generated narrative that may omit uncertainty or describe intended work as completed work.

For consequential actions, connect the proposal, reviewer decision and execution result. If an approval was rejected, record that outcome distinctly. If a tool call timed out after a possible write, preserve uncertain completion rather than normalizing it into a generic failure.

The SDK human-in-the-loop documentation describes approval interruptions and resuming execution. Observability should show those pauses as meaningful states, so an operator can distinguish a task waiting for a person from a task stuck in a technical failure.

Protect the trace as a data product

Model prompts and tool responses can contain confidential records, personal information and credentials accidentally returned by an integration. Logging them indiscriminately creates another repository that must be secured, retained and eventually cleaned up. More data is not automatically better observability.

The SDK tracing documentation explicitly discusses sensitive data in generation and function spans and provides configuration controls. Review those settings before deployment. Where possible, record event categories, record references and outcomes instead of full content, and apply redaction before export to a monitoring service.

Define who can inspect traces, for what purpose and for how long. Support staff may need status information without access to the underlying customer document. Security investigators may need a controlled path to deeper evidence. Build those access levels deliberately rather than giving every developer a copy of production conversations.

Measure the workflow, not only the model

A slow run may spend most of its time waiting for a connector or reviewer. A costly run may repeat a failing lookup. Capture durations and counts at the relevant boundaries so the team can identify the actual bottleneck. Model-token metrics alone cannot explain the whole service.

Useful operational views include completed, blocked, denied, canceled and failed runs; tool error categories; approval wait time; and verification failures. Define each status consistently. A task that stopped generating text should not automatically enter the completed bucket.

• Can an operator locate a run from the user-visible task reference?
• Can they identify the last verified milestone and current blocker?
• Can they connect an external change to its approval?
• Can they distinguish attempted work from confirmed outcomes?
• Can they inspect configuration versions without reading private content?
• Can they revoke or stop the affected capability when necessary?

Use traces to improve tests

A production failure can become a valuable regression case after appropriate privacy review and simplification. Preserve the conditions that caused the failure, not every incidental detail of the original conversation. Label whether the case reflects a model error, a tool problem or an application-control defect.

The OpenAI Agents SDK testing guide includes workflow inspection and failure-injection patterns. Use controlled tests to reproduce the relevant behavior, then verify the fix. A trace explains what happened once; a test helps determine whether the repair continues to work.

Prefer an actionable alert to a noisy dashboard

Define alerts around conditions someone can address: repeated authorization failures, unusual outbound destinations, stuck approvals or a rise in verification errors. Assign an owner and response path. A dashboard with no operating decision behind it can become another place where failures are visible but unattended.

Good agent observability makes the work legible. It tells a business what the system attempted, what the environment confirmed and what remains unresolved. With that evidence, teams can improve reliability without collecting every private detail or pretending that a trace reveals the model’s entire internal process.

Sources and further reading

OpenAI Agents SDK: Tracing. Trace/span mechanisms and sensitive-data configuration; checked October 2026.

OpenAI Agents SDK: Human-in-the-loop. Approval interruptions and resume states; checked October 2026.

OpenAI Agents SDK: Testing. Workflow inspection and failure-testing recipes; checked October 2026.