# Add observability to your AI system

You cannot fix what you cannot see. Capture the prompt, every tool call, the
latency of each step, and the final outcome of each run, then link them by a
shared id. A bad answer stops being a mystery. You trace it back to the exact
step that went wrong.

## What is observability for an AI system?

A normal program is deterministic: same input, same output, so a stack trace
tells the whole story. An AI system is not. The same prompt can take a different
path on two runs, call different tools, and land on a different answer. When a
user reports "it gave me the wrong total," you have nothing to grep unless you
recorded what happened.

Observability
is that recording. For an AI system it rests on three kinds of signal:

| Signal | What it is | Answers |
| --- | --- | --- |
| Traces | One run, broken into nested steps | Where did this run go wrong |
| Metrics | Counts and timings over many runs | Is latency or cost creeping up |
| Logs and events | Discrete things that happened | What did this exact step see |

You want all three. Metrics tell you something is off, traces tell you which run,
and logs tell you why.

## Capture the whole run as a trace

The single most useful structure is the trace. A
trace is one user request from start to finish. Inside it, each step is a
span:
the model call, each tool call, each retry. Spans nest, so a tool call that
itself triggers a model call shows up as a child of that tool.

The trace id is the thread that ties it together. Stamp one id on the whole run
and propagate it to every step. When a user reports a bad answer, you ask for
that one id and pull the entire run: the prompt that started it, the tools it
called and in what order, what each returned, and where the time went.Do not export spans synchronously inside the request path, and never let a telemetry failure break a run. A slow or down collector then adds its own latency, or takes your agent offline, when you wrapped a try in the wrong place. Use a batched, async exporter and swallow its errors so observability can never become the outage.

## What should I capture from each run?

Capture enough to reconstruct a run without guessing. The checklist below is the
core of it.

- **The input prompt.** What the user asked, plus the system
  instructions in force at the time. A wrong answer often starts with an
  ambiguous prompt or a stale instruction. This is sensitive, so treat it
  carefully (see redaction below).
- **The resolved context.** For a retrieval system, the chunks that were
  fed to the model, as well as the query. If the retrieved context was
  wrong, the answer never had a chance. See
  [make RAG answers accurate](/guides/make-rag-answers-accurate).
- **Every tool call.** The tool name, the arguments, the result, and whether it
  succeeded. A tool that returned an error or empty result is the most common
  hidden cause of a confidently wrong answer.
- **Latency per step.** Time each span as well as the whole run. A slow run is
  usually one slow step. You cannot optimise what you have not measured.
- **Token usage and cost.** Input and output tokens per call. Cost surprises and
  runaway loops both show up here first. If you use
  [prompt caching](/guides/prompt-caching), track cache reads separately so you
  can see the savings.
- **The outcome.** Did the run finish, error, refuse, or hit a limit. Tag each
  run with a clear terminal state so you can count failures without reading
  every trace.
- **A correlation id.** The trace id, on every signal. Without it the other
  fields are scattered notes you cannot reassemble.

A useful rule: if a field would change your diagnosis of a bad run, capture it.
If it would not, leave it out so your traces stay readable and cheap to store.

## Use a standard schema

Do not invent field names per service. The OpenTelemetry (OTel) project
publishes semantic conventions
for generative AI, so a model call records the same attribute names everywhere.
Name things this way and your traces work in any compliant backend instead of
locking you to one tool.

The convention prefixes AI fields with `gen_ai`. A few you will use constantly:

```json
{
  "gen_ai.operation.name": "chat",
  "gen_ai.provider.name": "anthropic",
  "gen_ai.request.model": "claude-sonnet",
  "gen_ai.usage.input_tokens": 1840,
  "gen_ai.usage.output_tokens": 320
}
```

The exact attribute set evolves, so treat the published convention as the source
of truth rather than memorising it. The point is the discipline: one agreed name
per concept, so a token-usage chart works no matter which service produced the
span.

## Turn it on in Claude Code

If you build with the Claude Code CLI, the recording is already built in. Claude
Code exports metrics, events, and (in beta) traces over OTel, so you do not write
any instrumentation yourself. You enable it with environment variables and point
it at a collector.

```bash
# Enable telemetry
export CLAUDE_CODE_ENABLE_TELEMETRY=1

# Choose exporters (configure only what you need)
export OTEL_METRICS_EXPORTER=otlp
export OTEL_LOGS_EXPORTER=otlp

# Point at your OTLP collector
export OTEL_EXPORTER_OTLP_PROTOCOL=grpc
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317

# Run it
claude
```

The same settings live in a project config file if you prefer not to export them
by hand:

```json
{
  "env": {
    "CLAUDE_CODE_ENABLE_TELEMETRY": "1",
    "OTEL_METRICS_EXPORTER": "otlp",
    "OTEL_LOGS_EXPORTER": "otlp",
    "OTEL_EXPORTER_OTLP_PROTOCOL": "grpc",
    "OTEL_EXPORTER_OTLP_ENDPOINT": "http://localhost:4317"
  }
}
```

Out of the box you get metrics such as `claude_code.token.usage` and
`claude_code.cost.usage`, plus events for prompts, tool decisions, and API
errors. Each user prompt opens a `claude_code.interaction` span, and the model
calls and tool calls under it nest as child spans, so a single run reads as one
tree.

For setup you can shorten the export interval so data shows up fast, then reset
it for normal use. The metrics interval defaults to 60 seconds:

```bash
export OTEL_METRIC_EXPORT_INTERVAL=10000
```

The official walkthrough is the
[Claude Code monitoring docs](https://code.claude.com/docs/en/monitoring-usage);
check it for the current full list of metrics and events before you build
dashboards on specific names.

## What should I redact before logging?

The most useful fields to capture are also the most sensitive. Prompts and tool
results can carry names, emails, secrets, and customer data. Logging that
verbatim turns your trace store into a second copy of your most private data, so
redact by default and opt in deliberately.While chasing a bug it is tempting to flip on full prompt and tool-content logging "just for now" in production and forget to flip it back. Those payloads carry PII and secrets, your trace backend rarely has the same access controls or retention limits as your primary store, and you have now quietly created a long-lived breach surface. Keep verbose content to a short-retention debug environment and capture only structure (tool name, status, token counts, latency) everywhere else.

Claude Code reflects this: prompt text and tool details are redacted unless you
explicitly turn them on. The content gates are separate from the on switch, so
you choose how much detail leaves the machine.

```bash
# Off by default. Turn on only where you control the data.
export OTEL_LOG_USER_PROMPTS=1
export OTEL_LOG_TOOL_DETAILS=1
```

Sensible defaults for any system:

- Redact prompt and tool content in shared or production environments. Capture
  the structure (which tool, success or failure, token counts, latency) without
  the payload.
- Strip secrets and obvious personal data before anything is written.
- Keep full content only in a tightly scoped debug environment, with a short
  retention window.

You lose nothing for diagnosis: a trace that shows tool `lookup_order` failed
with a timeout at step three tells you the cause without storing the customer's
order. For the wider risk model around tools and data exposure, see
[MCP security basics](/guides/mcp-security-basics).

## Which metrics should I track?

Traces explain one run. Metrics tell you which runs to go look at. Track a small,
honest set rather than a wall of charts nobody reads.

| Metric | Why it matters |
| --- | --- |
| Latency, including p95 | Averages hide the slow tail users feel |
| Token usage per run | Catches prompt bloat and cache regressions early |
| Cost per run | The number that gets a system shut down if it drifts |
| Error rate | Tool failures, refusals, and timeouts as a share of runs |
| Tool failure rate | The hidden cause behind most confidently wrong answers |

Use the percentile, not the mean. If average latency is two seconds but p95 is
twelve, a real slice of users is waiting twelve seconds, and the average hid it.
Alert on a change in these, not on a fixed line, since healthy numbers drift as
traffic shifts.

## Connect the dots across steps

The payoff of a shared trace id is correlation. When something breaks, you do not
read logs in isolation. You pivot.

A worked example. Error rate ticks up. You filter traces to failed runs and
notice they all called the same tool. You open one trace and the tool span shows
a timeout, and the model, given an empty result, made up a plausible answer
instead of saying it did not know. The fix is in the tool, not the prompt, and
you found it in minutes because every step shared one id. Without the trace you
would have been re-reading the prompt, the wrong place entirely.

This is also how you tell a model problem from a plumbing problem. A bad answer
with clean tool calls and good context points at the prompt or the model. A bad
answer sitting on a failed tool call or empty retrieval points at the plumbing.
The trace makes the difference obvious.

## Have Claude wire it up

Once you know what to capture, getting it instrumented is ordinary engineering
work, the kind you can hand to Claude Code. Be specific about the fields and the
schema so the result is consistent.

```text
Add tracing to the request handler in src/agent/run.ts.

Requirements:
- start one span per request and generate a trace id
- create a child span around every tool call with name, args, result status, and latency
- record input and output token counts on the model-call span
- use the OpenTelemetry gen_ai semantic convention attribute names
- redact prompt and tool content behind a LOG_CONTENT env flag, off by default

Show me the plan before editing, then add a test that asserts a failed tool call is recorded with status error.
```

Ask for the plan first so you can check the field names and the redaction default
before any code changes. For breaking the rollout into reviewable steps, see
[break big tasks into steps](/guides/break-big-tasks-into-steps).

## Start small, then grow

You do not need a full platform on day one. The order that pays off fastest:

1. **Stamp a trace id on every run** and log the prompt, the tool calls, and the
   outcome against it. This alone makes most bad answers debuggable.
2. **Add latency and token counts per step.** Now you can see cost and the slow
   tail.
3. **Adopt the standard schema** so the data is portable and your dashboards are
   not bespoke.
4. **Add metrics and alerts** on error rate, p95 latency, and cost so problems
   find you instead of the other way around.
5. **Lock down redaction** before any of this runs near real user data.

The throughline is simple: capture the run, link it by id, and keep the sensitive
parts out by default. Do that and the next bad answer is a five-minute trace pull
instead of a guess.