Observability with OpenTelemetry and .NET Aspire

Observability allows you to understand what your agent is doing at runtime — how long model calls take, how many tokens are consumed, which tools are invoked, and what log messages are produced. The Microsoft Agent Framework integrates with OpenTelemetry out of the box and emits traces, metrics, and logs that can be sent directly to the .NET Aspire Dashboard for real-time visualization.

What Does the Agent Framework Emit?

Traces (Spans)

1. invoke_agent {agent_name} — top-level span, one per agent turn
2. chat {model_name} — child span for every model call; includes token counts and (optionally) the full prompt and response text
3. execute_tool {function_name} — child span for each tool invocation

Metrics

1. gen_ai.client.operation.duration — histogram of model call durations in seconds
2. gen_ai.client.token.usage — histogram of input and output token counts
3. agent_framework.function.invocation.duration — histogram of tool execution durations

Logs

All ILogger output from the Agent Framework and your own application code is routed through the OpenTelemetry log provider and forwarded to Aspire via OTLP.

How the Aspire Integration Works

When you start the app through the AppHost project, .NET Aspire automatically injects the OTEL_EXPORTER_OTLP_ENDPOINT environment variable into every child process it manages. The OTLP exporter in your app reads this variable and sends all telemetry data to the Aspire Dashboard without any hard-coded URLs.

After starting, open http://localhost:18888 to view traces, metrics, and logs in the Aspire Dashboard.

Required NuGet Packages

The OpenTelemetry package provides the core SDK builders (Sdk.CreateTracerProviderBuilder, Sdk.CreateMeterProviderBuilder, and the AddOpenTelemetry logging extension). The OpenTelemetry.Exporter.OpenTelemetryProtocol package provides AddOtlpExporter() for all three signal types.

ObservabilitySetup.cs — OpenTelemetry Configuration

This class creates and owns the three OpenTelemetry providers (traces, metrics, logs) and builds the instrumented agent. It implements IDisposable so that all buffered telemetry is flushed before the process exits.

Key points in ObservabilitySetup

1. ResourceBuilder.CreateDefault().AddService(SourceName) — attaches a service name to every piece of telemetry so the Aspire Dashboard knows which service produced it.
2. AddSource("*Microsoft.Extensions.AI") — the wildcard prefix * matches the experimental source name Experimental.Microsoft.Extensions.AI used by the chat client pipeline. Without this prefix, model-call spans would not be captured.
3. AddOtlpExporter() (no arguments) — reads OTEL_EXPORTER_OTLP_ENDPOINT from the environment. Aspire injects this variable automatically; you never need to hard-code the URL.
4. AIAgentBuilder.UseOpenTelemetry() — wraps the inner agent in an OpenTelemetryAgent that emits invoke_agent spans and augments child chat spans with agent-specific attributes.
5. EnableSensitiveData = true — records the full prompt and response text in span attributes. Only use this in development; never in production.
6. IDisposable — the using statement in Program.cs guarantees that buffered telemetry is flushed before the process exits, so no data is lost.

Demo1_BasicObservability.cs — Sending a Prompt and Capturing Telemetry

Key points in Demo1

1. setup.LoggerFactory.CreateLogger<T>() — creates a logger whose output is routed through the OTel log provider set up in ObservabilitySetup, so every LogInformation call appears as a structured log record in the Aspire Dashboard.
2. agent.RunAsync(prompt, session) — no extra options are needed; the OpenTelemetryAgent wrapper intercepts the call and emits spans and metrics automatically.
3. response.Usage?.OutputTokenCount — the token usage is available on the response and also captured automatically in the gen_ai.client.token.usage metric.

Program.cs — Entry Point

The using var declaration is critical. When the program finishes, the providers are disposed in order, which triggers a final flush of any buffered spans and metrics to the OTLP endpoint.

What to See in the Aspire Dashboard

Traces tab

After running the demo, the Traces tab shows one trace with two spans:

1. invoke_agent ObservabilityDemoAgent — duration of the entire agent turn
2. chat gpt-4o (child) — duration of the model API call, plus attributes:
3. gen_ai.usage.input_tokens
4. gen_ai.usage.output_tokens
5. gen_ai.request.model
6. Full prompt and response text (because EnableSensitiveData = true)

Metrics tab

1. gen_ai.client.operation.duration — histogram showing how long the model call took
2. gen_ai.client.token.usage — histogram showing token counts (broken down by input/output)

Logs tab

Structured log records from the demo appear here, including the Sending prompt to agent and Agent response received messages logged via ILogger.

OpenTelemetry Source Names to Register

The * prefix is required because the SDK emits these spans under an experimental source name prefixed with Experimental.. The wildcard ensures both the stable and experimental source names are matched.

Best Practices

1. Never enable sensitive data in production. Set otelAgent.EnableSensitiveData = true only in development or test environments. Sensitive data includes the full prompt and response text.
2. Always dispose providers before exit. Use using var setup = new ObservabilitySetup(); in Program.cs to guarantee the final OTLP flush.
3. Use AddOtlpExporter() without arguments. This reads OTEL_EXPORTER_OTLP_ENDPOINT from the environment, which Aspire injects automatically. Never hard-code the URL.
4. Start from the AppHost, not from the ConsoleApp directly. Only the AppHost injects the OTLP endpoint environment variable. Running the ConsoleApp directly will produce output to the console but no telemetry will reach the Aspire Dashboard.