Skip to main content

Agents in CI pipelines

A coding agent that runs unattended in a build (Azure DevOps, GitHub Actions, …) starts a new session on a fresh machine for every run and has no user identity that survives between runs. Give it a name through a resource attribute and all its runs roll up under one actor, tagged as an agent:

# Azure DevOps step running Claude Code headless
- script: claude -p "Review this pull request"
  env:
    CLAUDE_CODE_ENABLE_TELEMETRY: "1"
    OTEL_LOGS_EXPORTER: otlp
    OTEL_METRICS_EXPORTER: otlp
    OTEL_EXPORTER_OTLP_PROTOCOL: http/json
    OTEL_EXPORTER_OTLP_ENDPOINT: https://costpilot.example.com
    OTEL_EXPORTER_OTLP_HEADERS: Authorization=Bearer $(COSTPILOT_INGEST_TOKEN)
    OTEL_RESOURCE_ATTRIBUTES: costpilot.agent=pr-reviewer,pipeline=$(Build.DefinitionName),build_id=$(Build.BuildId)
  • costpilot.agent is the actor name. Use one name per job the agent does (pr-reviewer, test-fixer), not per run. It wins over any user the pipeline's account reports. Usage without it counts as human when it carries a user, and as "No actor" otherwise (e.g. product traffic).
  • Every other attribute (pipeline, build_id, …) is kept as a custom dimension, e.g. to group the runs of one build, except the reserved namespaces (service., telemetry., os., host., process., user., session., costpilot. and the like). Values must not contain spaces or commas unless percent-encoded.
  • The prompt hook does not run in a pipeline; to attribute a run to an issue, add issue_id=<key> to the same attribute list, and repo=<owner/name> so the run lands in the project that lists the repository. A resource attribute issue_id always wins over hook reports.
  • A pipeline run counts as one turn.

In the dashboard the Actor tile lists humans and agents side by side; the Actor type dimension separates them. An agent's drilldown shows runs and cost per run.