magus v0.4.2 is out. See what's new
¶ View markdown source · ✎ Suggest an edit
8 min read

Telemetry (OpenTelemetry)

magus can export metrics and traces to any OTLP collector you run. Telemetry is OFF by default: there is no magus-operated backend. The collector is yours, and magus connects only to the endpoint you configure.

Instrument definitions live in internal/observability/otlp/provider.go and internal/observability/otlp/families.go; the interface every producer records through is internal/observability/provider.go, and config is in internal/config/config.go.

Enabling

In magus.yaml:

telemetry:
  enabled: true
  endpoint: "localhost:4318" # host:port, no scheme
  protocol: "http" # "grpc" (default) or "http"
  insecure: true # plaintext for local collectors
  service_name: "magus" # resource attribute service.name
  sample_ratio: 1.0 # head-based trace sampling, [0,1]
  headers: # static headers on every OTLP request
    x-api-key: "..."

Or via the environment:

Env var YAML key Default Purpose
MAGUS_TELEMETRY_ENABLED telemetry.enabled false Turn OTLP export on; magus connects to the endpoint when true
MAGUS_TELEMETRY_ENDPOINT telemetry.endpoint - OTLP collector address as host:port (no scheme); required
MAGUS_TELEMETRY_PROTOCOL telemetry.protocol grpc OTLP wire protocol: grpc or http
MAGUS_TELEMETRY_INSECURE telemetry.insecure false Disable TLS for the OTLP exporter (plaintext local collectors)
MAGUS_TELEMETRY_SERVICE_NAME telemetry.service_name magus Value of the resource attribute service.name
MAGUS_TELEMETRY_SAMPLE_RATIO telemetry.sample_ratio 1.0 Head-based trace sampling ratio in [0,1]

Export details. magus pushes metrics over OTLP on a periodic reader every 30s. It batches traces and samples them head-based by sample_ratio (1.0 = every trace). Metrics are unsampled: every recorded point is exported.

Resource attributes

Every metric and span carries these resource attributes:

Attribute Source
service.name telemetry.service_name (default magus)
service.version the magus build version
magus.workspace.root the workspace root, when set
process.* detected process metadata (pid, runtime, ...)
host.* detected host metadata

Metrics

Cache (local)

Low-cardinality aggregate view of the on-disk content-addressed cache; no per-project attribute.

Metric Instrument Unit Attributes Meaning
magus.cache.hits counter {call} outcome=hit Cache replays (a Cache.Run served from disk)
magus.cache.misses counter {call} outcome=miss Genuine builds (no entry found)
magus.cache.errors counter {call} outcome=error Build steps that failed
magus.cache.duration histogram s outcome ∈ {hit, miss} Wall-clock of a single Cache.Run
magus.cache.saved.duration histogram s - What a hit avoided: the duration the entry recorded when it was written

magus.cache.duration says what a hit cost; magus.cache.saved.duration says what it saved, one observation per hit. Its sum is the wall-clock the cache has given back, and its distribution says which hits are worth having. An entry written before the manifest carried a duration reports nothing rather than zero, so the total understates and never overstates - the same measured-not-modeled rule the cache_saved_ms field in magus status follows.

Remote cache

The shared backend (S3, GitHub Actions, ...), exported only when one is wired via magus\cache.remote(...). These mirror the local-cache vocabulary under a .remote prefix so a remote hit is never folded into the local counters: a remote hit is still a local miss, because the remote fetch only runs after the local store misses. Instrumentation wraps the backend interface, so it applies to every backend (including ones you write) with no backend changes.

Metric Instrument Unit Attributes Meaning
magus.cache.remote.hits counter {call} op=get, outcome=hit Remote get returned an entry
magus.cache.remote.misses counter {call} op=get, outcome=miss Remote get found nothing
magus.cache.remote.errors counter {call} op ∈ {get, put}, outcome=error A remote operation failed
magus.cache.remote.duration histogram s op ∈ {get, put}, outcome Wall-clock of a single remote operation
magus.cache.remote.io.size histogram By op ∈ {get, put} Bytes transferred, recorded on hits and puts only (egress/ingress cost)

outcome ∈ {hit, miss, stored, error}; stored is a successful put. Remote hit-rate is hits / (hits + misses). Put success is the magus.cache.remote.duration count for op=put minus magus.cache.remote.errors for op=put.

Graph

Metric Instrument Unit Attributes Meaning
magus.graph.queries counter {call} op, strategy Number of graph query operations
magus.graph.query.duration histogram s op, strategy Wall-clock of a single graph query

strategy is present only when the query reports one. op=build is the full graph build.

Target runs

Per-project, per-spell. magus.project has unbounded cardinality; see Cardinality.

Metric Instrument Unit Attributes Meaning
magus.target.runs counter {call} magus.project, magus.spell, magus.target, outcome, cache.hit Target executions, including cache replays
magus.target.duration histogram s same Wall-clock of a single target execution

outcome ∈ {success, error} · cache.hit ∈ {true, false} · one row is emitted per resolved spell per project.

Concurrency pool

Metric Instrument Unit Attributes Meaning
magus.pool.wait.duration histogram s - Time a target waited for a slot
magus.pool.slots.running up-down counter {slot} - Concurrency slots currently running (gauge-like)
magus.pool.slots.queued up-down counter {slot} - Callers currently queued for a slot (gauge-like)

magus.pool.slots.running is an up-down counter: it rises as targets acquire slots and falls as they release, so its value reads as the live running depth.

Agent surface

Leases, attention requests and paired review: the three places a fleet of agents and the people working with them meet. Every producer here runs in the daemon, which is what makes them collectable at all - a magus CLI invocation is a one-shot process, and the CLI halves of these same surfaces (raising an attention request, the agent guard grading a write) reach the activity trail instead. Read each row for what it counts, not for the whole surface.

Metric Instrument Unit Attributes Meaning
magus.lease.registrations counter {registration} verdict A worker registered the base it actually landed on
magus.attention.disposition.duration histogram s severity How long a request waited, from raised to disposed
magus.review.remarks counter {remark} author ∈ {human, agent} A remark drafted on a change
magus.review.publishes counter {publish} verdict, downgraded ∈ {true, false} A review published, by the verdict that landed

verdict on registrations is match, revision-match, diverged or unknown; a rising diverged share says a fleet's workers are building on trees their orchestrator never handed them. verdict on publishes is comment, approve or request_changes, and it is the verdict that landed: downgraded=true means the asserting verdict the person asked for was refused (a review cannot approve a change its own credential opened), which is invisible in the forge's record and known only here.

magus.attention.disposition.duration counts the disposals made through the console route. magus session dispose closes a request from a one-shot process with no collector, so those are absent - the instrument is a wait-time distribution rather than a queue depth, which would read as the whole queue and be neither. severity is re-read from the store and clamped to the declared tiers (info, notice, warning, critical, plus unset), so a record written by another build can never mint a series.

Metrics observe, they never gate. magus.review.remarks is attributed by author because that says which door a remark came through, and both doors write to one store. magus.review.publishes is deliberately not: how a change was judged, split by who wrote it, is the first half of a threshold that blocks work by author kind, and magus does not build the instrument that invites one.

Traces (spans)

Spans are sampled head-based by telemetry.sample_ratio.

Span Attributes When
magus.target.run magus.project, magus.target One per target execution (miss path)
magus.cache.hash - Hashing a target's inputs (every Cache.Run)
magus.cache.replay - Restoring outputs from a cache hit
magus.cache.snapshot - Capturing outputs after a build (miss path, writable cache)
magus.cache.remote.get magus.project A remote fetch; spans the network round-trip and the import
magus.cache.remote.put magus.project A remote upload
magus.cache.remote.prune - A retention sweep (magus config cache prune --remote)

The magus.cache.hash / replay / snapshot spans break a target's latency down by phase: hashing vs. building vs. I/O. The remote get/put spans put a network fetch or upload inline in the build trace with its own latency, so a slow remote round-trip is visible instead of opaque time inside the target.

Cardinality

magus.target.runs and magus.target.duration include magus.project, which is unbounded in large monorepos. If your setup has thousands of projects, drop or relabel the attribute at the collector:

# OpenTelemetry Collector - attributes processor
processors:
  attributes/drop_project:
    actions:
      - key: magus.project
        action: delete

Alternatively, use an SDK View at startup to drop the attribute before it leaves the process. Three other instruments carry a workspace-sized attribute and the same recipe applies: magus.sandbox.checks and magus.sandbox.env.dropped carry magus.project, and magus.buzz.spell.resolve.duration and the magus.buzz.spell.builtins.* family carry spell, a workspace-authored name. Every other metric omits them. The remote get/put spans carry magus.project, but spans are sampled and not aggregated into time series, so they don't create cardinality the way a metric attribute would.

telemetryopentelemetryotlpobservabilitymetricstraceslogsmonitoring
Last updated (4f8cc295)
Earlier changes on this page (3)

Full history ↗ · Blame source ↗

Glossary

Workspace

The magus root directory that owns a set of projects and shared config; the unit magus operates over. See workspace.

Project

A directory magus recognizes as a unit of work (it has a magusfile); the unit of caching, scheduling, and dependency tracking. See workspace.

Target

A named operation (build, test, ...) you invoke with magus run <target>; it may compose a spell's tool-native operations and depend on other targets. See targets.

Op

A single tool-native command a target composes (long form: operation); the middle of the work hierarchy (Spell to Op to Target). See operations.

Spell

A language/runtime adapter (e.g. go, md) that maps generic targets onto a toolchain's real commands. See spells.

Buzz

The language magusfiles are written in (the .buzz engine). See engines.

Cache

The content-addressed store magus consults before running a target, so unchanged work is skipped. See cache.

Sandbox

The restricted filesystem and environment a target runs in, so builds stay reproducible and side-effect-free. See sandbox.

Service

A long-running or shared process magus manages across runs, distinct from a one-shot target. See services.

Daemon

The background magus host that owns shared state such as services and the warm knowledge graph. See daemon.

CI

An ordinary magusfile-defined target you compose yourself with magus\needs - magus does not hardcode its stages. Magus.RunCI treats it specially only in that it strips the rw charm, it is the anchor magus affected ci keys off, and a selected scope with no project declaring it is a load error rather than a silent no-op. See targets.

Trace

OpenTelemetry's name for one whole magus invocation; every target it runs is a span beneath it. See telemetry.

Span

OpenTelemetry's name for one unit of work under a trace - a target execution, whose sub-operations are child spans. An output reference points at a span's captured output. See telemetry.

Pool

The concurrency pool: the shared set of slots that caps how many targets run in parallel on one machine. Its capacity defaults to MAGUS_CONCURRENCY, then 4 on GitHub-hosted runners, then min(NumCPU, 8); magus status and the dashboard report it live. See daemon.

Slot

One unit of the pool's capacity. A target acquires the slots it needs to run (most take one) and releases them when it finishes; the pool tracks capacity (total slots), running (acquired), and queued (blocked). See daemon.

Concurrency

How many targets run at once. It is bounded by the pool's capacity and set with --concurrency, MAGUS_CONCURRENCY, or the concurrency config key. See daemon.

Queued

A target that wants a slot while the pool is full; it blocks first-in-first-out until a slot frees. The dashboard colors a sample with queued > 0 accordingly. See daemon.

Remote cache

A CI-only backend that shares content-addressed artifacts across runners: a cold machine replays a build another runner already did instead of rebuilding. Every remote artifact must be signed by a trusted key. See remote.

Snapshot

A point-in-time view of live state - the pool's occupancy or a tick of exported metrics - as opposed to accumulated history. See daemon.

Latency

How long an operation takes. magus records latency as OpenTelemetry histograms per family - target execution, cache op, pool wait, and graph query - and reports each as a count, sum, and percentiles. See telemetry.

Session

One magus process's recorded facts - the targets it finished, their outcomes, and the lease it acted as - kept in a repo-scoped store every worktree shares. magus session lists them; the store prunes itself by last-fact age.

Attention request

A durable "an agent is blocked" record, opened when a magus session notify event carries the waiting or permission outcome and held until a person disposes it. magus session attention lists what is open. Nothing closes one on its own - see doctrine.

Dispose

The human act of closing an attention request: a judgment rendered, recorded with who and why. Distinct from resolving a review thread or a merge conflict - a disposition answers a request; it does not merge anything.

Lease

One row of the lease ledger: a piece of work an orchestrating agent handed out, with its goal, the checkpoint it was cut against, and the paths it owns or must not touch. The ledger records; the agent guard is what reads those facts back when grading a write. See doctrine.

Conventions

This page uses none of the site's convention markers. The full set is on the conventions page.