magus v0.4.3 is out. See what's new
¶ View markdown source · ✎ Suggest an edit
4 min read

MGS1030: memory declaration disagrees with what was measured

magus doctor compared each target's declared memory_mb against the peak resident memory magus recorded the last several times it ran, and they disagree.

advice  memory declarations
        2 target(s) declare memory that disagrees with what magus measured;
        admission is only as good as the declarations it arbitrates
        - console test declares no memory_mb anywhere in what it runs and its passing
          runs reached from 2900MB to 5312MB; a claim measured from them carries a
          quarter's headroom, which that spread outgrows
        - docs site-generate declares 2048MB and reached at least 6100MB; admission
          seats it against a figure it exceeds

Why

memory_mb is not decoration, and it does two jobs. Inside a run it converts to concurrency slots on the limiter, so a target declaring a large figure holds several slots and throttles the peers magus would otherwise start alongside it. A 16GB runner and a 64GB workstation get different parallelism from the same declaration, without the magusfile naming either machine. Across runs it is the figure the broker arbitrates machine-wide, so a target that says what it needs queues behind the peers in other worktrees instead of racing them (MGS3009).

That makes both exactly as good as the declarations, and declarations rot quietly. A suite written at 2GB grows over a year into one that reaches 9GB, and nothing says so: the target keeps passing, it keeps taking one slot's worth of room, and the protection is gone long before anyone notices.

magus already measures the resident memory of every process a target runs, for every run, so the disagreement is a fact it holds rather than a question the author has to think to ask.

Two shapes are reported:

  • A target is under-declared when the measurement is more than a quarter above the declaration, by at least 512MB. This is the dangerous direction.
  • A target is unstable and undeclared when it has no memory_mb at all, a measurement of at least 2GB, and passing runs whose peaks spread more than a quarter apart. An undeclared target claims 1.25 times its highest passing peak once three runs measured it, so a stable one needs no declaration. A spread wider than that headroom means the next run can outgrow a claim sized from the last ones, and only a declaration of the worst case covers it.

Below those thresholds nothing is reported: the run-to-run variation in measuring a small target is larger than the disagreement would be.

What the measurement is, and is not

The recorded figure is a floor, not a true peak. magus folds a target's processes as a maximum, and the kernel does the same upstream: wait4 propagates a subtree's peak by taking the largest process, never the sum. A driver that forks four concurrent 800MB children reports 800MB. So a suite that fans out, which go test does by default, records roughly its largest single binary rather than what its tree held together.

Both findings above survive that, because a floor only argues in the safe direction: a declaration a floor already exceeds is exceeded by more, and peaks that spread wide as floors spread at least as wide underneath.

The symmetric finding does not survive it, which is why magus does not report an over-declared target. "You declared 10GB and never used more than 3.5GB" needs a real peak to be true, and a figure that undercounts a parallel suite would argue for lowering exactly the declarations written from a machine that died. If a declaration looks generous, measure the target's whole process tree before trusting the gap.

Resolution

  1. Declare the worst case, at least the highest figure magus measured, rounded up to leave headroom for a heavier input than the runs on record:

    import "magus";
    
    magus\project({
        "target_policies": {
            "test": {"memory_mb": 10240},
        },
    });
    
  2. Leave it alone deliberately. A ceiling above the measurement is a legitimate thing to declare when a target's memory varies with its input, or when the figure came from watching a real machine die rather than from a recorded run. Nothing nags you about the gap: this check reports only under-declaration and unstable non-declaration, and it is advice rather than a failure so the decision stays yours.

  3. Reduce what the target needs, if the measurement is the surprise rather than the declaration. A suite whose peak is dominated by the race detector's shadow memory across concurrent test binaries can often be sharded or run with lower in-tool parallelism.

See also

  • memory_mb: declaring what a target needs.
  • MGS3009: the machine-wide budget these declarations are arbitrated against.
MGS1030memory_mbdoctormemorydeclarationsadmission
Last updated (29d8b3de)
Earlier changes on this page (5)

Full history ↗ · Blame source ↗

Glossary

Project

A directory magus recognizes as a unit of work (it has a magusfile); the unit of caching, scheduling, and dependency tracking. Project in full.

Magusfile

The magusfile.buzz that declares a project's targets (as export funs) and binds its spells. Magusfile in full.

Target

A named operation (build, test, ...) you invoke with magus run <target>; it may compose a spell's tool-native operations and depend on other targets. Target in full.

Op

A single tool-native command a target composes (long form: operation); the middle of the work hierarchy (Spell to Op to Target). Op in full.

Buzz

The language magusfiles are written in (the .buzz engine). Buzz in full.

Sandbox

The restricted filesystem and environment a target runs in, so builds stay reproducible and side-effect-free. Sandbox in full.

Broker

The per-user background process that holds this host's capacity: the machine budget every run claims slots from, and the shared services runs keep warm. A run starts it on demand; broker: off in magus.yaml runs without one. Broker in full.

CI

An ordinary target you compose yourself with ctx.needs; magus does not hardcode its stages. It is the anchor magus affected ci keys off, and ci strips the rw charm. CI in full.

Slot

One unit of the pool's capacity. A target acquires the slots it needs to run (most take one) and releases them when it finishes; the pool tracks capacity (total slots), running (acquired), and queued (blocked). Slot in full.

Concurrency

How many targets run at once. It is bounded by the pool's capacity and set with --concurrency, MAGUS_CONCURRENCY, or the concurrency config key. Concurrency in full.

Tool

An action a model can call through its host: a name, a description, a typed input and a result. magus serves its tools over MCP, each a read of the workspace model or a run of the CLI it already has. Tool in full.

Job

The unit of delegated work, and one row of the job store: what an orchestrating agent handed out, with its goal, the checkpoint it was cut against, the paths it may write or must not touch, and the one check it runs. A job's holder is either a session, for work an orchestrator handed out, or the daemon, for its own maintenance. The store records; the agent guard is what reads those facts back when grading a write. Job in full.

Run

One target executing under one magus invocation, such as magus run test web or magus affected ci. A run keeps its captured output behind an output reference. Every magus run is a run whether or not any job asked for it; see Job for how the two relate. Run in full.

Conventions

This page uses none of the site's convention markers. The full set is on the conventions page.