magus v0.4.3 is out. See what's new
¶ View markdown source · ✎ Suggest an edit
4 min read

MGS4008: unordered same-step write

Two targets run inside one step: a composed target's ctx.needs chain. One of them declares that it writes files the other declares that it reads, and nothing in the chain says which goes first.

[MGS4008] refusing to run . ci: . coverage-badge reads "**/*.go", which . mocks-generate
  writes as "internal/mocks/gen/**/*.go", and nothing orders the two. Both run inside .
  ci's ctx.needs chain, with no needs path between them, so the reader can start before
  the writer finishes and can wedge the run waiting for a slot the writer needs. Declare
  ctx.needs(mocks-generate) in coverage-badge so it runs after the writer, or narrow
  coverage-badge's ctx.readsFiles so it no longer matches what the writer produces.

The run is refused before any target starts. Nothing has executed, nothing is half-written, and the fix is a line in a magusfile.

Why a refusal rather than an ordering

Across steps magus derives the order itself: a step whose declared writes reach another step's declared reads runs first, and the reader waits on the barrier. That machinery cannot reach inside a step. A step is one target's body, and the sequence its body runs is the sequence the body wrote down: ctx.needs and nothing else. A scheduler that reordered chain members would be rewriting the body.

So there is no order to install, and the two outcomes left are both bad:

  • The reader wins the race. It hashes and reads the tree as it stood before the writer produced anything, and records a result under a key describing files that changed a moment later. Nothing about the entry says so.
  • The reader waits. A chain member that ends up behind the writer waits while holding the concurrency slot it was admitted on. When the composers of one gate do that together they hold every slot in the pool, and the writers they are waiting for can never be admitted. That is MGS3013, and before it existed the run simply hung until the stall watchdog (MGS3012) killed it fifteen minutes later with every project lock held.

Resolve it

Order the reader after the writer. This is the fix in nearly every case, and it is one line:

export fun coverage_badge(ctx: magus\Context, args: [str]) > void {
    ctx.needs(generate);           // the writer, or whatever composes it
    ctx.readsFiles("**/*.go");
    ...
}

Naming the composer (generate) rather than each generator is usually right: the chain already sequences those, and the memo means a target reached twice still runs once.

A second ctx.needs call in the composer's own body is an ordering too. One call fans its arguments out together and returns when all of them have run, so ctx.needs(format); ctx.needs(conventions); runs conventions after everything format reached, and the check reads it that way. Put the reader in a later call when the sequencing belongs to the composer rather than to the reader.

Or narrow the reader's declaration. A reader that never meant to depend on generated output should say so, and a footprint that matches files the reader does not care about is over-declared anyway:

ctx.readsFiles("coverage/**/*.json");   // not "**/*.go"

Both fixes are real answers. The one to avoid is making the overlap invisible by deleting a declaration the target does rely on: the cache key would then omit an input the result depends on, which is MGS1028's question rather than this one.

What it looks at

Declared footprints on both sides. A target that names no ctx.readsFiles falls back to the project's source baseline, a whole-project over-approximation. An overlap through one of those is a guess, and this refuses runs, so a guess is not enough. Fallbacks are ignored on either side.

ctx.needs reachability, in either direction. Reader-after-writer is what this asks you to add. Writer-after-reader clears it too: the reader reading what was there beforehand is a sequence the author wrote down, and staleness is MGS4006's question. A later ctx.needs call in a composer counts as reaching everything an earlier call ran.

A file, not a glob. Two globs can intersect as patterns and never meet on a path (**/MAGUS.md against cmd/magus/completions/*). The refusal stands only on a file in the tree that both patterns match, and on one the reader's key would hash: a pattern read never descends into a pruned directory (gen, vendor, node_modules, and whatever the project's spells add), so a writer's file under one of those is no witness for a pattern. An exact path is.

One project. A pair whose reader and writer belong to different projects is reported as advice rather than refused: the fix is a line in one project's magusfile, and the sequencing usually belongs to the other project's chain.

Meeting it before your first gate

magus doctor runs the same predicate over every composed target in every project, as the same-step-writes check. It reads declarations only, so it needs no run and no cache:

magus doctor

A pair the check fails on is one a gate will refuse; a cross-project pair it advises on is the one a run warns about; a workspace it passes cannot be refused for this.

See also

  • MGS3013: the wedge this prevents, caught at run time when it happens for some other reason.
  • MGS3012: the stall watchdog, which is what used to end these runs.
  • MGS4006: stale generated output, the staleness question.
  • MGS1020: two targets declaring one output, the ownership violation with no ordering answer at all.
MGS4008raceorderingctx.needsschedulingdeadlockdeclarationsdoctor
Last updated (0c2c811f)
Glossary

Workspace

The magus root directory that owns a set of projects and shared config; the unit magus operates over. See workspace.

Project

A directory magus recognizes as a unit of work (it has a magusfile); the unit of caching, scheduling, and dependency tracking. See workspace.

Magusfile

The magusfile.buzz that declares a project's targets (as export funs) and binds its spells. See targets.

Target

A named operation (build, test, ...) you invoke with magus run <target>; it may compose a spell's tool-native operations and depend on other targets. See targets.

Spell

A language/runtime adapter (e.g. go, md) that maps generic targets onto a toolchain's real commands. See spells.

Module

A magus stdlib namespace a magusfile imports for host capabilities: filesystem, exec, vcs, and more. See the module reference.

Buzz

The language magusfiles are written in (the .buzz engine). See engines.

Cache

The content-addressed store magus consults before running a target, so unchanged work is skipped. See cache.

Sandbox

The restricted filesystem and environment a target runs in, so builds stay reproducible and side-effect-free. See sandbox.

CI

An ordinary magusfile-defined target you compose yourself with magus\needs - magus does not hardcode its stages. Magus.RunCI treats it specially only in that it strips the rw charm, it is the anchor magus affected ci keys off, and a selected scope with no project declaring it is a load error rather than a silent no-op. See targets.

Pool

The concurrency pool: the shared set of slots that caps how many targets run in parallel on one machine. Its capacity defaults to MAGUS_CONCURRENCY, then 4 on GitHub-hosted runners, then min(NumCPU, 8); magus status and the dashboard report it live. See server.

Slot

One unit of the pool's capacity. A target acquires the slots it needs to run (most take one) and releases them when it finishes; the pool tracks capacity (total slots), running (acquired), and queued (blocked). See server.

Concurrency

How many targets run at once. It is bounded by the pool's capacity and set with --concurrency, MAGUS_CONCURRENCY, or the concurrency config key. See server.

MAGUS.md

The committed routing index at a workspace root, regenerated from the knowledge graph: it lists every node and points at the exact query for a given question, so it is the entry point an agent reads first. See knowledge.

Ownership

An insight lens: author concentration - the primary author and their share, the distinct-author count (the bus factor), and abandonment. See insight.

Run

One target executing under one magus invocation, such as magus run test web or magus affected ci. A run keeps its captured output behind an output reference. Every magus run is a run whether or not any job asked for it; see Job for how the two relate.

Conventions

This page uses none of the site's convention markers. The full set is on the conventions page.