Cache model

magus's build cache is content-addressed: a target's outputs are keyed by the SHA-256 of its inputs, so an unchanged target replays its previous outputs instead of rerunning. This page is the local model: what magus hashes, what invalidates a key, what "replay" restores, and where it all lives on disk. The remote cache shares these same artifacts across machines and layers a signed trust model on top; this page is the substrate it references, so we describe it once here and link there for the distributed story.

Design intent

• Correctness is a declaration contract. magus caches what a target declares, not what it touches. A target's needs and provides (see below) define its whole cache footprint. Under-declare an input and a stale hit slips through; over-declare an output and every replay snapshots more than necessary. The cache is only as correct as those declarations, which is why the vocabulary is explicit rather than inferred from a traced filesystem.
• Identical inputs replay. The key is a pure function of the inputs. Two runs with byte-identical sources, tool versions, charms, and dependency keys produce the same key and so the same hit. Nothing about wall-clock time, machine, or run order enters the key.
• A hit never runs the body. On a hit magus restores the recorded outputs and emits the result event; the target's export fun never executes. The saved work is the point.
• It is just files. The store is a directory of blobs, JSON manifests, and captured logs under .magus/. There is no database and no server in the read path. You can ls it, cat a manifest, and reason about a hit or miss with ordinary tools.

needs and provides: a target's cache footprint

A bound spell contributes two glob sets to its project. Only operations are runnable; these two are metadata that make caching correct (see What a spell provides). Binding a spell contributes its needs/provides to a project's cache key even before you wire a target.

Internally these map to a Step the cache hashes and replays: needs become Step.Sources, provides become Step.Outputs. Two rules follow directly:

• Declare every input in needs. A source file that isn't matched by a needs glob doesn't enter the key, so editing it produces no miss and you replay a stale build.
• Keep provides tight and complete. Under-declare and the cache can't replay an output it never recorded; over-declare and every hit restores files that were never outputs.

The output tree is never treated as an input: source expansion excludes the provides globs and prunes their static directory prefixes, so a generated file can't feed back into its own key.

Per-target inputs and outputs

A spell contributes its globs to every target on the project. To attach a glob to one target, declare it in that target's body with ctx.readsFiles(...) / ctx.writesFiles(...):

export fun build(ctx: magus\Context, args: [str]) > void {
    ctx.readsFiles("schema/**", "codegen.config.json");
    ctx.writesFiles("dist/**");
    go["go-build"](ctx);
}

An explicit ctx.readsFiles(...) declaration defines that target's source footprint; an explicit ctx.writesFiles(...) declaration defines its snapshot/replay footprint. Magus retains the magusfiles and any spell sources specific to that target, but it does not inherit the broad project baseline. This is what lets one target be precise without making its siblings under-declared (see Granularity).

Files a target edits rather than produces

The declaration names encode ownership, not merely direction:

That last case is for a hand-written page with a generated region between markers, or a manifest a tool rewrites in place. It is deliberately not an output Magus owns.

export fun content_generate(ctx: magus\Context, args: [str]) > void {
    ctx.writesFiles("reference/buzz/*.md");          // created and fully owned
    ctx.modifiesExistingFiles("concepts/spells.md"); // existing page; only the table changes
}

ctx.modifiesExistingFiles is never deleted by magus clean and never replayed from a snapshot, because the bytes magus produced are only part of the file. It still folds into the target's cache key exactly as an input does - so editing the prose around a generated region invalidates the target that maintains that region, which declaring the file as an output could not do (an output is excluded from its own source hash).

Unlike reads and writes, a modification infers no ordering edge in either direction: "I edit one region of a file someone else authored" says nothing about build order. Declare ctx.needs if you need it.

The globs are read statically, before the target runs - a cache hit skips the body, so the run can't be the source of truth. magus recovers them from the source: it walks each target body and the helpers it calls by name, collecting the string-literal globs. Two disciplines follow, both enforced:

• A non-literal argument (ctx.readsFiles(someVar)) is a magusfile load error - a computed glob is invisible to the static read, and silently dropping it would risk a stale hit.
• A call the walk can't reach (in an unreferenced helper, or the identifier used as a value) never enters a key; magus doctor flags it as MGS1004.

This shares the literal-first discipline of magus\needs: declare the footprint at the target, in literals magus can see.

The cache key

The key is the hex SHA-256 of a deterministic, newline-delimited serialization of the Step. magus writes these lines, in this order, into one hash:

• keyVersion - an internal schema version. Bumping it (when the set of hashed fields changes) forces a global rebuild.

• os and arch - the host platform, each independently switchable with cache.include.os.enabled and cache.include.arch.enabled. Both default to off, so a macOS laptop and a Linux runner mint the same key for identical sources, and an output ref names the same run on both. Neither switch carries correctness: every entry records the platform it was built on, and a replay onto a different one is refused as a miss whatever these say. Turn them on for a cache shared across platforms, where one key per platform beats every platform colliding on one key and taking that miss.

• projectPath and target - so the same sources under different targets key separately.

• spell - the explicit spell::op filter, written only on such runs. An explicit op bypasses a magusfile export that shadows the same name, so the two forms run different definitions under one target name and must not share an entry: without this line a compile-only go::go-build recorded a pass that the real go-build target then replayed. Plain target runs hash without it.

• charm: lines - the active charms, sorted by name. A charm-variant run (lint:rw) hashes differently from the bare run, because the charm changes behavior. Empty charms add nothing, so charm-less runs are unaffected.

• arg: lines - one per argument after -- (magus run test -- -run TestFoo), in the order given. Unlike charms and env these are never sorted, since -run X is not X -run; a run with different trailing args must not replay another run's result. Empty when no args are forwarded, so an ordinary run hashes unaffected.

• src: lines - for every file matched by needs, its workspace-relative path, its content SHA-256, and its executable bit. Files are discovered by a single walk, sorted by path, and hashed in parallel. Only the executable bit of the mode is folded in (not the full permissions, which would differ across machines with different umasks), so chmod +x on a script - which changes no content - still invalidates the key.

  Magusfiles are always in this set, whether or not you declare them. magus appends the project's own magusfile and the workspace root's to every step's sources. You never need to list magusfile.buzz in a project's sources, and listing it changes nothing.

  This is load-bearing rather than a convenience. A target's BODY is not hashed - only its NAME goes into the key, as target: - so if the magusfile were not a source, editing what a target actually does would leave the key unmoved and replay the previous result. Everything a target does that magus can see comes in through the files it reads; the file that DEFINES it has to be one of them.

• env: lines - each allow-listed environment variable name and its value, sorted, distinguishing unset from set-to-empty. A variable's value contributes to the key only if the spell opted it in.

• obs: lines - each observation of a fact outside the tree: a ctx.observes(key, value) declaration as key=value, and the output of any observation probe a spell declares for a tool the target's ops drive, sorted. Its own class rather than a fold into env: or exec:, because an observation names a fact outside the tree entirely and changes nothing about how the target runs. A target declaring none writes no line, so an ordinary run hashes unaffected.

• exec: lines - per-op ctx.withEnv/ctx.withCwd execution overrides, sorted. Unlike env: lines, which read a variable's live process value at hash time, an override's value is fixed in the magusfile source itself, so it hashes directly - two runs differing only by a derived override must not share an entry.

• dep: lines - the resolved cache keys of upstream dependencies, sorted. This is how a change ripples: a dependency's new key becomes an input line here, so a dependent misses transitively.

• spellDefVersion - a binary fingerprint of the spell definition, so a magus upgrade that changes a spell forces a miss.

• tool: lines - spell:version strings, sorted, so a toolchain upgrade (a new go or prettier) invalidates the key even when no source changed.

Because the serialization is stable and sorted, the key is reproducible: identical inputs anywhere yield the identical key. A src file's content hash uses an mtime + size fast path (a per-file memo persisted under the cache dir), so an unchanged tree re-keys without re-reading every byte; the memo is a performance cache for the hash, never a substitute for it.

The whole fold at a glance - every row is an input, each merged cell consumes everything beside it, and the read ends at one verdict. If a change is not on this list, it cannot move the key:

Invalidation: what busts a key

A miss is "no manifest stored under this key." Anything that changes a hashed line above yields a new key, and thus a new (empty) slot:

• editing, adding, or removing a file matched by needs;
• toggling the executable bit on a needed file;
• changing the value of an allow-listed env var (or setting/unsetting it);
• bumping a declared observation's value (ctx.observes), or a probed one moving (a scanner database refreshing);
• an upstream dependency's key changing (transitive invalidation);
• a spell definition change (spellDefVersion) or a tool-version bump;
• applying or dropping a charm;
• renaming the project or target.

What does not invalidate: a file's mtime alone (content is what's hashed), or anything outside the declared needs.

Old keys are never mutated - a miss writes a new entry beside the old one - so invalidation is additive. Reverting a change restores the earlier key and replays its still-present entry. Disk is reclaimed separately by eviction and pruning (see On disk).

Anti-pattern: a shared manifest as an input

The commonest way to wreck a cache is to reach for the file that pins your tools - mise.toml, package.json, go.mod, a lockfile - and declare it an input, usually project-wide.

The intent is right: a tool version really is part of what produced the output, and a bump really should invalidate. The result is not. A manifest pins many tools, and it moves for reasons unrelated to most of them. Wire it into every project and one linter bump rebuilds the entire graph - in CI, the difference between an affected run and a from-scratch build, for a change that could not have altered almost any of it. Do that a few times and people stop trusting the affected set, which is the actual loss: a cache nobody believes is worse than no cache.

Declare what changed, not what contains it. The blast radius should match the tools a project genuinely uses:

• a tool with a version probe (mgs_getVersionProbe, or mgs_getVersionProbes for a spell driving several binaries) contributes spell:tool:version to the key of every project binding that spell, and nothing to any other. Bumping hadolint moves projects using the docker spell; a Go project's key never notices. This is almost always the right answer for an external binary.
• a manifest that is genuinely a source of one project - go.mod for a Go project whose build reads it - belongs in that project's sources, where it already is. That is not this anti-pattern: the file really does feed those targets.
• a pin that reaches one target only wants a per-target declaration, not a project-wide one. magus\inputs in that target's body keeps a sibling target's key still.

If you find yourself adding a manifest to sources to fix a staleness bug, the question to ask first is which tool went stale, and whether it can be probed instead. A probe invalidates the projects that use the tool. A manifest invalidates everyone who happens to live near it.

The opposite failure: tools outside the key entirely

Over-invalidating is loud and annoying. Under-invalidating is quiet and much worse, and it is the more common default: most build caches key on file contents and nothing else, so the tools themselves are invisible.

The failure does not look like a cache bug. A linter upgrades, and suddenly code that passed yesterday fails - or worse, code that should fail passes, because the verdict was replayed from an entry the old linter wrote. A formatter upgrades and a "clean" tree starts failing a drift gate on a file nobody touched. A codegen plugin upgrades and the committed output no longer matches what the generator would emit, but the generate step is a cache hit, so it never runs to notice.

What makes it expensive is that every one of those looks like a bug in your change. You bisect, you re-run, you blame the flaky test, you diff the branch - and the answer was never in the repository at all. Someone's toolchain moved.

magus keys on the tool versions for this reason. Each spell declares how to ask:

export fun mgs_getVersionProbe() > [str] { return ["go", "version"]; }

// A spell driving more than one binary declares each, so all of them move the key.
export fun mgs_getVersionProbes() > {str: [str]} {
    return {"golangci-lint": ["golangci-lint", "--version"]};
}

Their output lands in the key as spell:version and spell:tool:version, so a tool that upgrades invalidates exactly the projects that bind that spell - the precise middle between the two failures. magus describe spells reports which spells probe.

A tool pinned by a manifest the project already reads needs no probe: go.mod is a source of the go spell, so bumping a go tool pin invalidates on its own. Probes are for binaries that live outside the project's declared inputs - a linter from PATH, a formatter from a version manager - which is precisely the set nothing else would catch.

Where the declaration lives is the whole point. Most build systems can express this - Nx, for instance, makes tool-version tracking technically possible through executors, but shifts the burden entirely to developers to implement it consistently. Correctness then depends on each of them remembering, in every project, in every repository. One person skips it and that project silently caches across toolchains, and the failure surfaces somewhere else entirely.

magus does not infer this. Nothing sniffs your PATH or guesses which binaries a target touched - the probe is a declaration someone wrote by hand, and mgs_getVersionProbe is as explicit as it looks. What differs is its location: it sits on the SPELL, the adapter that already knows it drives golangci-lint, rather than being restated by every project that uses one. A project binding spells: [go] inherits that declaration the same way it inherits the spell's sources and ops.

So the trade is not magic against discipline. It is declaring a fact once, where it is true, instead of once per consumer - the same reason a spell declares its needs globs rather than each project re-listing **/*.go.

Set MAGUS_CACHE_TOOL_VERSION=off to drop probes from keys, or =workspace to probe once per workspace instead of per project.

Facts outside the tree: ctx.observes

A toolchain is not the only thing that lives outside the key. A vulnerability scan is keyed on the image and the tree, but its ANSWER also depends on the scanner's vulnerability database, which moves daily and belongs to no spell. A cache hit would report yesterday's CVEs against today's image.

The blunt fix is skip_cache, and it is a bad trade: it forfeits caching forever to avoid staleness that only matters when the fact actually moved. ctx.observes makes the invisible input visible instead, so caching becomes correct rather than forbidden:

export fun scan(ctx: magus\Context, args: [str]) > void {
    ctx.observes("trivy-db", "2026-08-15");
    trivy.scan(ctx);
}

The value joins the key as its own obs: line. A value that moves is a miss; a value that holds still replays. magus never interprets it - it is a stamp to compare, so a version string, a digest, and a date are all equally good, and anything that changes when the fact changes will do. describe target --cache reports an obs class beside src and env, so a rerun names the external fact instead of blaming a file.

Name the key for the fact, not for the target that reads it. The key is a label someone else meets in a rerun explanation, so scope it to whatever owns the fact - trivy-db, npm-advisories, schema-rev - and keep it identical everywhere that fact is observed. Two targets watching one feed writing one key is what makes an obs: line legible when two machines disagree.

Repeats accumulate; they never replace. Every call in a body lands in the key. One key declared twice with the same value collapses to a single line; declared twice with two different values, both survive, so the key moves when either does. That is the same rule ctx.withEnv already follows, and it is the safe direction: keeping both over-invalidates, where picking a winner would drop a fact the target really does depend on.

It is observation, not verification. The value is a cheap thing the magusfile already knows, stated where the target is declared. Real work - fetching the feed, scanning the image - belongs in the target BODY, where its cost is paid on a miss and skipped on a hit. That is also why both arguments must be literals: a target's key is computed before its body runs, so a value computed in the body could only reach the NEXT run's key, which is the staleness this exists to prevent. A computed argument is rejected at load rather than quietly accepted.

An observation is a claim someone maintains. Both arguments are literals in the magusfile, and the magusfile is already a source of every target it declares (see the cache key), so editing the value was going to move the key either way. What the declaration adds is the NAME of the cause: a src: line says the magusfile changed, an obs: line says which fact changed and to what. Declare one for a fact a person deliberately bumps, where writing it down is the point.

For a fact that moves on its own - a vulnerability database that refreshes every few hours, a feed that publishes whenever it likes - a literal is a stamp nobody remembers to update, and a hit would then claim an observation that had already stopped holding.

The same input class, probed

An observation has two sources, and they are one input class: both land in the key as an obs: line, both are opaque values magus compares and never interprets, and describe target --cache prints them together. What differs is who supplies the value.

ctx.observes is the STATED form, for a fact a person maintains. The PROBED form is for a fact the tool can be asked, and it is declared by the spell rather than by the target, beside the version probe:

export fun mgs_getTools() > {str: Tool} {
    return {
        "trivy": Tool{
            probe = Command{bin = "trivy", args = ["version"]},
            key = VersionKey{upTo = VersionComponent.patch},
            observe = Command{bin = "trivy", args = ["version", "--format", "json"]},
        },
    };
}

The two probes answer questions on different clocks, which is why there are two. probe answers "which build of the tool", and trivy ships a release every few weeks. observe answers "which copy of the world it is reading", and the trivy database is built every six hours. Keying only the version is what makes a cached scan report yesterday's CVEs against today's image.

A probed observation is scoped to the targets that USE the tool, unlike a version probe, which keys every target in every project that binds the spell. It has to be: folding a value that moves every six hours into every target's key would cost the project's whole cache on a clock. The scoping comes from the same static op list describe target prints, so a target reaching the op through a helper the walk cannot follow gets no observation, and no protection.

The other half of a probed observation is that the copy the probe OBSERVES has to be the copy the scan READS. A tool holding a local copy satisfies that by not refreshing it behind your back. trivy image passes --skip-db-update by default and answers from the copy on disk, and the update charm drops the flag so the scan refreshes first. In magus's own workspace that is the whole difference between the two spellings:

magus run image-scan          # scan against the database on disk; cacheable, keyed on it
magus run image-scan:update   # refresh the database first, then scan

Without that split there is nothing stable to observe: a scan that silently refreshed its own feed would change its verdict under an unchanged tree and an unchanged key.

A tool that holds NO local copy satisfies the same invariant the other way. govulncheck reads https://vuln.go.dev on every run and caches nothing, so its probe and its scan read one feed: when the database moves, the key moves with it. There is no update arm because there is no pin to move forward, and update is the grant to replace a pin. The residual is a seconds-wide window where a publication lands between the probe and the scan, which costs a re-run rather than a stale pass, since the next probe carries the new date.

magus doctor checks the pairing. A cacheable target composing an op that declares external and has neither a probe nor skip_cache is MGS1033.

Four declarations, four different answers to "the answer depends on something that is not a source file":

Reach for skip_cache only when replaying would be wrong, not when it would be stale. Staleness has a declaration now.

Opting out and busting

Six controls, at different scopes:

skip_cache states that replaying this target would be wrong: it signs a fresh artifact, records a screen capture, mutates go.mod, rewrites a badge, or never returns at all. It is a claim about the target's nature, which is why it lives in the magusfile rather than in the operator's fingers. --no-cache says something entirely different and far weaker: I do not trust the cache for this one run. That is a session-level judgment, so it belongs on the command line.

The two are not interchangeable, and collapsing them breaks in both directions. Move a skip_cache target to --no-cache and correctness now depends on everyone remembering a flag, so a forgotten one replays a cached signature into a release. Reach for skip_cache when you merely wanted a fresh run and the target stops caching forever, for everyone.

skip_cache is not how you handle a target that produces no files. A pure orchestration target - a ci that only composes lint, build, and test - caches correctly with no policy at all: it snapshots an empty manifest and replays as a hit, while its stages keep their own entries. Output globs inherited from the project or a bound spell are allowed to match nothing, and only a glob the target declared itself via ctx.writesFiles must produce a file. If a no-output target ever fails at snapshot time, that is a bug to report, not a reason to opt out of the cache. Opting out instead costs the replay AND is indistinguishable from a real never-replays defect, which is what MGS1009 exists to catch.

Both skip_cache and --no-cache force a genuine re-execution; the mechanical difference is what happens to the cache entry afterward (never snapshot vs. snapshot-and-refresh). bust_cache and clean --cache both delete entries, at different granularities and from different sides of a run. cache.immutable is the odd one out: it does not force anything to re-run, it just stops the cache from ever writing - the common case is a read-only CI runner or a shared cache mirror that must not accumulate local entries.

Granularity: project-wide vs per-target

Without a target declaration, baseStep seeds the cache key with the project sources (its own globs plus every bound spell's needs) and the magusfile. That conservative default keeps an undeclared target safe, but it can make unrelated work invalidate together.

An explicit ctx.readsFiles(...) call changes that contract. It is the target's exact source footprint: magus keeps the magusfiles and that target's spell inputs, then hashes only the declared inputs. A ctx.readsFiles("src/**") build therefore does not re-run for a sibling Dockerfile. Use it when a target has a genuinely narrower domain, and name every source that domain reads.

That gives a clean rule for where to declare a glob:

• affects every target (a shared schema, a project-wide config) -> project-wide magus\project({sources = [...]}), declared once;
• affects one target -> ctx.readsFiles(...), ctx.writesFiles(...), or ctx.modifiesExistingFiles(...) in that target's body, according to the file relationship above.

Outputs are almost always target-specific (build -> dist/, test -> coverage/), so a project-wide outputs - which makes every target snapshot it - is usually the wrong tool; prefer ctx.writesFiles(...).

Replay: a hit restores outputs, not execution

On a run, magus computes the key, then looks for a manifest stored under it:

1. Hit. The manifest is read and its outputs are restored into the workspace. The target's body does not run - the export fun never executes on a hit. Each output is materialized from the content-addressed store by reflink (a copy-on-write clone) where the filesystem supports it, falling back to a byte copy. (Hard-linking is deliberately avoided: it would alias the shared blob and a later in-place rewrite would silently poison the cache.) Symlink outputs are restored as symlinks. Any captured build log recorded on the original run is replayed to stdout, so a cached pass looks like the real one.
2. Miss. The body runs. On success, magus snapshots the provides outputs: each file's content is hashed, its bytes are stored once in the content-addressed store (deduplicated by hash), and a manifest is written atomically recording every output's path, content hash, mode, and size. A subsequent identical run hits.

This is why the target result is emitted, not returned. A return value can't exist on a hit, since the body never ran - but a hit is exactly what you most want to report. So the dispatcher emits a run.target.result event ({project, target, status, cache_hit, duration_ms}) for both the ran and the cached case, sourced from the cache's per-run callback. See Results for how the event fits the run hierarchy.

A run that "wins the race" against a cancellation is neither snapshotted nor published: its outputs may be incomplete, so magus surfaces the cancellation instead of recording a poisoned entry.

One owner per generated file

A generated file has exactly one owning target: the one that declares it. Two targets writing the same bytes is the most common way to get a build that never settles, and it is worth being blunt about why, because the instinct it provokes is wrong.

Say generate writes gen/** and a formatter rewrites the same tree. The instinct is to call this a race and fix it with a dependency edge. It is not a race, and ordering cannot fix it:

• Generate, then format: the formatter's bytes land. The next generate regenerates unformatted output, sees it differ from what is committed, and fails its drift gate.
• Format, then generate: the formatting is immediately undone, and the formatter's own check fails instead.

Whichever runs last wins, and the loser's gate fails on the next run, at every possible ordering. A dependency edge resolves a producer and a consumer. Two producers of one file is an ownership violation, and there is no order that makes both correct.

If generated output needs formatting, the generator formats it, as the last thing it does. It still owns the final bytes, so its drift gate compares formatted output against formatted output and settles. Generated Go needs no special handling here for exactly this reason: mockery and protoc emit gofmt-clean output already.

Excluding generated trees from a formatter is the weaker fallback, and it is correct when nothing else needs to read that output. Every formatter in this workspace does it, and .markdownlintignore states the reasoning: a lint rule "fixed" in generated output is a fix in the wrong place, because the generator overwrites the edit on its next run. The fix belongs in the generator.

Declaring the same output glob from two targets is MGS1020; the cross-project shape, where two projects claim one glob under the same target, is MGS4002. Neither can see an undeclared write, which is why formatters are excluded by configuration as well as caught by a diagnostic. When a target genuinely needs to amend part of a file it does not own, that is ctx.modifiesExistingFiles, not a second output declaration.

The two roles of an output (maintainer note)

An output glob answers two different questions, and magus keeps them on two different code paths. Confusing them is the easiest way to introduce a stale-hit or a broken magus clean, so the model is worth stating once.

The cache role is per-target on purpose. A miss snapshots exactly the outputs in that target's Step, and a hit replays exactly those - so an output must be declared on the target that produces it. This is the producer-ownership rule, and violating it is a real bug, not a style nit: a glob declared project-wide is in every cacheable target's Step.Outputs, including targets that never write it. When one of those unrelated targets gets a cache hit, its replay restores the file to whatever it was when that target last ran - so a go-build hit can silently revert a freshly regenerated MAGUS.md. Scoping the output to its producer with magus\outputs means only the producer's hit replays it. Project-wide outputs is correct only when every target genuinely produces the glob, which is rare - most outputs belong to one generator.

The generated-files role is the union, because "clean everything this project generates" and "which project owns this path?" don't care which target produced what. A consumer that asks a generated-files question goes through AllOutputs(), never raw p.Outputs, or it silently misses per-target declarations. Today that means magus clean --outputs (CleanOutputs), output-ownership (FindOutputOwner), and the git merge driver (workspaceOutputGlobs). The cache path is the one place that stays per-target.

Inputs have just the one role (the cache key), so there is no AllInputs: a source glob that isn't in a given target's Step.Sources simply doesn't key that target, which is a footprint question, never a "what does the project consume" one.

Should generated output be committed?

Two ecosystems answer this in opposite directions, and each answer follows from its own build model. Go projects commit generated code - *.pb.go, stringer output, mocks - and a clean clone then builds with only the Go toolchain. TypeScript projects regenerate at build time and gitignore the result, since you cannot build without node_modules in the first place, so the committed copy would duplicate something the build already produces.

Both follow the same rule applied to different starting conditions: is the generator already required to build? Ask that first.

The first question is the one that decides it outright. A file recording its own commit cannot be committed and stay correct, whatever the other answers are - that is the next section. The rest trade cost against reach:

Commit it when a consumer cannot regenerate it. A Go module's generated code ships in the module zip. Leave it out and everyone importing your package needs protoc and your buf.gen.yaml to build. Committing moves that dependency from every consumer onto you, which is what the Go convention buys.

Regenerate it when the build already needs the generator. A generated TypeScript client is the usual case: the package manager and the bundler are prerequisites either way, so the committed copy duplicates them. It also churns, since bundled output shifts on a dependency bump.

Regenerate it when it is large or churns per commit. Every clone pays for committed output, CI included, and it pays forever. Untracking the rendered docs site and the console here removed 27% of this repository's blob history. Untracking does not shrink what earlier commits already hold, which is a separate problem that a blobless clone solves.

What each choice costs, and where magus sits

Not committing turns an artifact into a build-order dependency. Something has to regenerate the client before the code importing it compiles, and a repository without a build graph records that ordering in a README or a script. A committed file carries no such edge: it is present before anything runs.

magus lets you declare the edge instead. A generator states what it writes (magus\outputs), a consumer states what it depends on, and the run order comes from those declarations - magus affected reruns codegen when a .proto changes, FindOutputOwner resolves which project owns a generated path, and MGS4004 reports a project reading a path another project wrote without declaring the dependency.

That is a deliberate bias: magus prefers a coupling you write down to one you remember, so it invests in making the "regenerate it" option checkable. Where the ordering is declared, the main argument for committing falls away, and the reasons that remain are about consumers outside the workspace and readers who never run a build.

Either way, gate it. Committed output needs a plain magus run generate that fails when the tree changes, so review catches a forgotten regeneration. Untracked output needs CI to build it on the path that publishes it, or a broken generator ships the last good copy without anyone noticing.

The self-staling output: generated files that record VCS state

There is one combination of ordinary decisions that produces a build which can never be clean. Each half of it reads as sound practice on its own:

1. A generator records VCS state in its output - a "Last updated" line, the commit that produced a page, a build stamp.
2. That output is committed, because generated files are usually committed so a reader can see them and CI can drift-gate them.

Each is defensible. Together they cannot converge. Committing the source changes the commit, the commit is an input to the output, so the output you just committed is now stale. Regenerate and commit that, and the new commit stales the output again. Amending does not escape it either: a new hash restales the footer that recorded the previous one.

The only stable resting point is a second commit containing nothing but regenerated output, because a commit that does not touch a page's source does not change the commit that page records. That is why repositories in this state grow a trail of "refresh generated metadata" commits after every real one. Those commits are not sloppiness; they are the fixed point of the loop.

How to recognize it

The tell is a drift gate that passes before you commit and fails immediately after, with a diff containing only timestamps, hashes, or "last updated" lines. If magus run generate is clean, you commit, and magus run generate is suddenly dirty again, you are in this loop.

magus does not diagnose this today. It is genuinely hard to detect without a false positive: after the fix below, the same generator still writes the same commit hash into the same files, and the only thing that changed is whether those files are tracked. Distinguishing the broken state from the fixed one needs a "is this path tracked?" primitive that types.VCSDriver does not currently expose. Until it does, this section is the diagnostic.

The fix: stop committing the output, or stop recording the state

Two ways out, and they are not equally good.

Untrack the output and render at publish time. The generator keeps its provenance line, and the deploy renders from source with the final commit already known, so there is nothing to restale. This is what this repository does: the rendered docs site is generated into docs/gen/ and never committed (.github/workflows/cd.yaml renders it on every push to main). Cost: the output is no longer reviewable in a diff, and a broken generator now blocks a deploy that a file copy could never fail.

Or drop the VCS state from the output. If the provenance line is not worth the cycle, remove it and the output becomes a pure function of its sources, which is what a drift gate wants anyway.

What does not work is keeping both and being disciplined about it. The loop is structural, so "remember to regenerate and commit again" is a rule that has to hold forever, and the failure mode when it lapses is a silent one: the committed output simply describes a commit that is no longer the one it sits in.

The narrower rule this is an instance of

A committed generated file must be a pure function of its committed sources. Anything else in its inputs - the clock, the machine, the branch, the commit - turns "regenerate and diff" from a correctness check into noise. magus's drift gate assumes that purity, which is why the termcast-record target here is deliberately kept out of the generate umbrella: it records a live session, so its bytes are never the same twice and a drift gate over it would fail every run by construction. Rendering that committed capture IS pure, so termcast-generate sits inside the umbrella and is gated.

On disk: just files

The cache lives at .magus/ in the workspace root (override with MAGUS_CACHE_DIR, or cache.dir in magus.yaml). Three directories plus a hash memo carry a cache hit; .magus/ holds more beside them, including the lock files, the run journals, the knowledge shards and the symbol index:

.magus/
├── cas/         content-addressed blobs, sharded by the first two hex chars
│   └── ab/ab34...f0      one file per unique output content (deduplicated)
├── manifests/   one JSON manifest per cache entry
│   └── api/<key>.json     project path flattened; file named by cache key
├── logs/        captured build output, replayed on a hit
│   └── api/<key>.log
└── mtimes/      the per-file hash fast-path memo

A manifest is plain JSON you can read directly. It records the project path, the cache key, the target, and one record per output - path, content-address (blob), mode, size, and (for symlinks) the link target:

{
  "projectPath": "api",
  "hash": "ab34...f0",
  "target": "build",
  "outputs": [
    {
      "path": "api/dist/server.js",
      "blob": "9c1f...",
      "mode": 420,
      "size": 20481
    }
  ],
  "createdAt": "2026-07-07T12:00:00Z"
}

That transparency is deliberate: a hit or miss is answerable with ls and cat, and a build log for any entry is a file you can open. magus never mutates an existing manifest, and a manifest read back under the wrong key or project (copied or renamed onto the wrong slot) is treated as a miss rather than trusted.

Space is bounded two ways. An optional size cap (cache.size_mb / MAGUS_CACHE_SIZE_MB) drives LRU eviction of the oldest manifests after a build, and orphaned blobs are garbage-collected once no surviving manifest references them (blobs are shared, so a blob's bytes are only reclaimed when its last referencing manifest is evicted). Out of band, magus config cache prune evicts entries older than a cutoff. To force a clean rebuild of specific projects, magus clean --cache <project> drops their entries. The whole store is portable: magus config cache export / import move it as a gzip-tar.

Connecting to the remote cache

Everything above is local to one machine. A remote cache shares these exact artifacts across CI runners: on a local miss magus asks the remote provider for the artifact keyed by the same (projectPath, hash), and if found imports it into the local store so the ordinary hit path replays it - no rebuild. After a genuine build, magus uploads the artifact so the next machine hits.

The artifact is the same content: the manifest, its blobs, and the build log, packed as a gzip-tar. The key computation, the replay path, and the manifest format are identical - the remote layer only moves those bytes between machines. On top of that it adds a signed trust model: because a replayed artifact injects files into a consumer's build, every remote artifact is verified against an Ed25519 trust set before it is allowed to replay, and an unsigned or untrusted one falls back to a local build. That trust boundary, the provider contract, and CI wiring are covered in full in remote-cache; this page's model is what it builds on.

Cache tiers

The cache is two tiers: the local tier (L1, .magus/) and, once a remote backend is wired, the remote tier (L2). The run header names them, cache: local (read+write) or cache: github-actions + local (read+write). They follow standard two-tier semantics:

1. Priority. Reads go L1, then L2. An L1 hit never reads L2.
2. Read-through with promotion. An L2 hit is verified, promoted into L1, and replayed from L1, so the next run is an L1 hit. Promotion obeys the local write gate: with local writes off, an L2 hit replays from a staging directory that is discarded afterwards and never persisted.
3. Write-through. A built entry is stored in L1, then L2, each under its own gate. L2 is never written without L1.
4. Backfill. On a run that may write L2, an L1 hit whose key L2 lacks is stored in L2 in the background; the run does not wait for it. The run asks the backend whether it holds the key (has_artifact, a lookup without a download) first, and a backend that cannot answer is not backfilled.
5. Failure isolation. An L2 that is unreachable or failing degrades the run to L1 only: the first failure is reported and counted as failed, never as missed, and the rest of the run stops asking. It fails a step only when remote writes were declared required (cache.remote.write.enabled: true).
6. No invalidation. Keys are content-addressed, so an entry is never stale; each tier evicts on its own, L1 by cache.size_mb, least recently used first (every hit, a promotion included, marks its entry used), L2 by the backend's retention (magus config cache prune --remote).

Each tier's write gate is decided once, when the cache opens:

Declaring cache.remote.write.enabled: true makes remote writes required: it is a config error with local writes off, and an error at startup when a trust set is declared but no signing key is held. Declared false, the remote tier is only read, which is what a pull request in CI runs with; the header then says why, for example (read+write local, read-only remote; remote writes are off).

Knowledge-graph shards and published output bundles ride the remote tier too, under their own namespaces and the same gates: signed on the way out, verified on the way in, and never written by a run that may not write the remote tier.

Glossary

See also

• spells: where needs/provides are declared, and what a bound spell contributes.
• dependencies: how depends_on's dep: propagation and a magus\needs call each interact with this cache key.
• operations: the run hierarchy and the run.target.result event that fires on a hit.
• targets: what a Target is - the unit a cache key is computed and replayed for.
• charms: the execution modifiers that key into the cache as charm: lines.
• cache/output-refs: how the key's hex digest becomes a portable reference id, what is deliberately excluded from it, and a known leak that puts a tool's database timestamp in the key.
• remote-cache: sharing these artifacts across machines under a signed trust model.