magus v0.4.3 is out. See what's new
¶ View markdown source · ✎ Suggest an edit
3 min read

MGS1034: a target reads a file it declares as its own output

One target names the same path in both ctx.readsFiles and ctx.writesFiles:

[fail] source-is-also-output: 3 path(s) declared as both a source and an output of one
target; the cache restores the file before the target reads it, so an edit to it can
neither miss the cache nor be seen
    magus: buzz-test declares "assets/buzz-coverage.svg" as both a source and an output
    magus: coverage-badge declares "assets/coverage.svg" as both a source and an output

Why it cannot work

An output is a file magus owns end to end. On a cache hit it is restored wholesale from the snapshot, which is what makes a hit equivalent to a run.

Declaring it as an input too puts that same file in the cache key. The two facts cannot both hold: the bytes the key is computed from are the bytes the cache itself wrote a moment earlier. Edit the file by hand and the next run still hits, because by the time anything hashes it, the restored content is back. The target never sees your edit, and neither do you.

It is worst in the case that most invites it: a target that GENERATES a file and then compares what it generated against what is on disk, to fail when the committed copy is stale. Declaring the file it verifies looks like honesty about the footprint. What it actually builds is a check that compares the cache against itself and passes whatever the tree holds.

Resolve it

Drop it from ctx.readsFiles. A derived file is a function of the real inputs, so keying on those is what regenerates it; keying on the derived file buys nothing.

// before: the badge is on both sides
ctx.readsFiles("badge.buzz", "coverage.buzz", "assets/coverage.svg");
ctx.writesFiles("assets/coverage.svg");

// after: an output, and only an output
ctx.readsFiles("badge.buzz", "coverage.buzz");
ctx.writesFiles("assets/coverage.svg");

The comparison itself needs no declaration. Reading a file in the target body works whether or not it is in the footprint; the footprint decides what keys the cache, not what the body may open.

When the file is not really yours

Sometimes the overlap is a signal that ctx.writesFiles was the wrong declaration rather than ctx.readsFiles being the wrong one. Ask who owns the file:

  • magus produces it end to end, every time, from the inputs. It is an output. Declare it with ctx.writesFiles and do not read it.
  • The file is committed, several machines each refresh a part of it, and no single run here can rebuild the whole. It is an update. Declare it with ctx.modifiesExistingFiles, which is deliberately absent from the replay set, so a snapshot taken on one machine never replays over another's contribution and magus clean never deletes what no run here can produce. It is then free to stay an input, which is what makes a teammate's commit invalidate this target.

A per-platform coverage record is the second kind. A rendered badge is the first.

What this check cannot see

Only DECLARED refs. A target that writes a file without ctx.writesFiles is invisible here, and so is a footprint call the static walk cannot reach (MGS1004). It under-reports rather than over-reports.

See also

  • Cache - what a key is made of, and what a hit restores
  • MGS1004 - a footprint declaration the extractor cannot reach
  • MGS1019 - a committed output whose own bytes make it stale
  • MGS1020 - one output glob claimed by two targets
magusfile diagnosticsLanding page for MGS1xxx diagnostics that flag authoring mistakes in a workspace's magusfile, such as missing targets or unresolved declarations. MGS1036: a narrowed footprint dropped the files its own ops readA target declares ctx.readsFiles, which replaces the project baseline, and then runs a spell op over file kinds that footprint never names. The op replays instead of running, and the gate stays green. MGS1018: dead output globA project declares an output glob that matches nothing while its other outputs match files. A target that inherits the glob fails its snapshot on a cold cache, long after the mistake was made. MGS4007: undeclared source modifiedFires when a target rewrites a file it declared as a source without declaring it with ctx.modifiesExistingFiles, so the cache key that identified the result no longer describes the inputs that produced it.
MGS1034magusfilecachefootprintreadsFileswritesFilesoutputs
Last updated (d966bfce)
Glossary

Magusfile

The magusfile.buzz that declares a project's targets (as export funs) and binds its spells. See targets.

Target

A named operation (build, test, ...) you invoke with magus run <target>; it may compose a spell's tool-native operations and depend on other targets. See targets.

Op

A single tool-native command a target composes (long form: operation); the middle of the work hierarchy (Spell to Op to Target). See operations.

Buzz

The language magusfiles are written in (the .buzz engine). See engines.

Cache

The content-addressed store magus consults before running a target, so unchanged work is skipped. See cache.

CI

An ordinary magusfile-defined target you compose yourself with magus\needs - magus does not hardcode its stages. Magus.RunCI treats it specially only in that it strips the rw charm, it is the anchor magus affected ci keys off, and a selected scope with no project declaring it is a load error rather than a silent no-op. See targets.

Snapshot

A point-in-time view of live state - the pool's occupancy or a tick of exported metrics - as opposed to accumulated history. See server.

Run

One target executing under one magus invocation, such as magus run test web or magus affected ci. A run keeps its captured output behind an output reference. Every magus run is a run whether or not any job asked for it; see Job for how the two relate.

Conventions

This page uses none of the site's convention markers. The full set is on the conventions page.