magus v0.4.3 is out. See what's new
¶ View markdown source · ✎ Suggest an edit
2 min read

Built for the largest monorepos

The problem magus exists for is hard, and I am not pretending otherwise. A monorepo build tool has to know every project, every input, and what a change reaches, and answer fast enough that a person or an agent asks it instead of guessing. At the scale of the largest monorepos anyone runs, that takes engineering a sane person would not do for a small repository: content-keyed caches, an index that answers without loading the tree, allocation counted on hot paths, a garbage collector tuned per phase.

So performance is not a polish pass at the end. A lookup that takes seconds teaches people and agents to grep instead, and then every guarantee the graph offers goes unused. The work is warranted, and it is also where clever code is most tempting, so it runs under the rule in Clear is better than clever: measure first, keep the measurement beside the code, and keep the clear version wherever it is fast enough.

The ambition sets the bar for design, not only for speed. A shape that works for a hundred projects and falls over at a hundred thousand is not done, and a feature that needs the whole tree in memory to answer one question is the wrong shape at any size.

Enforced by

  • The benchbeside analyzer: every // optimization: comment and every cache or memo type sits in a package with a benchmark.
  • Benchmarks publishes every run beside the hardware, the date and the competing tool versions.
  • Nothing yet commits a scale target (projects, files, a lookup's latency at that size); that is a debt in Where this is strained.

Good

Answer from the index, keyed on content, without loading or rebuilding the tree:

magus affected ci --plan -o json
magus refs decodeFile --occurrences

Bad

for each project in the workspace:
    load its magusfile
    hash every input
    compare against the last run

It is correct and it is linear in the size of the repository on every invocation, so the largest repositories, the ones that need the answer most, wait longest for it.

See also

designprincipleperformancescale
Last updated (2df714e8)
Glossary

Workspace

The magus root directory that owns a set of projects and shared config; the unit magus operates over. Workspace in full.

Project

A directory magus recognizes as a unit of work (it has a magusfile); the unit of caching, scheduling, and dependency tracking. Project in full.

Magusfile

The magusfile.buzz that declares a project's targets (as export funs) and binds its spells. Magusfile in full.

Target

A named operation (build, test, ...) you invoke with magus run <target>; it may compose a spell's tool-native operations and depend on other targets. Target in full.

Op

A single tool-native command a target composes (long form: operation); the middle of the work hierarchy (Spell to Op to Target). Op in full.

Engine

The interpreter a magusfile runs on; magus embeds the Buzz engine. Engine in full.

Cache

The content-addressed store magus consults before running a target, so unchanged work is skipped. Cache in full.

Affected

The set of projects touched by a change; magus affected <target> runs a target only over them. Affected in full.

CI

An ordinary target you compose yourself with ctx.needs; magus does not hardcode its stages. It is the anchor magus affected ci keys off, and ci strips the rw charm. CI in full.

Latency

How long an operation takes. magus records latency as OpenTelemetry histograms per family (target execution, cache op, pool wait, and graph query) and reports each as a count, sum, and percentiles. Latency in full.

Agent

A model working through an agent host such as Claude Code, Codex, Cursor or OpenCode. magus treats an agent as one more caller of its CLI, MCP tools and console, held to the same checks as a person. Agent in full.

Tool

An action a model can call through its host: a name, a description, a typed input and a result. magus serves its tools over MCP, each a read of the workspace model or a run of the CLI it already has. Tool in full.

Invocation

One magus process's recorded facts (the targets it finished, their outcomes, the lease it acted as, and the session it ran in when a host delivered one), kept in a repo-scoped store every worktree shares. magus session lists them; the store prunes itself by last-fact age. Invocation in full.

Run

One target executing under one magus invocation, such as magus run test web or magus affected ci. A run keeps its captured output behind an output reference. Every magus run is a run whether or not any job asked for it; see Job for how the two relate. Run in full.

Conventions

This page uses none of the site's convention markers. The full set is on the conventions page.