magus v0.4.3 is out. See what's new
¶ View markdown source · ✎ Suggest an edit
3 min read

MGS3011: target exceeded its declared timeout

A target ran longer than the ceiling its magusfile declared. magus cancelled it, killed its process tree, and failed that target. The message carries every input to the decision, so a reader can reconstruct it without opening any code:

[fail] magus security (ran, 15m0s)
  cause: [MGS3011] target "security" exceeded its declared timeout of 15m after 15m0s;
    its process tree was killed; captured output:
    /repo/.magus/logs/9c4f...log

A timeout is a failure. Not a skip, not a warning, and never a pass: a target that was told it may take fifteen minutes and took more has not done its job, and a build tool that reports that as success is worse than one with no ceiling at all.

Why

The incident this exists for: a gate held three project locks for over an hour with zero concurrency slots in use and nothing running. No target failed, so nothing named a culprit, and the invocation had to be killed by hand. The likely cause was a network-bound scanner whose far end stopped answering, a shape with no natural failure mode, because waiting forever is what a socket does.

Nothing else in magus reclaims that. The cache waits on the target, the limiter waits on the cache, and the machine-wide budget (MGS3009) hands out claims that are only released when a run exits. One target that never returns holds all of it.

What was killed

The ceiling bounds the whole target, subprocesses included. magus starts every subprocess in its own process group for exactly this reason, so expiry sends SIGTERM to the group and then SIGKILL to whatever ignored it, grandchildren the target's own script forked included. There is no orphaned scanner still holding the network socket after this fires.

Because the deadline rides the context, a ceiling on the target you SCHEDULED bounds every target it composes with ctx.needs. A composed target that declares its own, tighter ceiling is bounded by that one first, and the message then names the composed target rather than the gate that ran it, which is the difference between "ci hung" and "security hung".

That tighter ceiling stops where the composed target's own dependencies begin. A dependency is dispatched once and awaited by everyone that needs it, so a ceiling declared halfway down the chain would govern work its siblings depend on too: security declaring fifteen minutes must not decide how long generate may take, because lint, build and test all wait on the same run of it. A dependency reached this way stays bounded by the scheduled unit's ceiling, and by the stall watchdog (MGS3012).

Resolution

  1. Read the captured log the message names. The ceiling says when magus gave up, not what the target was doing. The log holds the last thing it printed, which is usually the answer.

  2. Fix the target, if the log shows it genuinely stopped making progress: an unreachable registry, a scanner database mirror that stalled, a prompt nobody is going to answer.

  3. Raise the ceiling, if the target legitimately got slower. Declare a multiple of the worst run on record, never a figure near its typical one:

    import "magus";
    
    magus\project({
        "targets": {
            "security": {"timeout": "30m"},
        },
    });
    

    MGS1032 is the check that tells you when a ceiling has drifted into either mistake, measured against the durations magus recorded.

  4. Remove the declaration, if the target should not be bounded at all. An undeclared target is unbounded, which is the default and stays the default; nothing infers a ceiling for you.

See also

  • MGS1032: a declared ceiling that no longer describes the target.
  • MGS3009: the machine-wide budget a hung target holds while it hangs.
  • MGS3012: the other half (an invocation making no progress, in work no target declared a ceiling for).
  • Spells: what magus bounds, and target_timeout, the workspace-wide runaway guard this is the per-target form of.
MGS3011timeouttarget_timeoutrunawayhanglocksprocess groupdeclarations
Last updated (353014be)
Earlier changes on this page (5)

Full history ↗ · Blame source ↗

Glossary

Workspace

The magus root directory that owns a set of projects and shared config; the unit magus operates over. Workspace in full.

Project

A directory magus recognizes as a unit of work (it has a magusfile); the unit of caching, scheduling, and dependency tracking. Project in full.

Magusfile

The magusfile.buzz that declares a project's targets (as export funs) and binds its spells. Magusfile in full.

Target

A named operation (build, test, ...) you invoke with magus run <target>; it may compose a spell's tool-native operations and depend on other targets. Target in full.

Op

A single tool-native command a target composes (long form: operation); the middle of the work hierarchy (Spell to Op to Target). Op in full.

Spell

A language/runtime adapter (e.g. go, md) that maps generic targets onto a toolchain's real commands. Spell in full.

Buzz

The language magusfiles are written in (the .buzz engine). Buzz in full.

Cache

The content-addressed store magus consults before running a target, so unchanged work is skipped. Cache in full.

CI

An ordinary target you compose yourself with ctx.needs; magus does not hardcode its stages. It is the anchor magus affected ci keys off, and ci strips the rw charm. CI in full.

Slot

One unit of the pool's capacity. A target acquires the slots it needs to run (most take one) and releases them when it finishes; the pool tracks capacity (total slots), running (acquired), and queued (blocked). Slot in full.

Concurrency

How many targets run at once. It is bounded by the pool's capacity and set with --concurrency, MAGUS_CONCURRENCY, or the concurrency config key. Concurrency in full.

Tool

An action a model can call through its host: a name, a description, a typed input and a result. magus serves its tools over MCP, each a read of the workspace model or a run of the CLI it already has. Tool in full.

Invocation

One magus process's recorded facts (the targets it finished, their outcomes, the lease it acted as, and the session it ran in when a host delivered one), kept in a repo-scoped store every worktree shares. magus session lists them; the store prunes itself by last-fact age. Invocation in full.

Job

The unit of delegated work, and one row of the job store: what an orchestrating agent handed out, with its goal, the checkpoint it was cut against, the paths it may write or must not touch, and the one check it runs. A job's holder is either a session, for work an orchestrator handed out, or the daemon, for its own maintenance. The store records; the agent guard is what reads those facts back when grading a write. Job in full.

Run

One target executing under one magus invocation, such as magus run test web or magus affected ci. A run keeps its captured output behind an output reference. Every magus run is a run whether or not any job asked for it; see Job for how the two relate. Run in full.

Lease

The grant a holder takes on a job: the write and read paths that job declared, enforced in the checkout that took it with magus job exec. A job is the piece of work; a lease is permission over it. Lease in full.

Conventions

This page uses none of the site's convention markers. The full set is on the conventions page.