magus v0.4.3 is out. See what's new
¶ View markdown source · ✎ Suggest an edit
1 min read

MGS3023: pipe cycle

A magus run whose stdin comes from another magus waits until that upstream stage is done with the projects the run needs (see Piping one magus into another). The chain of writers is followed through other tools too (cat, tee, jq). When it leads back to the run itself past another magus stage, every stage in the loop would wait on the one before it. None of them could ever start, so the run is refused before it takes any lock:

[MGS3023] this run's standard input is written, through magus pid 40118, by this run
  itself; each stage would wait on the one before it, so it is refused instead. Break
  the loop so one stage's input does not depend on its own output.

A plain shell pipe cannot form this loop. It takes a construct that wires a process's output back into its own input, such as bash's coproc, or a program that connects two pipes by hand.

Resolution

  • Break the loop. Write the upstream's output to a file, then run the downstream stage with the file as its input.

See also

  • MGS3007: a nested run that needs a project its own ancestor holds.
  • Concurrency: how one magus waits on another it reads from.
MGS3023lockspipecyclecoprocstdin
Last updated (ff2197b9)
Glossary

Project

A directory magus recognizes as a unit of work (it has a magusfile); the unit of caching, scheduling, and dependency tracking. See workspace.

Op

A single tool-native command a target composes (long form: operation); the middle of the work hierarchy (Spell to Op to Target). See operations.

Concurrency

How many targets run at once. It is bounded by the pool's capacity and set with --concurrency, MAGUS_CONCURRENCY, or the concurrency config key. See server.

Run

One target executing under one magus invocation, such as magus run test web or magus affected ci. A run keeps its captured output behind an output reference. Every magus run is a run whether or not any job asked for it; see Job for how the two relate.

Conventions

This page uses none of the site's convention markers. The full set is on the conventions page.