magus v0.4.3 is out. See what's new
¶ View markdown source · ✎ Suggest an edit
4 min read

Engines

A magusfile is written in Buzz and runs on the embedded Buzz VM through a small internal seam. A magusfile.buzz exposes the magus.* API and composes spells, targets, and charms. This page covers the seam and how a new language would plug in.

The engine interface

The backend implements one small interface, engine.Engine, which is a factory for engine.Session:

// internal/interp/engine/engine.go
type Engine interface {
    ID() string
    NewSession(ctx context.Context) (Session, error)
}

type Session interface {
    Close() error
    SetGlobal(name string, v Value)
    GetGlobal(name string) Value
    NewTable() Table
    LoadString(code string) (Value, error)
    DoString(code string) error
    Call(p CallParams, args ...Value) error
}

Value and Table are engine-neutral handles, so host code reads and writes script values without knowing the concrete VM. The backend registers itself at init() time:

// internal/interp/engine/buzz/buzz.go
func init() { engine.Register("buzz", engineImpl{}) }

and is found by name through engine.Lookup(name). The registered backend in a stock magus binary is buzz.

The backend is pulled into the binary by blank import in cmd/magus/packs_interp.go:

_ ".../internal/interp/engine/buzz"

The spell contract

A spell exports a fixed set of mgs_-prefixed functions (see spells). The list of optional functions and the decoder keys they map to is single-sourced in internal/spell/contract.go as OptionalContract, and the Buzz resolver (internal/spell/resolve.go) iterates that one list. A spell's mgs_ functions decode to a Spec for every scalar and list contribution (needs, provides, version_cmd, opaque) and for record-shaped ops ({bin, args, charms}).

An op is a command or a service; a function-valued op is fun(Target) > Command or fun(Target) > Service, called once at load to record the declarative value it returns (a Command's {bin, args, charms} or a Service's {command, readiness?, stop?}, a long-running process magus run blocks on). The op's kind is inferred from that value, so one spell mixes both. A remote cache backend is not an op: it is a spell that exports the backend functions (enabled/get_artifact/put_artifact/has_artifact/prune), detected by name and wired with magus\cache.remote (see Remote caching).

Doc-comment capture. Buzz captures a handler's doc comment at compile time (the parser binds the comment to the function node; FunDoc reads it back), and magus doctor enforces one on each function-handler target. Note Buzz's Chunk.Doc is in-memory only and not serialized to bytecode, so Buzz captures docs only for freshly-compiled workspace .buzz spells, never the embedded built-ins.

Host modules are a superset of Buzz's stdlib

magus layers its host methods onto Buzz's own stdlib modules under the same bare names: import "os" carries both Buzz's os.* (sleep, env, execute) and magus's additions (proc\exec, proc\which, ...); import "fs" carries Buzz's fs plus fs\glob/readFile; and magus adds whole modules Buzz lacks (vcs, archive, http, charm, ...). One import per domain covers the union, with no separate extra namespace to remember which side a call lives on.

Where a method overlaps a Buzz stdlib call, the magus form is sandbox-aware while the bare stdlib is not. For example, env\get/lookup honor the env allowlist, whereas Buzz's os\env is raw. Those overlaps are noted per-method in the module reference (either works); the cross-reference lives in std/buzz_stdlib.go.

A few entries are not treated as duplicates because the magus behavior the stdlib can't reproduce: magus's os\exit raises a lifecycle error (Buzz's hard-exits the process), magus's os\sleep is cancellable (Buzz's blocks), and magus's crypto.*_file hashes a file (Buzz's hash only takes a string). These stay on the magus surface.

A workspace spell lives at spells/<name>/spell.buzz (or flat spells/<name>.buzz).

"Built-in spell" vs language "builtins"

A built-in spell is a spell whose bytecode is compiled from spells/<name>/spell.buzz and embedded in the magus binary (go, typescript, docker, ...; see spells). This is a magus concept and is unrelated to Buzz's language builtins (spawn, list/map methods, etc.), which are part of the Buzz language itself. The docs always write "built-in spell" when they mean the former.

Adding a new language

The engine interface is the stable, clean part of the seam. Plugging in a second language today, however, also touches a handful of hard-coded dispatch spots above the interface. This is the current state, not the end state:

  1. Implement engine.Engine/Session for the VM and engine.Register it (the clean part).
  2. Map the file extension to the engine in engineForExt (internal/interp/source.go) and add the glob to scriptExts / the magusfile.<ext> lists.
  3. Branch the runtime where it special-cases an engine by name (src.Engine == "buzz" in internal/interp/runtime.go).
  4. Provide the per-engine host bindings (the magus.* surface), as internal/interp/bindings/buzz.go does today.

Future direction: registry-driven discovery. The intent is to derive extensions, magusfile filenames, and dispatch from the engine registry itself, so adding a language means registering a backend (with its extensions and binding installer) and nothing else, with no edits to source.go, runtime.go, or switch statements. That refactor is deliberately out of scope for now. The hard-coded spots above are the seam's known leaks, documented so they are visible rather than surprising.

See also

enginesbuzzvminterpreterruntimemagusfilepluginsession
Last updated (c44f35b3)
Earlier changes on this page (7)

Full history ↗ · Blame source ↗

Glossary

Workspace

The magus root directory that owns a set of projects and shared config; the unit magus operates over. See workspace.

Magusfile

The magusfile.buzz that declares a project's targets (as export funs) and binds its spells. See targets.

Target

A named operation (build, test, ...) you invoke with magus run <target>; it may compose a spell's tool-native operations and depend on other targets. See targets.

Op

A single tool-native command a target composes (long form: operation); the middle of the work hierarchy (Spell to Op to Target). See operations.

Spell

A language/runtime adapter (e.g. go, md) that maps generic targets onto a toolchain's real commands. See spells.

Charm

An execution modifier attached with : (lint:rw) that changes how a target runs, not which one; the built-in rw flips a check-only target to mutate in place, and ci always strips it. See charms.

Module

A magus stdlib namespace a magusfile imports for host capabilities: filesystem, exec, vcs, and more. See the module reference.

Buzz

The language magusfiles are written in (the .buzz engine). See engines.

Engine

The interpreter a magusfile runs on; magus embeds the Buzz engine. See engines.

Cache

The content-addressed store magus consults before running a target, so unchanged work is skipped. See cache.

Sandbox

The restricted filesystem and environment a target runs in, so builds stay reproducible and side-effect-free. See sandbox.

Service

A long-running or shared process magus manages across runs, distinct from a one-shot target. See services.

CI

An ordinary magusfile-defined target you compose yourself with magus\needs - magus does not hardcode its stages. Magus.RunCI treats it specially only in that it strips the rw charm, it is the anchor magus affected ci keys off, and a selected scope with no project declaring it is a load error rather than a silent no-op. See targets.

Remote cache

A CI-only backend that shares content-addressed artifacts across runners: a cold machine replays a build another runner already did instead of rebuilding. Every remote artifact must be signed by a trusted key. See remote.

Session

An agent host's conversation, by the id the host delivers to its hooks. magus never mints one: a record with no session is unattributed, and the OS user it carries says whose account ran it.

Run

One target executing under one magus invocation, such as magus run test web or magus affected ci. A run keeps its captured output behind an output reference. Every magus run is a run whether or not any job asked for it; see Job for how the two relate.

Conventions

Placeholders

Angle brackets mark a value you replace with your own - never type the brackets:

magus run <target>
magus completion <shell>    # e.g. bash, zsh, fish

<target>, <path>, <shell>, <name> and the like are stand-ins, not literal text.