magus v0.3.0 is out. See what's new
¶ View markdown source · ✎ Suggest an edit
4 min read

Engines

A magusfile is written in Buzz and runs on the embedded Buzz VM through a small internal seam. A magusfile.buzz exposes the magus.* API and composes spells, targets, and charms. This page covers the seam and how a new language would plug in.

The engine interface

The backend implements one small interface, engine.Engine, which is a factory for engine.Session:

// internal/interp/engine/engine.go
type Engine interface {
    ID() string
    NewSession(ctx context.Context) (Session, error)
}

type Session interface {
    Close() error
    SetGlobal(name string, v Value)
    GetGlobal(name string) Value
    NewTable() Table
    LoadString(code string) (Value, error)
    DoString(code string) error
    Call(p CallParams, args ...Value) error
}

Value and Table are engine-neutral handles, so host code reads and writes script values without knowing the concrete VM. The backend registers itself at init() time:

// internal/interp/engine/buzz/buzz.go
func init() { engine.Register("buzz", engineImpl{}) }

and is found by name through engine.Lookup(name). The registered backend in a stock magus binary is buzz.

The backend is pulled into the binary by blank import in cmd/magus/packs_interp.go:

_ ".../internal/interp/engine/buzz"

The spell contract

A spell exports a fixed set of mgs_-prefixed functions (see spells). The list of optional functions and the decoder keys they map to is single-sourced in internal/spellruntime/contract.go as OptionalContract, and the Buzz resolver (internal/spellruntime/resolve.go) iterates that one list. A spell's mgs_ functions decode to a Spec for every scalar and list contribution (needs, provides, claims, version_cmd, opaque) and for record-shaped ops ({bin, args, charms}).

An op is a command or a service; a function-valued op is fun(Target) > Command or fun(Target) > Service, called once at load to record the declarative value it returns (a Command's {bin, args, charms} or a Service's {command, readiness?, stop?}, a long-running process magus run blocks on). The op's kind is inferred from that value, so one spell mixes both. A remote cache backend is not an op: it is a spell that exports the backend functions (enabled/get_artifact/put_artifact/prune), detected by name and wired with magus\cache.remote (see Remote caching).

Doc-comment capture. Buzz captures a handler's doc comment at compile time (the parser binds the comment to the function node; FunDoc reads it back), and magus doctor enforces one on each function-handler target. Note Buzz's Chunk.Doc is in-memory only and not serialized to bytecode, so Buzz captures docs only for freshly-compiled workspace .buzz spells, never the embedded built-ins.

Host modules are a superset of Buzz's stdlib

magus layers its host methods onto Buzz's own stdlib modules under the same bare names: import "os" carries both Buzz's os.* (sleep, env, execute) and magus's additions (os\exec, os\which, ...); import "fs" carries Buzz's fs plus fs\glob/readFile; and magus adds whole modules Buzz lacks (vcs, archive, http, charm, ...). One import per domain covers the union, with no separate extra namespace to remember which side a call lives on.

Where a method overlaps a Buzz stdlib call, the magus form is sandbox-aware while the bare stdlib is not. For example, env\get/lookup honor the env allowlist, whereas Buzz's os\env is raw. Those overlaps are noted per-method in the module reference (either works); the cross-reference lives in std/buzz_stdlib.go.

A few entries are not treated as duplicates because the magus behavior the stdlib can't reproduce: magus's os\exit raises a lifecycle error (Buzz's hard-exits the process), magus's os\sleep is cancellable (Buzz's blocks), and magus's crypto.*_file hashes a file (Buzz's hash only takes a string). These stay on the magus surface.

A workspace spell lives at spells/<name>/spell.buzz (or flat spells/<name>.buzz).

"Built-in spell" vs language "builtins"

A built-in spell is a spell whose bytecode is compiled from spells/<name>/spell.buzz and embedded in the magus binary (go, typescript, docker, ...; see spells). This is a magus concept and is unrelated to Buzz's language builtins (spawn, list/map methods, etc.), which are part of the Buzz language itself. The docs always write "built-in spell" when they mean the former.

Adding a new language

The engine interface is the stable, clean part of the seam. Plugging in a second language today, however, also touches a handful of hard-coded dispatch spots above the interface. This is the current state, not the end state:

  1. Implement engine.Engine/Session for the VM and engine.Register it (the clean part).
  2. Map the file extension to the engine in engineForExt (internal/interp/source.go) and add the glob to scriptExts / the magusfile.<ext> lists.
  3. Branch the runtime where it special-cases an engine by name (src.Engine == "buzz" in internal/interp/runtime.go).
  4. Provide the per-engine host bindings (the magus.* surface), as internal/interp/bindings/buzz.go does today.

Future direction: registry-driven discovery. The intent is to derive extensions, magusfile filenames, and dispatch from the engine registry itself, so adding a language means registering a backend (with its extensions and binding installer) and nothing else, with no edits to source.go, runtime.go, or switch statements. That refactor is deliberately out of scope for now. The hard-coded spots above are the seam's known leaks, documented so they are visible rather than surprising.

See also

enginesbuzzvminterpreterruntimemagusfilepluginsession
Last updated (63b34b40)
Earlier changes on this page (3)

Full history ↗ · Blame source ↗

Glossary

Workspace

The magus root directory that owns a set of projects and shared config; the unit magus operates over. See workspace.

Magusfile

The magusfile.buzz that declares a project's targets (as export funs) and binds its spells. See targets.

Target

A named operation (build, test, ...) you invoke with magus run <target>; it may compose a spell's tool-native operations and depend on other targets. See targets.

Spell

A language/runtime adapter (e.g. go, md) that maps generic targets onto a toolchain's real commands. See spells.

Charm

An execution modifier attached with : (lint:rw) that changes how a target runs, not which one; the built-in rw flips a check-only target to mutate in place, and ci always strips it. See charms.

Module

A magus stdlib namespace a magusfile imports for host capabilities: filesystem, exec, vcs, and more. See the module reference.

Buzz

The language magusfiles are written in (the .buzz engine). See engines.

Engine

The interpreter a magusfile runs on; magus embeds the Buzz engine. See engines.

Cache

The content-addressed store magus consults before running a target, so unchanged work is skipped. See cache.

Sandbox

The restricted filesystem and environment a target runs in, so builds stay reproducible and side-effect-free. See sandbox.

Service

A long-running or shared process magus manages across runs, distinct from a one-shot target. See services.

CI

An ordinary magusfile-defined target you compose yourself with magus\needs - magus does not hardcode its stages. Magus.RunCI treats it specially only in that it strips the rw charm, it is the anchor magus affected ci keys off, and a selected scope with no project declaring it is a load error rather than a silent no-op. See targets.

Remote cache

A CI-only backend that shares content-addressed artifacts across runners: a cold machine replays a build another runner already did instead of rebuilding. Every remote artifact must be signed by a trusted key. See remote-cache.

Conventions

Placeholders

Angle brackets mark a value you replace with your own - never type the brackets:

magus run <target>
magus completion <shell>    # e.g. bash, zsh, fish

<target>, <path>, <shell>, <name> and the like are stand-ins, not literal text.