This handbook records how Shoal fits together: the boundaries that matter, the invariants hidden behind them, and the paths a value takes from source text to a terminal, a journal entry, or an agent. It is written to make architectural drift visible before it becomes archaeology.
Architecture
Architecture Atlas
A maintainer's map of Shoal: crate boundaries, evaluation flow, kernel sessions, execution, protocols, persistence, security, and design invariants.
- For
- Contributors and maintainers
Browse the collection
Books and chapters
Book 012 chapters
Orientation
Composition roots, ownership boundaries, and the fastest paths into the codebase.
Book 0213 chapters
Language & runtime
Parsing, evaluation, values, streams, and the invariants that connect them.
03 Language engineHow source becomes a mode-aware AST, how the evaluator assigns meaning, and where command and expression semantics meet. 04 AST model, spans, and serializationEvery syntax node, the byte-span contract, command structure, serde shape, AST version boundary, and safe evolution workflow. 05 Lexer, parser, and formatter internalsModal tokenization, statement dispatch, Pratt precedence, command grammar, ambiguity resolutions, incomplete-input classification, and canonical formatting. 06 Evaluator state and control flowA field-by-field map of the tree-walk evaluator, statement and expression flow, scope transitions, echo semantics, and host boundaries. 07 Calls, closures, modules, and namespacesCallable dispatch, strict argument binding, closure capture, module evaluation and caching, constructors, and synthetic namespace behavior. 08 Builtin registry and command dispatchThe complete builtin inventory, dispatch precedence, typed argument path, structured filesystem commands, special verbs, outcome wrapping, and safe addition workflow. 09 Language and conformance contractThe stable authority model for Shoal syntax and semantics, executable registries, error/render contracts, and the normative TOML corpus. 10 Values, streams, and executionThe runtime value algebra, lazy stream protocol, command outcomes, process capture, PTYs, cancellation, and byte boundaries. 11 Runtime value algebraEvery Value variant, equality and condition contracts, error anatomy, feed serialization, JSON projection, rendering, and lazy CAS bytes. 12 Value method dispatchDispatch precedence, complete method families, collection and string semantics, path effects, outcome forwarding, task methods, and extension workflow. 13 Streams, channels, and backpressureThe pull protocol, single-consumption state, every/watch/tail sources, lazy stages, tee queues, channel replay, sinks, cancellation, and overflow semantics. 18 Local shell hostHow the shoal binary assembles configuration, evaluator state, line editing, completion, prompt snapshots, jobs, history, and local journal behavior. 22 Reef, adapters, and configurationProject-scoped tool resolution, lock and drift semantics, executable views, polyglot runners, declarative CLI adapters, and their host integration.
Book 034 chapters
Execution & security
Process boundaries, host behavior, effects, policy, and best-available enforcement.
14 Effects, plans, ports, and authorityHow Shoal describes side effects, derives stable plans, evaluates principal policy, lowers sandboxes, and keeps testable capabilities at the runtime edge. 15 Process execution and captureExecSpec resolution, capture-mode pipes, cancellation escalation, result status, bounded memory and disk spill, stdin modes, sandbox wrapping, and failure ownership. 16 PTYs, terminal sessions, and job controlHuman PTY tee mechanics, raw-mode handoff, stop/park/resume lifecycle, evaluator job bridging, agent screen sessions, key encoding, and cleanup invariants. 17 Authority, authentication, secrets, and sandbox threat modelTrust boundaries from bearer attachment through Leash plans and OS confinement, secret storage/injection, WASM validation, known bypasses, and fail-open/fail-closed choices.
Book 043 chapters
Kernel & agents
Sessions, wire contracts, bounded values, tasks, events, and MCP projection.
19 Kernel, sessions, and protocolThe Unix-socket server, session and connection lifetimes, RPC router, exec lifecycle, wire elision, events, tasks, plans, and PTYs. 20 Kernel RPC handler referenceA method-by-method reference for Shoal kernel attachment, execution, value, plan, task, PTY, journal, event, and introspection handlers. 21 Agent and MCP surfaceHow shoal-mcp projects kernel methods into bounded tools, URI resources, subscriptions, and a zero-config stdio server.
Book 056 chapters
Storage & tooling
Reef, adapters, configuration, journal persistence, CAS, and durable reconstruction.
23 Configuration reference and wiring auditThe exact shoal-config schema, layer and environment precedence, validation behavior, host consumers, and settings that are currently inert. 24 Prompt, editor, completion, picker, and LSPThe pure prompt renderer and host snapshot seam, Reedline editing stack, context-sensitive completion and highlighting, fuzzy picker, and deliberately lexical language server. 25 Reef resolution, locks, providers, and executable viewsThe complete source-derived path from scoped manifests through constraint refinement, provider ranking, lock/hash validation, PATH synthesis, runners, and evaluator integration. 26 Journal, CAS, undo, and persistenceThe SQLite entry lifecycle, compressed content-addressed output store, lazy bytes, safe undo, garbage collection, and durable event reconstruction. 27 Journal, CAS, undo, and storage referenceThe exact SQLite schema, entry lifecycle, content-addressed store, spill, undo, garbage collection, query, migration, and kernel replay contracts. 28 LSP, doctor, testing, and quality gatesEditor services, installation diagnostics, the normative conformance corpus, integration-test ownership, property/fuzz coverage, CI, and release verification.
Book 065 chapters
Maintenance
Change boundaries, known debt, test ownership, quality gates, and release discipline.
29 Inter-crate and protocol contractsStable dependency, API, port, wire, error-code, storage, and compatibility boundaries across the Shoal workspace. 30 Change map, invariants, and known debtA contributor blast-radius guide, layered error model, non-negotiable invariants, and a source-grounded register of current architecture risks and incomplete paths. 31 Implementation status and evidence ledgerA source-audited maturity ledger for every Shoal subsystem, host surface, security boundary, platform promise, and historical design claim. 32 Roadmap and engineering prioritiesA dependency-ordered roadmap for closing Shoal's authority, protocol, composition, persistence, language, and portability gaps with testable exit criteria. 33 Documentation governance and canonicalizationHow the Zola atlas, executable schemas, source comments, generated references, and historical root docs stay synchronized without creating parallel truth.