This page freezes the findings of the 2026-07-16 read-only deep audit so remediation can be tracked against a stable record. Every finding has an ID (A1, B2, …). The hardening roadmap maps every ID to at least one atomic task; completing that roadmap addresses every finding on this page. Do not edit findings here — if a finding turns out to be wrong, mark it withdrawn with a note and adjust the roadmap.
Audit context: repository at the 2026-07-16 state (~65.8k tracked Rust lines, 22 workspace crates, 394 registry dependencies, 917 test annotations, 1,310 conformance cases). Full workspace gate (fmt, clippy -D warnings, tests, conformance) passed on the audited snapshot. Overall verdict: an unusually coherent prototype whose feature breadth has grown faster than its semantic and security invariants; the agent capability/leash system is not yet a trustworthy boundary.
A — Effect planning is fail-open (P0)🔗
crates/shoal-eval/src/plan_derive.rs does not conservatively cover all effectful language forms. Plan → approve is therefore unsound for untrusted code. Dynamic probes against the real binary confirmed each gap: all of the following returned effects: [], reversible: true, spawns: false.
plan { "x".save("p") } plan { spawn { "x".save("p") } }
plan { echo hi > p } plan { parallel(() => "x".save("p")) }
plan { env.AUDIT_ONLY = "y" } plan { run("echo", "hi") }
plan { path("Cargo.toml").read } plan { open("Cargo.toml") }
plan { use ./module } plan { save("x", "p") }- A1 —
Stmt::Usefalls through with no effects, but module loading reads a file and executes every top-level statement (modules.rs). - A2 —
Stmt::Assignplans only the RHS; persistentenv.NAME = ...is not anEnvWrite. - A3 — Command redirects are not traversed by
plan_call;>and>>can write without anFsWriteplan effect. - A4 — Method calls only special-case
http.*; path/stream.save,.append, path reads, and.feedare not classified. - A5 —
FnCallexpands only functions declared in the same submitted program. Session closures and effectful runtime functions escape static derivation. - A6 — Generic external commands become
Opaque, not a concreteProcSpawn; adapter spawns are concrete. The asymmetry hides what actually spawns. - A7 — Adapter effect parsing understands only
fs.read,fs.write,fs.delete, andnet.connect; declarations such asproc.spawn(container)are silently ignored. - A8 — Effectful builtins (
run,open,save) andspawn/parallelbodies derive no effects (see the probe list above). - A9 —
plan { "x".feed(cat) }reported onlyread /tmpandspawns: false: planning resolvescatas a Shoal builtin while runtime.feedbypasses normal command dispatch and callsrun_argvdirectly — a concrete planner/runtime resolution mismatch. - A10 — The required design is fail-closed: every AST form exhaustively classified; an unrecognized or unclassifiable form must become an approval-requiring unknown effect, never an empty effect set.
- A11 — There is no negative-coverage suite asserting that each effectful syntax route produces its effects; the probes above must become pinned tests.
B — Child evaluators lose active security/session context (P0)🔗
Fresh evaluators created by spawn, parallel, on(channel, handler), and .shl script execution copy environment/adapters/effect ports but not the active leash policy/principal, and omit various reef/config/journal/session settings. Sites: script.rs (spawn_block, run_script_file), host.rs (builtin_parallel), channels.rs (builtin_on), lib.rs (inherit_ports copies only ports).
- B1 — Child construction is manual field copying at several call sites; each new execution route must remember dozens of fields, and some do not.
- B2 — The active leash policy/principal is not propagated to children.
- B3 — Reef overrides/locks, config, journal/session settings propagate inconsistently.
- B4 — Consequences: runtime spawn-hash gating and OS sandbox selection can disappear inside child evaluation; background/parallel/handler/script work can behave differently from foreground work; a restricted command can become unrestricted when wrapped in
spawnorparallel. Release-blocking for the agent security story.
Post-freeze additions (2026-07-16, evaluator field census for the decomposition design):
- B5 —
spawn_blockalso drops the injectedconfigport: it hand-copies fs/exec/clock/opener/secrets but not config, and onlyrun_script_file(the soleinherit_portscaller) keeps it. Injected configuration is invisible insidespawn { … }. Refines B3. - B6 — Cancellation propagation is silently inconsistent across child routes:
spawn/onwire a child cancel token to the task’son_cancelhook, butrun_script_filegives its.shlchild a fresh unlinked token (parent cancellation is ignored) andbuiltin_parallelneither links a token nor installs anon_cancelhook —parallelchildren are effectively uncancellable. Relates to I5.
C — In-process filesystem effects bypass ports and sandbox (P0/P1)🔗
- C1 —
shoal-value/src/methods/path.rs::saveand stream save code usestd::fs::OpenOptionsdirectly; these effects happen in the kernel process, not the sandboxed external child, and bypass the Fs effect port. - C2 — The stated hexagonal/effect-port boundary is therefore incomplete; direct
std::fsuse in value methods has not been inventoried. - C3 — Combined with the A-findings, an in-process method can write a file without being planned, approved, port-mediated, or child-sandboxed.
D — Approval is not an independent boundary (P0/P1)🔗
- D1 —
cap.requestis callable without attachment and receives no caller attachment/principal. - D2 — It looks up a plan ref, checks the stored plan principal’s policy is not
Deny, and setsapproved = true; the MCP surface exposes this to the same agent that requested the plan — self-approval. - D3 — There is no auditable binding between requester, plan contents/hash, approver, approval scope, and the execution consuming the approval.
- D4 — Docs describe a human/supervisor in the approval channel, but code does not require a distinct authorized approver. Either that model must be implemented or the intended self-acknowledgement model documented explicitly.
E — Zero-config MCP defaults to the unrestricted local principal (P1)🔗
- E1 — MCP autostarts a kernel and obtains a token only from
SHOAL_TOKEN. With no token,session.attachassigns the same-UIDlocal-humanprincipal and the permissive policy. The socket is mode 0600 (same-user boundary), but an MCP agent normally runs as that user. - E2 — Capability isolation is opt-in rather than the zero-config default. A zero-config MCP-launched kernel should attach agents as a restricted agent principal by default.
F — journal.query bypasses documented attachment (P1)🔗
- F1 — Dispatch passes no attachment to
handle_journal_query, though documentation says every method except attach/parse/complete/cap.request requires attachment. Confirmed live: a fresh socket connection that never calledsession.attachread stored journal entries. - F2 —
limit: 0returned the full history rather than zero rows — a surprising unbounded limit edge case.
G — Session identity and attribution are inconsistent (P1)🔗
- G1 — A named kernel session creates its evaluator/journal using only the first attaching principal; later principals in the same session inherit that evaluator journal identity.
- G2 — Kernel-level exec entries use the current actor while evaluator statement-level entries may be attributed to the first principal.
- G3 — Tasks and PTYs are session-scoped, not principal-scoped; another principal attaching to the same named session can control them. Possibly intentional pair-shell sharing, but it weakens token isolation and needs an explicit model.
H — Architecture and hygiene (P1/P2)🔗
- H1 —
Evaluatoris a large mutable god-context (environment, cwd, adapters, jobs, reef, journal, security, modules, event bus, ports, config, …); manual child setup directly caused the B-findings. - H2 — Command resolution is a long hand-ordered precedence chain across bindings, builtins, adapters, externals, special commands, module functions, and command references; centralized resolution remains incomplete and shadowing interactions are high-risk.
- H3 — The kernel uses thread-per-connection/task/subscription with a per-session evaluator mutex; long evals serialize a session and unbounded connections/tasks can exhaust resources.
- H4 — Many daemon locks use
.lock().unwrap(); a panic can poison shared locks and cascade failures. - H5 — Sessions retain transcript values, tasks, and plans with little or no garbage collection; long-lived agent sessions grow indefinitely.
- H6 — The newline frame size limit is checked only after
read_lineallocates the full line (shoal-proto, MCP), so the 16 MiB cap does not prevent allocation-based resource exhaustion. - H7 — Root workspace lint policy is staged but member crates do not inherit it; manifest lint policy is not actually active per-crate.
- H8 — Unix assumptions are pervasive; CI/release cover Linux/macOS only. Windows is not supported (and should be stated as out of scope, not left implicit).
- H9 — ~394 registry dependencies: large supply-chain, compile-time, binary, and maintenance surface, with no advisory audit in CI.
- H10 — Wasmtime/Cranelift lives in the separate
shoal-wasmcrate (good for shipped binary scope), but--workspace --all-targetsCI still pays its compile/storage cost for an integration not wired into language dispatch. - H11 — Stable Rust is unpinned; compiler drift can break builds.
- H12 — Fuzz targets exist but nothing ever runs them on a schedule.
- H13 — Five syntax-highlighter tests assume color is enabled and fail under
NO_COLOR=1; the product honorsNO_COLORcorrectly — a test-isolation defect.
I — Semantic stubs and docs-reality gaps (P2)🔗
- I1 — The vision says one kernel / three surfaces, but the shipped human REPL owns an embedded evaluator and never attaches to
shoal-kernel(acknowledged in code comments). Human–agent pair-shelling is not the default real architecture yet. - I2 —
StreamVal::buffer(n)is an identity function despite docs implying a bounded decoupling buffer (stream/mod.rs). - I3 —
flat_mapsequentially drains substreams rather than interleaving as documented. - I4 — In-language channel subscribers use unbounded
std::sync::mpsc::channeldespite bounded/backpressure language in docs (the channel ring is bounded; live subscriber queues are not). - I5 — Cancelling an
on(channel, handler)task cannot interrupt a blocking channelrecv; cancelled handler threads can remain stuck. - I6 — Incremental stream
.feedis explicitly unimplemented. - I7 — WASM validation/registry code exists but WASM commands are not integrated into evaluator dispatch.
- I8 — Kernel task suspend/resume returns unavailable.
- I9 — Wire stream values are labels/refs; chunk streaming is deferred.
- I10 — The LSP is basic lexical/editor assistance and should be described as such, not as a mature semantic IDE stack.
- I11 — Docs/roadmap counts and open items drift rapidly from current source.
- I12 — Benchmark coverage is thin and unenforced. The prompt benchmark’s source comment claims per-PR runs and a protected p99 contract that CI never executes; the
table_1m_where_sortbenchmark hand-filters a RustVecand never invokes Shoal table methods or the evaluator, providing no evidence for language-level million-row performance. - I13 —
distinctuses aVec: O(n) membership per item, O(n²) total, unbounded memory on an unbounded stream. - I14 —
zip/mergeare synchronous pull operators; rate-skew/backpressure claims need precise documentation and tests.
J — Secrets and token storage (P2)🔗
- J1 — The secret-store AES key is stored beside the ciphertext in the same protected directory; OS permissions are the real boundary, and copying the directory yields both key and ciphertext. This must be documented honestly or improved (OS keyring).
- J2 — Secret values are ordinary
Arc<str>and are not zeroized; environment injection creates additional copies. - J3 — Whole-map read/modify/write of the secret store has no interprocess lock; concurrent updates can be lost.
- J4 — Token hashing is reasonable (high-entropy bearer tokens), but token-store updates need the same concurrency review as J3.
Priority direction recorded by the audit🔗
- Make effect derivation exhaustive and fail-closed (A).
- One authoritative child-evaluator/context constructor (B).
- Route every filesystem/process/network effect through enforceable ports (C).
- Define the threat model; zero-config agents attach restricted (E).
- Authenticate approval; separate requester from approver (D).
- Require attachment/authorization for journal and session operations (F, G).
- Decide whether the REPL truly joins the kernel (I1).
- Quotas and lifecycle cleanup for sessions, tasks, plans, transcripts, PTYs, connections, subscribers; no allocation before frame-size enforcement (H3–H6).
- Label incomplete semantics explicitly (I2–I10).
- Add negative-coverage and cross-principal integration testing; actually run fuzzers (A11, D/E/F tests, H12).