Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

RFD 0065 — The runtime⟷engine seam: oxc-value + the QueryEngine facade

  • State: accepted
  • Depends on: RFD 0020 (the unified runtime data engine — the data system and the inference engine are the same system, the invariant this seam preserves), RFD 0021 (the as-built reasoner engine — Engine::evaluate, the CompiledRule executed form, the RelationCatalog public-API seam this facade extends), RFD 0035 (the composable operator-tree pipeline — the logical::lower → optimizer → PhysicalOp path whose live consumer this RFD records), RFD 0036 (heterogeneous stores — RFD 0035’s forcing consumer, the federation path that revived the tree), RFD 0064 (compiler source decomposition — the file-level seams this crate-level seam is orthogonal to)
  • Prior art: Parnas (1972) — information hiding: name a module by the stable contract it exports, not by the internals a client happens to reach; the leaf-crate / facade-trait idiom (a shared low vocabulary crate + a narrow trait over a thick implementation crate) as the standard Rust cut for breaking a thick crate’s accidental coupling without splitting its cohesive core; CQRS read-model framing (RFD 0020 D10) for why the value vocabulary is a shared asset, not the reasoner’s private one.

Question

A measured analysis of Argon’s crate graph found the layers cleanly separable — the DAG is acyclic, leaf hubs are thin (RFD 0064) — except one seam: reasoner ⟷ runtime, which is fused two ways, and the fusion is what blocks disjoint engine/runtime ownership (one owner evolving the inference engine, another the data-system runtime, against a stable contract rather than against each other’s internals).

The fusion has two distinct faces, both firsthand-measured against the tree:

  1. The runtime is built on the reasoner’s value model. The store, write path, and read model name oxc_reasoning::compile::Value and its cluster at 299 sites in oxc-runtime alone (Value, Tuple, the CBOR/numeric codec — encode_tuple/decode_tuple/enum_*_cbor_bytes/struct_*_cbor_bytes/rational_to_value/decimal_str_to_rational/the *_CBOR_TAG constants — and the civil/tz/wall temporal helpers, ids::InternalId). This is the shared runtime value vocabulary, but it physically lives inside the reasoner’s compile module, so the runtime cannot name a value without depending on the whole engine.

  2. The store reaches around the Engine into its internals. On the store/persist/read-model path the runtime names ~32 engine-internal symbols across logical/physical/maintainer/executor/optimizer/runtime (e.g. maintainer::{FactDelta, ProjectionMaintainer, FrozenMaintainer, IncrementalMaintainer}, physical::{PhysicalOp::Materialize, materialize_into_catalog}, executor::temporal::interval::{Interval, coalesce}, logical::lower::{evaluate_via_physical, try_evaluate_via_physical, …, pipeline_compiled_rules}, runtime::relation::Relation). The runtime couples to how the engine is built, not to a contract it exports.

The unified-system design (RFD 0020: “the data system and the inference engine are the same system”) is deliberate and must be preserved — Engine::evaluate is the single fused evaluation entry, and splitting it into a separate database product plus a separate engine product would discard the whole RFD 0020 thesis. So: what is the smallest cut that makes engine and runtime disjointly ownable without splitting the unified design — i.e., gives each side a stable contract to evolve behind, while the fused evaluation path stays one path?

Context

  • The crate DAG is already acyclic; the fusion is symbol-level, not topological. The runtime depends on the reasoner crate (correct — RFD 0020), but it reaches into the reasoner’s private surface rather than a contract. The lever is two surgical seams inside the existing dependency, not a re-crating of the engine.
  • oxc-protocol is a confirmed leaf. Its only dependencies are blake3, ciborium, semver, serde, serde_json (and optional sqlx) — zero intra-workspace oxc-* deps. A new value crate depending on oxc-protocol + numerics (num-bigint/num-rational/num-traits, ciborium, jiff) introduces no cycle.
  • The value cluster is mis-located, not engine-coupled. Value is a pure-data enum (Bool/Int/Date/Individual/Internal/Name/Null/Symbol/Cbor); Tuple, SymbolCompare, the CBOR/numeric codec, and the civil/tz/wall calendar helpers (delegating to jiff) have no dependency on join algorithms, the optimizer, or the fixpoint loop. They sit in compile for historical reasons — they were authored where they were first needed.
  • Engine::evaluate is genuinely fused and must stay so. It reorders each body (SIP), dispatches to the executor, and is the sole entry behind PhysicalOp::Recurse (the fused Datalog-fixpoint operator). This RFD does not unbundle it; it makes the runtime name it through a facade instead of reaching past it.
  • FORK-2 (the operator-tree live/dead question) had to be settled before cutting the facade, because the facade’s boundary line (what stays public, what goes private) depends on which parts of logical/physical/optimizer are on the production path. The resolution is recorded below as a first-class decision.

Decision

Two surgical seams make engine and runtime disjointly ownable while keeping the unified system one system. The runtime and the engine couple only through {oxc-value, QueryEngine}.

CUT-A — oxc-value: the shared value vocabulary as a leaf crate

Extract the value cluster out of oxc-reasoning::compile (and the crate root) into a new leaf crate oxc-value, depending only on oxc-protocol (the confirmed leaf) + numerics. It carries:

  • Value — the pure-data enum, no impl that reaches the engine; the data vocabulary, nothing else.
  • Tuple, SymbolCompare, the CBOR/numeric codec (encode_tuple/decode_tuple, the enum_*/struct_*/ref_blob_*/instant_*/plain_time_* CBOR (de)serializers, rational_to_value/rational_to_wire_string/real_wire_str_to_rational/decimal_str_to_rational, the *_CBOR_TAG constants).
  • The civil/tz/wall temporal helpers (parse_date/render_date/parse_duration/render_duration/epoch_day_to_nanos/is_temporal, render_instant, …).
  • ids::InternalId — the identifier the value layer and the engine share.

oxc-reasoning and oxc-runtime both depend on oxc-value; the reasoner re-exports the moved names at their old paths so the cut is a re-export, not a rename churn. The 299 runtime sites continue to compile; their meaning moves from “reaching into the engine” to “naming the shared vocabulary.”

CUT-B — QueryEngine: a facade trait over the engine internals

Introduce a QueryEngine facade trait (~7 methods + 1 associated type, ProjectionMaintainer) over oxc-reasoning’s logical/physical/maintainer so the runtime store names a stable API instead of engine internals. The boundary line is drawn by FORK-2 (below):

  • Moves down to the shared low crate (so both sides name them without engine coupling): Interval, Bound, coalesce, FactDelta, ProjectionMaintainer (the associated type).
  • Stays public on the facade (the contract the runtime is allowed to name): CompiledRule, Term, AtomIR, RelationCatalog (extending the RFD 0021 RelationCatalog public-API seam).
  • Becomes private to the engine (no longer nameable by the runtime): LogicalPlan::Apply and ApplyGate (the store is their sole external consumer → pub(crate)). Note: the materialize unit is physical::MaterializedRelation (there is no MaterializedLeaf type); it is shared by the federation/pipeline API and stays public — privatizing it is out of this seam’s scope. The runtime nonetheless names no engine internal on the analytical-freeze path once the fused freeze_apply (build→lower→materialize) method exists.

Engine::evaluate stays the single fused evaluation entry behind PhysicalOp::Recurse. The facade is a re-export + wrapper boundary, not a re-architecture: it renames what the runtime is allowed to see, it does not change how evaluation runs.

The established invariant

The runtime and the engine couple ONLY through {oxc-value, QueryEngine}. Any new runtime reach into a logical/physical/optimizer/executor internal is a regression against this seam; the contract is the two named surfaces, and the fused Engine::evaluate path stays one path behind them.

FORK-2 resolution — the operator tree is LIVE on the production query path

The boundary in CUT-B depends on a fact that two prior RFDs record as the opposite of what is now true, so it is settled here as a first-class decision.

The LogicalPlan/optimizer/physical operator-tree is live on the production query path, not dead scaffolding:

  • oxc-runtime’s federation path (federate_translate.rs) calls oxc_reasoning::logical::lower::{try_evaluate_via_physical, try_evaluate_with_frozen_stratified_naf, try_evaluate_with_frozen_lower_stratum}; checks.rs calls evaluate_via_physical; strict_cache.rs and the rule tests call pipeline_compiled_rules.
  • That path runs lower_rule/lower_body → OptimizerPipeline::with_default_passes() (logical/lower.rs) → PhysicalOp::{Materialize, Recurse} (physical/mod.rs), where Recurse is the fused semi-naive fixpoint and Materialize is the frozen-EDB seam.

This is exactly the inversion RFD 0021 D1 said to perform “when a consumer demands the tree”, revived by RFD 0035’s federation consumer (RFD 0036) precisely as designed. Therefore:

  • RFD 0021 D1’s “dead/reserved, not wired” prose is now STALE — the operator tree it deferred is on the production path.
  • RFD 0035’s Context (“two halves that don’t meet”; LogicalPlan “orphaned … never instantiated by any front-end”) is now STALE — its own decision has landed.

The genuinely-dead residue is narrow and was confirmed firsthand: runtime/operators.rs (the Z-set operator vocabulary, zero forward-path callers), the unproduced LogicalPlan::{Distinct, Recurse} variants (constructed only in optimizer rebuild-on-match arms, never originated by lower_body), and the docstring-only SLG/DBSP/SMTExecutor stubs. That residue is being deleted separately — it is not part of this seam, and its removal is what makes the facade’s “private” set genuinely the live tree only.

Rationale

  • Two seams, not one split. The fusion has two independent faces (a mis-located shared vocabulary; a store reaching past the engine), so it takes two independent cuts. Neither cut touches the fused evaluation path — they relocate a vocabulary and rename a surface. That is why disjoint ownership is achievable without contradicting RFD 0020.
  • oxc-value is a relocation, not an extraction of behaviour. The cluster is pure data + codec + calendar helpers with no engine dependency; it is the runtime’s value vocabulary that happens to live in the reasoner. Moving it to a leaf crate both sides depend on is the information-hiding cut: name the value by its own crate, not by the engine that first defined it.
  • QueryEngine names the contract, not the construction. The runtime needs to drive evaluation and maintain the read model; it does not need to know the operator tree exists. A ~7-method facade gives it the verbs (CompiledRule, RelationCatalog, the maintainer associated type) while LogicalPlan/ApplyGate/MaterializedLeaf become the engine’s private business — so the engine owner can refactor the tree freely behind the facade.
  • Settling FORK-2 is load-bearing for the cut. The facade’s public/private line is the live/dead line of the operator tree. Drawing it on stale “the tree is dead” prose would have wrongly privatized a live path or wrongly exposed dead residue; the firsthand measurement is what makes the boundary correct.

Alternatives considered

  • Split the unified system into separate DB + engine products. Rejected: directly contradicts RFD 0020 — the data system is the inference engine. Disjoint ownership is the goal, not disjoint products; two owners against one stable contract is the win, two products is the loss.
  • Wrap Engine::evaluate instead of cutting at the logical::lower seam. Rejected: a thin wrapper over the single evaluation entry re-exposes the very logical/physical internals the cut is meant to hide (the runtime would still name them to build the call). The facade must sit at the contract the runtime consumes, not at the fused entry it ultimately reaches.
  • Keep the fusion; rely on discipline not to reach past the engine. Rejected: the fusion is exactly what blocks disjoint ownership — the owner’s goal. Discipline does not give a second owner a contract to evolve behind; structure does.
  • Cut only oxc-value (CUT-A) and leave the store reaching into engine internals. Rejected: it fixes the value face but leaves the ~32-internal store reach, so the engine owner still cannot refactor logical/physical/maintainer without breaking the runtime. Both faces must be cut for disjoint ownership.

Consequences

  • Disjoint ownership becomes possible. The engine owner evolves logical/physical/optimizer/executor behind QueryEngine; the runtime owner evolves the store/persist/read-model path against {oxc-value, QueryEngine}. The fused Engine::evaluate path stays one path — the unified design is intact.
  • The coupling surface is now exactly two named crates/traits, and the established invariant makes any new runtime reach into an engine internal a legible regression.
  • oxc-value is a new leaf crate depending only on oxc-protocol + numerics; the DAG stays acyclic (no cycle, confirmed against the leaf check).
  • The narrow dead residue (runtime/operators.rs, unproduced LogicalPlan::{Distinct, Recurse}, the SLG/DBSP/SMT docstring stubs) is removed separately, so the facade’s “private” set is the live tree only.
  • Sequencing is fixed: CUT-A strictly before CUT-B — CUT-B’s moved types (Interval/Bound/coalesce/FactDelta/ProjectionMaintainer) land in oxc-value, the crate CUT-A creates. CUT-B cannot move them down until the low crate exists.

Invariants preserved

  • RFD 0020 unified-system design: the data system and the inference engine remain one system; Engine::evaluate stays the single fused evaluation entry.
  • RFD 0021’s RelationCatalog public-API seam — extended onto the facade, not replaced.
  • RFD 0035’s operator-tree pipeline — now recorded as live; its public/private split is the facade’s boundary line.
  • The acyclic crate DAG (RFD 0064) — oxc-value is a leaf addition, no cycle.

Open questions

  • The exact QueryEngine method set. ~7 methods + the ProjectionMaintainer associated type is the measured shape; the final signatures are settled in the implementing PR against the live call sites (the facade must cover every verb the store currently reaches an internal for, and no more).
  • Whether the read-model maintainer surface (FrozenMaintainer/IncrementalMaintainer) belongs on the facade or stays an oxc-value trait the engine implements. ProjectionMaintainer as an associated type points at the latter; the maintainer constructors the runtime names today (FrozenMaintainer::new, IncrementalMaintainer::new) are the boundary case to settle when CUT-B lands.