Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

RFD 0052 — Deployment topologies, the connection abstraction, and host-language parity

  • State: discussion

Question

Argon is a language backed by a database. A model is rarely a standalone artifact — it is embedded into systems written in general-purpose languages, or deployed as its own service those systems call. How is Argon deployed and consumed across three independent choices — where the database runs (embedded in a host process vs. a standalone process), how a host reaches it (a native call, a wire protocol, or a generated SDK over either), and which host language drives it (Rust and TypeScript, co-equal) — and where exactly does the embedded/standalone line fall for concurrency?

This RFD does not define the serving API (RFD 0014), the in-process runtime contract (RFD 0020), persistence swap and connectors (RFD 0036), or the derivation surfaces (RFD 0046). It sits above them and fixes the consumption model they compose into.

Context

Most databases ship an object-relational mapper because there are two models that do not agree: the store’s data model and query language on one side, the host’s object model on the other. The ORM is the impedance-matching middleman — leaky, drift-prone, two sources of truth kept in sync by hand.

Argon has one model. Schema, type system, constraints, queries, and reasoning are all declared in Argon under a single type discipline. So a host-language binding is not a mapping between two models; it is a faithful typed projection of the one model into the host language, drift-gated the way the Lean↔Rust interface is. The generated client is thin and total: it exposes the model’s declared query / mutate / derive / compute surface (and the ad-hoc surface of RFD 0033) as typed host functions, and the host types are the schema. The schema-migration drift an ORM fights does not arise — the host’s types are regenerated from the model and gated on change.

Two facts follow and frame everything below:

  • The generated SDK is one access mechanism, not the data model. It must be orthogonal to where the database runs.
  • Argon must be deployable as a database, not only embeddable as a library. Embedded (linked into a host process), sidecar (a process beside the host), and standalone (its own networked, durable service) are all first-class.

Consumers include ordinary Rust and TypeScript services, the Tide TypeScript workflow runtime, and — by construction — a future Rust durable-execution library. None is privileged; each reaches Argon through the same surface.

Decision

D1 — Two orthogonal axes: topology × access

TopologyWhere it runsAccessAnalogue
Embeddedruntime linked into the host process; local storagenative call (Rust crate / TS napi) or SDK over an embedded handleSQLite, DuckDB
SidecarArgon process beside the host on one nodewire (/v1) or SDK over a local clientlocal Postgres
StandaloneArgon as its own durable, networked servicewire or SDK over a remote clientmanaged Postgres

The generated SDK sits on top of the access column: it wraps the native handle in the embedded case and the wire client in the sidecar/standalone cases. A host writes to the SDK and chooses a topology at deploy time by which handle it constructs.

D2 — One connection abstraction

There is a single Connection abstraction whose surface is the OxbinRuntime semantics of RFD 0020, with two implementations: Embedded (links the runtime, local storage) and Remote (a /v1 client, capability-gated). ox gen emits code written against Connection, so host code — model::queries::all_staff(conn) — is identical whether conn is in-process or a network client. This one indirection is what makes “use Argon any of these ways” a fact rather than a slogan.

D3 — Host-language parity

ox gen --target rust and ox gen --target ts are co-equal deliverables. Both project the same model from the same source, expose equivalent typed surfaces, and are both in-process-capable and remote-capable. Both must emit the model’s rich result shapesTruth4, standpoint-tagged, set-valued, ordinal results — idiomatically (a Rust enum; a TS discriminated union), never a scalarized flattening. In-process mechanics differ by host: Rust links the runtime crate; TypeScript uses a napi bridge (D7); HTTP is the universal floor for both.

D4 — The concurrency contract is set by topology

This is the line between an embedded library and a production database, and it is deliberate:

  • Embedded ⇒ single-process ownership. Durable embedded storage is owned by exactly one process: in-memory for tests, a single-owner file backend otherwise. There is one writer. Embedding does not promise multi-process shared-file access; a host that needs concurrent multi-writer access has, by that need, chosen the sidecar or standalone topology.
  • Standalone ⇒ the serve layer is the single logical writer, and must handle concurrent multi-caller load gracefully and performantly. A standalone Argon is a world-class production database server: connection handling and admission control, snapshot-isolated concurrent reads over the bitemporal log, serialized promotion of writes (one logical writer; no silent merge), backpressure, and fair scheduling. The detailed design of this layer is the significant engineering effort this RFD opens (see Consequences); this RFD fixes the contract, not the mechanism.

D5 — Access is provided; durability layers on top

Argon provides access: the connection, the generated SDK, and the serve layer. A workflow runtime provides durability: journaling, deterministic replay, and fork-scoped writes, layered over a Connection. Determinism is the workflow wrapper’s concern — pin a transaction-time read point, journal the reads, scope writes to a fork — and works over any connection regardless of topology. Argon stays workflow-agnostic; Tide is the TypeScript implementation of this wrapper; a Rust durable-execution library would be another. Workflow semantics are never baked into the runtime.

D6 — Semantic transparency across transports

Embedded and remote differ only in latency and in the capability boundary — never in expressivity or in the shape of what comes back. as_of(vt, tt) means the same; rich results survive the wire (CBOR) identically to in-process. The /v1 protocol and the in-process trait are two encodings of one semantics. Capabilities are enforced at the network edge — generic writes denied, only the declared (and ad-hoc-permitted) surface dispatches — and are not imposed on the in-process, in-trust embedded caller. The SDK surface is identical across topologies; the remote path additionally enforces capabilities.

D7 — Tide extraction and the JS bridge

Tide is extracted to its own repository, structured like Argon, with its orca-mvp couplings (world-state, kernel-storage) severed to traits so its core depends on no orca-mvp component and runs non-ontology workflows. The dependency direction is tide → argon; the Argon binding lives on the Tide side; Argon remains ignorant of workflows. The JavaScript↔runtime in-process bridge (napi, later optionally WASM) is an Argon component — it is about embedding Argon in JavaScript, not about workflows — and Tide reuses it for its ops, so standalone Bun/Node services get the same in-process embedding a Tide workflow gets.

Rationale

The orthogonality in D1/D2 is the whole point of removing the ORM. Because there is one model, the typed projection is faithful and thin, so it can be a facade over any transport without re-introducing a second model. Collapsing topology and access — making the SDK mean “remote” or making “embedded” mean “no SDK” — would rebuild the middleman it eliminated.

D3’s parity is a requirement, not a courtesy: Argon is consumed at least as much from Rust as from TypeScript, and today only the TypeScript projection exists (because Tide needed it). Rust consumption is currently “link the runtime and hand-wire it,” which is below parity. The rich-shape clause is where the no-silent-scalarization discipline lives — a contested or set-valued result that the embedded path returns intact and the remote path or the codegen flattens would make D2/D6 a lie.

D4 is the production hinge. Embedded single-ownership keeps the SQLite/DuckDB contract honest and cheap. Standalone multi-caller concurrency is the price of being a real database, and it is where the hard engineering is; naming it as a contract now prevents an embedded-shaped design from being quietly assumed to scale to a server.

D5 keeps the runtime clean and reusable. Journaling and replay are properties of orchestration, not of data access; pushing them into Argon would couple the database to one workflow system and bar the plain (non-durable) Rust or TS consumer. The determinism design composes with D6: a tx-pinned as_of read is reproducible whether the connection is embedded or remote.

D7’s direction is the standard foundation rule: a foundation does not depend on its consumers. Locating the napi bridge in Argon, not Tide, follows from what it is — Argon-in-JavaScript — and lets one bridge serve both standalone JS services and Tide ops.

Alternatives

  • A meta-build-system wrapping Cargo and npm/Bun. Rejected. Integration follows the protobuf/protoc precedent: the model is the neutral source (like a .proto), ox gen is the generator, and the host’s native build drives codegen (build.rs for Rust; a prepare step for TS). The moment the toolchain owns the host build it becomes a framework and stops being portable — the ODE failure mode.
  • SDK-only; no native embedding. Rejected. Argon is a database; embedded in-process operation (SQLite/DuckDB-shaped) is a primary topology, not a remote-only convenience.
  • Multi-writer embedded shared-file access. Rejected for v1. Concurrency control across processes sharing one on-disk store is the standalone topology’s problem; embedded promises single-owner.
  • A dedicated binary wire protocol now. Deferred. /v1 HTTP is the one wire protocol initially; a binary/streaming protocol can later sit behind the same Remote handle without changing host code.
  • A third storage time-axis for law enactment/effective dates. Rejected as a storage concern. The store stays two-axis (valid-time, transaction-time); rule version and effective date are ordinary bitemporal facts interpreted by the reasoning layer (consistent with RFD 0036’s “richer temporal structure as payload”).

Consequences

  • ox gen --target rust is a new first-class deliverable, co-equal with the TypeScript target: typed concept types, typed query/mutate/derive wrappers over Connection, the CBOR codec, and a generated /v1 client.
  • A JavaScript↔runtime napi bridge becomes an Argon component, shared with Tide’s ops.
  • The standalone serve concurrency layer requires real engineering — admission control, snapshot-isolated concurrent reads, serialized promotion, backpressure, scheduling — and is the next deep design effort this RFD opens, deferred behind the portable-substrate phases and tracked in issue #978. It extends RFD 0014’s surface with a concurrency-and-load contract.
  • Tide is extracted to its own repository with orca-mvp couplings reduced to traits; the Argon binding and ox-tide plugin shim live Tide-side.
  • ox grows cargo-style plugin discovery (ox <name>ox-<name> on PATH), so ox tide run is convenience and tide run standalone always works.
  • Host integration follows the native build of each language; ox.toml remains the consumer-agnostic model manifest and gains no host-codegen configuration (that lives host-side).

Open questions

  • TypeScript in-process: napi-first or WASM-first? Leaning napi-first (Bun and Node both support it; shares code with Tide ops), WASM later for edge/browser. HTTP is the floor regardless.
  • ox gen --target rust: build.rs/OUT_DIR or a published crate? Leaning build.rs (the prost/tonic model, no per-version crate churn), with “emit a crate” as a flag when several hosts share one model.
  • One wire protocol or two? /v1 HTTP now; a binary protocol only if the hot path demands it, behind the same Remote handle.
  • Embedded durable concurrency: what does single-owner mean concretely for the file backend — advisory lock, lockfile, exclusive open?
  • Standalone concurrency model specifics: how reads achieve snapshot isolation over the bitemporal log; how promotion serializes against concurrent readers; the admission/backpressure policy under load.
  • Does oxup distribute Tide as a component (matching-version pinning, the rustup-toolchain model), or does Tide ship a standalone installer first?