Durability & Storage

Choosing a backend

The backend is fixed when the database is constructed. The embedded Rust API exposes three factories, and the server builder mirrors them (server::memory(), server::sqlite(config), server::sqlite_without_buffer(config)):

// Nothing persists. Tests, ephemeral state, the browser playground.
let db = embedded::memory().build()?;

// The default durable shape: hot writes stay in memory,
// a background flusher migrates them into SQLite.
let db = embedded::sqlite(SqliteConfig::new("/var/lib/myapp/data")).build()?;

// No in-memory buffer: every commit is written to SQLite
// before it returns. Durable at commit, slower to write.
let db = embedded::sqlite_without_buffer(SqliteConfig::new("/var/lib/myapp/data")).build()?;

RQL is identical on all three. Tables, views, subscriptions, TTLs - everything behaves the same; the backend only changes where committed bytes end up and what a restart brings back.

The write path: commit now, disk later

When a transaction commits, its writes are published to the commit buffer - an in-memory tier that keeps every recent version of every key, stamped with its commit version. That publish is what makes the commit visible; no disk I/O happens on the commit path. Commit latency is memory latency.

A background flush actor wakes on an interval (5 seconds by default, the MULTI_FLUSH_INTERVAL engine setting) and runs a sweep. The sweep is gated by a watermark: the oldest commit version still needed by any active query snapshot, version lease, or change-log consumer. For every key with versions at or below that watermark, the sweep writes the newest such value into SQLite, then drops those versions from the buffer. Everything above the watermark stays resident, because someone may still read it.

• --Each sweep is applied to SQLite as one transaction. A crash mid-flush leaves the persistent file at the previous sweep, never half-written.
• --Deletes are persisted as tombstones, so a deleted row cannot resurrect from an older persisted value after a restart.
• --If persisting fails, the sweep aborts and the buffer keeps the data. The engine never drops the only copy of a committed write.
• --Flushed keys are seeded into a read cache on the way out, so reads that would otherwise fall through to SQLite are usually served from memory anyway.

The consequence of this design: the commit buffer holds the recent, multi-version working set; SQLite holds the newest flushed value per key. Old row versions are garbage collected once no active reader can need them - MVCC history is a runtime structure, not an archive on disk.

What "durable" means

Memory backend: nothing is durable. Stop the process - cleanly or not - and all data is gone. That is its contract; use it for tests and for state you can rebuild.

SQLite backend (buffered, the default): a commit is durable once the flusher has swept it to disk. A clean shutdown (stop()) makes that unconditional: it drains the change-log consumers, then runs a full flush that persists every committed write regardless of the watermark - after a clean stop, a reopen sees every commit. A crash (kill -9, power loss) is where the asynchrony shows: commits that had not been flushed yet are lost. That window is roughly the flush interval, plus whatever the watermark was holding back for long-running readers or lagging consumers. What is on disk stays consistent - each file is a SQLite database in WAL mode, so a crash never corrupts it - but the database reopens at the last flushed state, not the last commit.

SQLite without buffer: commits write straight into SQLite and are durable the moment they return. You pay for that on every write, and memory-only stores (persistent: false) are rejected at creation (CA_086) because there is no buffer tier for them to live in.

On disk: three SQLite files

Point SqliteConfig::new at a directory-style path and the engine lays out one directory with three databases (each with its own -wal / -shm companions):

/var/lib/myapp/data/
  multi.db     # row data: tables, views, ring buffers, series, operator state
  single.db    # small system state, including the commit-version clock
  cdc.db       # the change log (CDC): every committed write, in order

All three run SQLite in WAL journal mode. The default preset (SqliteConfig::new) uses synchronous = NORMAL: an OS crash or power loss can drop the last moments of WAL activity but never corrupts the files. SqliteConfig::safe switches to FULL for maximum sync strictness; SqliteConfig::fast turns syncing off for bulk or throwaway workloads. Since persistence is already asynchronous in the buffered shape, these presets mostly matter for sqlite_without_buffer, where the SQLite sync mode is the durability guarantee.

Flush cadence and WAL checkpointing are tunable through engine settings (MULTI_FLUSH_INTERVAL, MULTI_WAL_AUTOCHECKPOINT, CDC_WAL_AUTOCHECKPOINT, and the read-cache sizing knobs). See Storage & Configuration for the full reference.

What survives a restart

• --All flushed data - tables, views, ring buffers, series, the catalog, operator state, and the change log come back exactly as persisted. After a clean stop, that is everything.
• --The commit-version clock. Versions are allocated in blocks of 100,000 and the block boundary is persisted in single.db before any version from the block is handed out. On restart the clock resumes past the last persisted boundary, so commit versions are monotonic across restarts and never repeat - the version numbers may jump, but order is preserved. Snapshots, storage, and CDC all keep speaking the same version language across the restart.
• --Not MVCC history. The persistent tier stores the newest flushed value per key, not the chain of older versions. History exists in memory, bounded by the read watermark, and is gone after a restart. There is no time-travel archive on disk.
• --Not memory-only rows. Stores created with persistent: false are skipped by the flush sweep entirely - their rows never reach multi.db and reopen empty, by design. See TTL & Row Settings for when and how to declare them.

On open, the engine also verifies a persisted storage-format version, so a data directory written by an incompatible storage layout fails loudly instead of being misread.

The change log is storage too

Alongside row data, ReifyDB keeps a change-data-capture log: every committed write, converted to change entries by a background producer after commit and appended in commit-version order. It is the source that deferred views, subscriptions, and replication consume. On the SQLite backend it lives in cdc.db and survives restarts with everything else; on the memory backend it is in-memory like the rest.

By default the log is retained indefinitely. Setting the CDC_TTL_DURATION engine setting puts an age bound on it: entries older than the duration are evicted by a background scan, regardless of consumer state. Independently of retention, a background compactor packs older entries into compressed blocks (zstd) to keep cdc.db small; compaction changes the encoding, not the contents.

Scope: one node, local disk

Everything above describes a single node: durability means the local SQLite files. A default deployment commits locally only. The engine does ship two distribution subsystems - synchronous Raft consensus inside the commit path, and asynchronous CDC replication that ships the change log to replicas - but both are opt-in server configurations, and neither changes the single-node story documented here. The mechanics are covered in Transaction Internals.