Wire Formats

One protocol, three encodings

A request carries RQL plus optional parameters and is tagged as a query (read), a command (write), or an admin statement (schema and identities, admin listeners only). The result is one or more frames - columnar tables, one per statement that produces output. What varies is only how those frames are encoded on the way back:

The encoding is chosen per request - a ?format= query parameter on HTTP, a format field on WebSocket - and never changes what a statement means or returns. When no format is given, the server defaults to frames. The official clients pick one per connection: the TypeScript client's format option defaults to 'frames', and the Rust client's WireFormat::Json (its default) also asks for frames on the wire - WireFormat::Rbcf is the binary alternative in both.

Rule of thumb: frames when you want types and column order preserved (it is what the client libraries decode), json when a human or a JSON-native tool reads the output, rbcf when results are large and decode cost matters.

Statements and parameters

A statement request has two fields: rql (the statement text, possibly several statements separated by ; - each result arrives as its own frame) and an optional params. Parameters travel as typed pairs, and the value is always a JSON string, whatever the type:

{
  "rql": "FROM app::users FILTER age >= $min AND name == $name",
  "params": {
    "min":  { "type": "Int4", "value": "18" },
    "name": { "type": "Utf8", "value": "alice" }
  }
}

Named parameters are an object keyed by name (referenced as $name); positional parameters are an array of the same pairs (referenced as $1, $2, ...). The accepted type names are exactly the engine's value types: None, Boolean, Float4, Float8, Int1 through Int16, Uint1 through Uint16, Utf8, Uuid4, Uuid7, Date, DateTime, Time, Duration, Blob (hex string), Decimal, and IdentityId. An unknown type name or an unparseable value is rejected with INVALID_PARAMS before anything executes.

This is what the client libraries produce under the hood: the TypeScript encoder maps a JS number to the narrowest fitting Int type, a string to Utf8 (UUID-shaped strings to Uuid4/Uuid7), null to None, and so on. A hand-rolled client just writes the pairs directly - the parameter's declared type carries through to the result, so MAP { answer: $n } with an Int4 parameter returns an Int4 column.

HTTP endpoints

The HTTP listener exposes a small, fixed surface. All statement endpoints take a JSON body and the optional format and unwrap query parameters:

Identity rides in the Authorization: Bearer <token> header. No header means the request runs as the anonymous identity - what anonymous may do is a policy decision, not a transport one. The simplest possible exchange, with the default frames encoding:

curl -s http://localhost:8090/v1/query \
  -H 'Content-Type: application/json' \
  -d '{"rql": "MAP { answer: $n }", "params": {"n": {"type": "Int4", "value": "42"}}}'

{"frames":[{"row_numbers":[],"created_at":[],"updated_at":[],
  "columns":[{"name":"answer","type":"Int4","payload":["42"]}]}]}

The same request with ?format=json returns the row shape instead - one array of row objects per frame:

curl -s 'http://localhost:8090/v1/query?format=json' \
  -H 'Content-Type: application/json' \
  -d '{"rql": "MAP { answer: $n }", "params": {"n": {"type": "Int4", "value": "42"}}}'

# [[{"answer":"42"}]]

Every statement response also carries two metadata headers: x-fingerprint (a stable hex fingerprint of the normalized request, useful for correlating logs) and x-duration (total execution time, e.g. 412us).

Logging in is a plain POST. Password users send their name as identifier; token users send method: "token" with a token credential. Creating users and credentials is covered on Connect:

curl -s http://localhost:8090/v1/authenticate \
  -H 'Content-Type: application/json' \
  -d '{"method": "password", "credentials": {"identifier": "alice", "password": "alice-pass"}}'

# {"status":"authenticated","token":"<session-token>","identity":"<identity-id>"}

curl -s 'http://localhost:8090/v1/query?format=json' \
  -H 'Authorization: Bearer <session-token>' \
  -H 'Content-Type: application/json' \
  -d '{"rql": "FROM app::users"}'

A failed login answers 401 with {"status":"failed","reason":"..."}; multi-step methods answer {"status":"challenge","challenge_id":...,"payload":...} and expect a second /v1/authenticate call that includes the challenge_id in the credentials. POST /v1/logout with the bearer header revokes the token server-side and returns {"status":"ok"}.

Errors come in two shapes. Engine errors - bad RQL, type errors, policy denials - return 400 (or 403 for POLICY_* codes) with a full diagnostic: {"diagnostic": {"code", "message", "rql", "fragment", "label", "help", "notes", ...}}. Transport-level failures return {"error": "...", "code": "..."} with the matching status: AUTH_REQUIRED/INVALID_TOKEN/TOKEN_EXPIRED (401), BAD_REQUEST/INVALID_PARAMS (400), FORBIDDEN (403), NOT_FOUND (404), METHOD_NOT_ALLOWED (405), QUERY_TIMEOUT (504), INTERNAL_ERROR (500).

The frames encoding

frames is the columnar result as JSON, and the shape the official clients decode. Each frame has three optional metadata arrays and a list of columns; each column carries its name, its engine type, and its cells as strings:

{
  "frames": [
    {
      "row_numbers": [5, 4, 3],
      "created_at": ["1970-01-01T00:00:00.000000000Z", "...", "..."],
      "updated_at": ["1970-01-01T00:00:00.000000000Z", "...", "..."],
      "columns": [
        { "name": "id",     "type": "Int2", "payload": ["5", "4", "3"] },
        { "name": "name",   "type": "Utf8", "payload": ["eve", "dan", "cid"] },
        { "name": "active", "type": { "Option": "Boolean" },
          "payload": ["true", "false", "⟪none⟫"] }
      ]
    }
  ]
}

The rules, straight from the encoder:

• --row_numbers, created_at, and updated_at are per-row metadata. Results that read stored rows carry them; computed results (a bare MAP) leave them as empty arrays.
• --type is the engine value type, PascalCase: Boolean, Int4, Utf8, DateTime, and so on - the same names parameters use. A nullable column wraps its type as an object, {"Option": "Boolean"}.
• --Every cell in payload is a string: the value's canonical text rendering. Missing values are the sentinel string ⟪none⟫ (U+27EA, "none", U+27EB) - a decoder must treat that exact string as absent, not as text. Blobs are rendered as hex.
• --Column order is the statement's output order, and all payload arrays have the same length as row_numbers would (the frame's row count). Pivoting to rows is a simple zip across columns.

On HTTP the body is the {"frames": [...]} object shown above; on WebSocket the same object appears as the response's body field.

The json encoding

json trades type fidelity for readability: the result is an array of frames, each frame an array of row objects. It is the format to reach for with curl, jq, or any consumer that just wants values keyed by column name. Its rendering rules differ from frames in ways that matter to a consumer:

• --Booleans are JSON booleans and missing values are JSON null, but numbers are JSON strings ({"answer": "42"}): Int16/Uint16/Decimal exceed what JSON numbers can represent, so everything numeric is stringified uniformly. Non-finite floats become null.
• --Keys within a row object are sorted alphabetically, not in column order, and row metadata (row numbers, timestamps) is not included. When order or metadata matters, use frames.
• --unwrap=true (query parameter on HTTP, request field on WebSocket) strips the nesting when the result is exactly one frame with one row: [[{"answer":"42"}]] becomes {"answer":"42"}. Handy for endpoints that return a single object.

One special case: if the first result frame contains a column named body, the response is built from that column alone - string values are embedded as raw, pre-rendered JSON rather than quoted. This exists for procedures that construct a JSON payload themselves and want it returned verbatim from an HTTP binding. Avoid naming an ordinary result column body when requesting json.

RBCF, the binary encoding

RBCF ("ReifyDB binary columnar format", application/vnd.reifydb.rbcf) carries the same frames as compact binary. Requests are unchanged - only the response bytes differ. On HTTP the body is one RBCF message; on WebSocket it arrives inside a small binary envelope (next section). All integers are little-endian. A message is a 16-byte header followed by frames back to back:

Message header (16 bytes)
  offset 0   u32  magic        0x46434252 (bytes on the wire: "RBCF")
  offset 4   u16  version      1
  offset 6   u16  reserved     0
  offset 8   u32  frame count
  offset 12  u32  total message size in bytes

Frame (repeated, frame count times)
  offset 0   u32  row count
  offset 4   u16  column count
  offset 6   u8   meta flags   bit 0 row numbers, bit 1 created_at, bit 2 updated_at
  offset 7   u8   reserved
  offset 8   u32  frame size in bytes (including this header)
  then, for each meta flag set, one array of row-count values:
    row numbers as u64, created_at / updated_at as i64 nanoseconds
  then column count columns back to back

Each column is a 28-byte descriptor, a padded name, and up to four data sections:

Column descriptor (28 bytes)
  offset 0   u8   type code    engine value type (see below)
  offset 1   u8   encoding     0 plain, 1 dict, 2 rle, 3 delta, 4 bitpack, 5 delta-rle
  offset 2   u8   flags        bit 0: none-bitmap present; bits 4-5: dict index width
  offset 3   u8   reserved
  offset 4   u16  name length in bytes
  offset 6   u16  reserved
  offset 8   u32  row count
  offset 12  u32  nones section length
  offset 16  u32  data section length
  offset 20  u32  offsets section length
  offset 24  u32  extra section length

followed by:
  name bytes (UTF-8), zero-padded to a 4-byte boundary
  nones    - bitmap of missing values (present iff flags bit 0)
  data     - the encoded values
  offsets  - end offsets for variable-length values (utf8, blob, ...)
  extra    - encoding-specific side data (e.g. the dictionary)

The server chooses an encoding per column by heuristics (sorted integers get delta, repetitive strings get a dictionary, and so on), so a decoder must handle all of them; an encoder may always emit plain. Type codes are pinned by the golden fixture crates/codec/golden/tag/kinds.json in the engine repository (None 0, Boolean 1, ..., Utf8 9, ..., Blob 22, ...), and crates/codec/golden/frames/ holds byte-exact sample messages to validate a decoder against.

Two reference implementations exist: the Rust codec at crates/codec/src/frame and the TypeScript port at pkg/typescript/client/src/rbcf, both in the engine repository, both tested against the same golden vectors. Clients opt in with WireFormat::Rbcf (Rust) or format: 'rbcf' (TypeScript).

The WebSocket protocol

WebSocket wraps the same requests in a persistent, multiplexed connection and adds subscriptions. Every client-to-server message is a text frame containing one JSON object with a client-chosen id (echoed in the matching response, so requests may overlap), a type, and a payload. Binary frames from the client are rejected.

{
  "id": "1",
  "type": "Query",
  "payload": {
    "rql": "MAP { answer: $n }",
    "params": { "n": { "type": "Int4", "value": "42" } },
    "format": "frames"
  }
}

Request types are Auth, Query, Command, Admin (admin listener 127.0.0.1:9091 only; on the application listener it answers NOT_FOUND), Subscribe, Unsubscribe, BatchSubscribe, BatchUnsubscribe, Call (invoke a bound procedure by name, named params only), and Logout (no payload). Query, Command, and Admin payloads take rql, params, format, and unwrap with the same semantics as HTTP. A statement response mirrors the request type and wraps the encoded result:

{
  "id": "1",
  "type": "Query",
  "payload": {
    "content_type": "application/vnd.reifydb.frames",
    "body": { "frames": [ { "row_numbers": [], "created_at": [], "updated_at": [],
      "columns": [ { "name": "answer", "type": "Int4", "payload": ["42"] } ] } ] },
    "meta": { "fingerprint": "9f3ac2...", "duration": "412us" }
  }
}

With format: "json" the body is the row shape and content_type is application/vnd.reifydb.json. With format: "rbcf" the response is not text at all: it arrives as a binary WebSocket frame laid out as [u8 kind][u32 id-length][id][u32 meta-length][meta JSON][RBCF message], all lengths little-endian, where kind is 0x00 for a statement response, 0x01 for a subscription change, and 0x02 for a batch change.

A fresh connection runs as anonymous - authentication is optional, not a handshake. To log in, send Auth with either a token ({"token": "..."}, acknowledged with an empty payload) or credentials, which returns the session token like HTTP does:

{"id": "1", "type": "Auth", "payload": {"method": "password",
  "credentials": {"identifier": "alice", "password": "alice-pass"}}}

{"id": "1", "type": "Auth", "payload": {"status": "authenticated",
  "token": "<session-token>", "identity": "<identity-id>"}}

Errors are a single shape: {"id", "type": "Err", "payload": {"diagnostic": {...}}} with the same diagnostic object HTTP returns. Engine errors carry their full code, message, fragment, and help text; transport failures use codes like PARSE_ERROR, AUTH_FAILED, AUTH_REQUIRED, INVALID_PARAMS, NOT_FOUND, QUERY_TIMEOUT, and INTERNAL_ERROR in the diagnostic's code field.

Subscriptions on the wire

Subscribe registers a live query; the server acknowledges with an id and then pushes changes as they happen. Pushed messages carry no id - they are server-initiated and correlated by subscription_id instead:

{"id": "2", "type": "Subscribe", "payload": {"rql": "FROM app::orders", "format": "frames"}}

{"id": "2", "type": "Subscribed", "payload": {"subscription_id": "12"}}

{"type": "Change", "payload": {"subscription_id": "12",
  "content_type": "application/vnd.reifydb.frames",
  "body": {"frames": [ ... ]}}}

Change frames contain the affected rows plus one synthetic column, _op, coding the change kind: 1 insert, 2 update, 3 remove. The official clients read and strip that column before handing rows to your callbacks - a hand-rolled client should do the same. Unsubscribe with the subscription_id stops the stream and is acknowledged with Unsubscribed.

Batch subscriptions group several queries so their changes arrive atomically: BatchSubscribe with {"queries": ["...", "..."], "format": ...} answers BatchSubscribed with a batch_id and one subscription_id per query, pushes arrive as BatchChange (a list of per-member entries), and the server signals BatchMemberClosed / BatchClosed when members or the whole batch end. With rbcf, batch pushes use the 0x02 binary envelope: [u8 0x02][u32 batch-id-length][batch id][u32 entry count] followed by [u32 sub-id-length][sub id][u32 rbcf-length][RBCF message] per entry.

What about gRPC?

A gRPC subsystem (protobuf-encoded frames, application/vnd.reifydb.proto) exists behind feature flags in the engine repository, but the stock server binary wires up only HTTP and WebSocket - it is relevant only if you assemble your own server with the gRPC subsystem enabled, as noted on Rust (Client).