Bytecode Compiler

DistributedBytecode VMQuery

Beta Works · surface still evolving · installed into the runtime's dispatch seam at daemon boot by the aetherius-orchestrator crate; the seam has no fallback compiler, so a statement it cannot compile is refused with the reason

What it is

The compiler lives in its own crate, aetherius-orchestrator, and reaches the runtime through one trait, FilterCompile, installed once at boot. The runtime never depends on the orchestrator crate; the orchestrator depends on the runtime’s VM definitions so the two cannot disagree about an opcode.

Its contract is the engine’s: zero heap allocations. The caller hands it a fixed [Instruction; 256] buffer; the compiler writes into it with a cursor and hands back the filled prefix. Registers come from a u8 stack counter. Recursion depth is the AST’s depth. A program that would not fit returns CapacityExceeded; one that would need a 256th register returns RegistersExhausted. Measured: 25.5 ns and 13 instructions for a three-predicate WHERE, with an allocation counter asserting zero.

The program shapes

Every compiled program starts with the same guard: the WHERE expression into one register, then JMP_FALSE to the end. What follows depends on the statement.

SQL shape	Program body	Reply frame
SELECT … WHERE (no projections at the seam)	YIELD_MATCH	RESULT_STREAM_CHUNK of row offsets
SELECT a, b … / SELECT *	LOAD_COL + YIELD_COL per slot, YIELD_MATCH	RESULT_MATERIALIZE_CHUNK
SELECT COUNT/SUM/MIN/MAX/AVG(…)	LOAD + ACC_* per accumulator into top registers	one terminal RESULT_AGGREGATE_CHUNK
… GROUP BY key	LOAD_COL key, then LOAD + HASH_ACC per accumulator	RESULT_GROUPED_AGG_CHUNKs
Build side of a join	LOAD_COL + YIELD_COL per key and payload column, YIELD_MATCH with WITH_OFFSETS	materialize chunks with a trailing offset column
Probe side of a join	see below	materialize chunks

AVG without an opcode

The VM does integer accumulation only; floating-point division in the row loop would cost more than it saves. AVG(x) compiles to two accumulators, ACC_SUM and ACC_COUNT over the same loaded value (two HASH_ACC slots under GROUP BY), and the orchestrator divides once per result row as f64, returning NULL when the count is zero. The worker’s ISA and executor did not change for it.

The join program

For SELECT … FROM probe JOIN build ON … [WHERE …] the probe worker receives, in order:

1. the probe-side guard;
2. LOAD_COL + HASH_MIX for every probe key column, in the build side’s mix order;
3. HASH_PROBE (or HASH_PROBE_OUTER for LEFT JOIN), miss → end (or → materialisation);
4. key verification: LOAD_BUILD + EQ + JMP_FALSE for every key column against the record, so a 64-bit hash collision can never produce a false pair;
5. the cross-side conjuncts — WHERE o.amount + c.tier > 1000 compiles with LOAD_COL for the probe column, LOAD_BUILD for the build column and ADD — then JMP_FALSE;
6. the SELECT list from both sides (LOAD_COL or LOAD_BUILD by qualifier), then YIELD_MATCH.

Every skip is a jump to the end, which the JOIN loop turns into “next match of this row’s list”. A LEFT JOIN miss jumps straight to step 6 with the build registers NULL.

Column resolution

The compiler resolves a column reference by qualifier: a name qualified with the build alias becomes LOAD_BUILD over the embedded record; an unqualified or probe-qualified name becomes LOAD_COL over the probe stripe. * expands to the probe columns followed by the embedded build columns. Nullability comes from the catalog (ColumnMeta.nullable) and sets the NULLABLE flag on the load — the flag the worker refuses to do without on a packed stripe.

Expressions

=, !=, <, <=, >, >=, AND (short-circuit via JMP_FALSE), OR (via JMP_TRUE), NOT, +, -, *, /, integer literals that fit 32 bits, and column references. Anything else — string literals, functions, subqueries, non-integer arithmetic — returns Unsupported and the statement is refused at the seam with that word in the message.

What the seam does with it

try_ship in the remote dispatcher compiles a routed single-table SELECT (with or without WHERE, with an optional single-column GROUP BY) and ships it. Two-table SELECT … JOIN … ON … statements enter the two-stage join pipeline. Everything else falls through to the existing refusal, which names the target node and the reason.

Tests that pin it

• Zero-allocation compile under a counting global allocator.
• Every operator against a Rust oracle over a real .acf.
• A mutation that removes a short-circuit jump turns a test red.
• A regression that once emitted YIELD_MATCH twice (every projected row came back doubled) is pinned by the projection test.

Tiny VM ISA →

What each emitted opcode does at run time.

Pushdown Joins & Aggregates →

The pipeline that drives the build and probe programs.

EXPLAIN →

How the single-node planner exposes its choices.