Query Pushdown
What it is
Section titled “What it is”When your data sits on more than one machine, there are two ways to answer a question about it. The usual way is to pull the rows to wherever the query is running and work on them there. Query Pushdown does the opposite: it sends the question to each machine and lets that machine answer for the part it holds.
For a query that counts, totals, or finds a smallest or largest value, what comes back from each machine is just the result — not the rows behind it.
Why it matters
Section titled “Why it matters”The saving is in what never moves.
- The answer is small, and stays small. Summarising two million rows sends back the same handful of bytes as summarising a hundred. The reply doesn’t grow as your tables do.
- Rows only travel when you asked for rows. A query that asks for a total gets a total back. The underlying rows stay where they live.
- Every machine works at the same time. Each one is reading its own local data, so the work spreads out rather than queueing behind a single machine pulling everything in.
For comparison: summarising a two-million-row block the old way meant moving several megabytes across the network to produce one number. Pushed down, that same query’s reply is a few bytes — and it would still be a few bytes if the block were ten times larger.
What you should know
Section titled “What you should know”Your results don’t change. Pushdown affects how quickly an answer comes back and how much moves across the network. It never changes the answer. If any part of a query can’t be sent to the machine holding the data, that part is handled the ordinary way and the result is identical — just less fast.
A few behaviours are worth knowing in advance:
- Some conditions can’t be sent. Text pattern matching,
ORconditions and conditions built from functions stay behind. When that happens to a query that filters, the extra rows come back and are filtered where the query is running. When it happens to a query that summarises, the summary isn’t pushed at all — because a total added up over the wrong set of rows can’t be corrected afterwards. Either way you get the correct answer. - Whole-number columns first. Conditions and summaries that travel are currently those over whole-number columns and timestamps. Text and decimal columns are handled the ordinary way.
- Nothing matched means empty, not zero. A total, smallest or largest value
over no matching rows comes back as no value (SQL
NULL), the same as it would on a single machine. A count in that situation is0. - A total too large to represent returns nothing rather than something wrong. If adding up a column would exceed the range a whole number can hold, the query reports no answer instead of a silently incorrect one. This is deliberate: a wrong total is harder to notice than a missing one.
- Averages stay correct however the data is split. An average is worked out from the totals and the row counts, so it doesn’t matter whether the rows are spread evenly across machines or piled onto one.
- Adding or removing a machine moves some data, not all of it. Resizing the cluster relocates roughly a one-in-N share rather than reshuffling everything. Expect a rebalance, not a rebuild.
- The cross-machine path needs Linux. Machines that exchange work this way run on Linux. Single-machine use is unaffected on every supported platform.
Maturity
Section titled “Maturity”| Capability | Maturity |
|---|---|
| Sending a condition to the machine holding the data | Beta — compiled to bytecode, executed in the worker VM, streamed back in fixed chunks |
| Counting, totalling, smallest and largest, answered remotely | Beta — one 80-byte frame per query |
| Averages combined across machines | Beta — SUM and COUNT in the worker, divided at the orchestrator |
| Grouped aggregates | Beta — one key column, up to four accumulators |
| Two-table equi-joins across clusters | Beta — INNER and LEFT, composite keys, cross-side conditions |
| Text and decimal columns | Roadmap — integer columns only in the VM today |
The path is on under AETHERIUS_CLUSTER_MODE=1 with a topology file, and every
stage of it is observable through counters. The detailed design, the exact
scope and the measurements are on the
Cluster Execution Engine pages.