How the systems underneath actually work

Each topic starts with a plain description of what the thing is and what it is for, then goes deep, with a worked example carried end to end and an animation where movement explains it better than words.

Ready to read

01 The query engine From SQL text to an execution plan to bytes read off disk. Eight stages, one query followed all the way down — the same query that takes 48 seconds and then 0.9 seconds — and where the performance actually comes from. Parsing Logical plans Optimization Physical operators Indexing Vectorized execution Columnar pushdown Distributed Read →

Planned next

02

Columnar file formats

Parquet and Arrow: row groups, encodings, footers, and why the in-memory and on-disk formats are different problems.

03

DataFrame engines

Polars, DuckDB and Spark side by side: lazy frames, what each one optimizes, and when to reach for which.

04

Indexing and data structures

B-trees, LSM trees and bitmap indexes — what each is shaped for, and why analytics often skips indexes entirely.

05

Memory management and spilling

Operator memory budgets, back-pressure, and what happens the moment a query no longer fits in RAM.