Columnar Data, from the inside out DuckDB, Explained →

Columnar / Query execution

DuckDB

DuckDB is an analytical query engine that runs inside your process, including a browser tab. Give it SQL and a Parquet file: it chooses columns, applies filters, joins inputs and aggregates values into an answer. Its optimizer turns the question into a plan; physical operators carry out that plan over batches of values. Applications in Python, R and JavaScript use DuckDB to read files and return typed results without operating a separate database server.

ParquetColumns stored as bytesIceberg · Delta LakeFiles selected by a versionArrowTyped arrays in memoryDuckDB · DataFusionOperators turn values into answers
Table metadata selects files. Readers decode stored columns into arrays. Engines compute on those arrays.