Columnar Data, from the inside out Query engines, Explained →

Columnar / Query execution

Query engines

A query engine turns SQL into a plan and executes that plan over data. Parquet supplies stored columns; the engine decides how to scan, join, order and return their values. Two engines can agree on every result while choosing different operators. This guide compares DuckDB with Apache DataFusion, a query framework used by applications building analytical systems. Their recorded plans show where the same question leads to different work.

ParquetColumns stored as bytesIceberg · Delta LakeFiles selected by a versionArrowTyped arrays in memoryDuckDB · DataFusionOperators turn values into answers
Table metadata selects files. Readers decode stored columns into arrays. Engines compute on those arrays.