Columnar Data, from the inside out Compare the layers →

Columnar / Changelog

What's changed.

Find a mechanism and follow its evidence · September 2026

All 99 lessons now have searchable summaries, useful prerequisites and links to companion evidence. Search concept aliases, filter by topic or reading level, and download the complete index. Parser checks now reject malformed page counts and truncated metadata more consistently, with bounded nesting and broader deterministic mutation coverage. Find a concept →

Browse schemas and revisit SQL · September 2026

Inspect each selected file’s SQL column names and types, then insert a quoted table or column at the editor cursor. Optional history keeps the newest successful queries on this device and lets you restore, remove or clear them. History starts off; restoring a query fills the editor without running it. Query local files →

Keep an inspection connected · September 2026

Follow a selected chunk or physical page between anatomy and the explorer. Row-group choices carry across Data, Records, Encoding and Bloom inspection; nested records link to their leaf-column explanations. Returning preserves selection and source provenance, while replacing a file clears the session. Parser error codes remain visible, and preloading the guide no longer moves explorer controls. Inspect a file →

Lance pages and index lifecycle · September 2026

Follow an application ID through native append, deletion, compaction and index maintenance. Compare multiple-page nullable string layouts, then decode the selected ID from original protobuf-described bytes. Follow changing addresses → · Inspect physical pages →

Local file diagnostics · September 2026

The explorer reports exact row/chunk distributions, statistics availability and trust, codec support and cumulative request overlap from an already-open footer. Inspect the source metadata and download a versioned report with its scope and limitations. Inspect a local file →

One history in Iceberg and Delta · September 2026

Follow identical native appends, updates, deletion, compaction and a column rename. Compare metadata dependencies, active files and exact rows, then return to the retained historical answer. Compare the histories →

Challenge optimizer estimates · September 2026

Twelve native runs compare missing and refreshed statistics, correlated and skewed keys, casts and join order. Inspect exact independently checked answers beside the original estimates and intermediate row counts. Explore the counterexamples →

Follow the native evidence · September 2026

Search the original artifacts, recorded versions and reproduction recipes behind the native examples. Each family states its limits and links back to its lesson. Shared loaders verify bounded downloads against recorded byte sizes and SHA-256 digests. Explore the evidence →

Inspect local table metadata · September 2026

Supply Iceberg metadata JSON or Delta logs with companion files, inspect original text and resolve exact paths with an explicit mapping. Missing references and future features stay visible. No external paths are fetched, and imported table rows are not reconstructed. Inspect metadata →

Return to a scenario and measure query memory · September 2026

Arrow and engine scenario links restore their authored controls. Optional local reading progress can be inspected and cleared. Ten native query runs compare memory settings, spill, aggregation and sorting against complete independently checked answers, including a recorded out-of-memory result. Inspect native profiles →

Iceberg writers, cleanup and v3 bytes · September 2026

Eight native experiments now expose rejected stale rewrites, compatible appends, composite/null keys, global deletes across partition specs, delete pruning and compaction, branch-protected expiration and physical orphan cleanup. Decode two original Puffin vectors to see why the second delete must retain the first position. Download the artifacts and rerun the pinned native recipe. Explore the boundaries →

Inside Iceberg delete files · September 2026

Follow actual position and equality delete files through manifest sequences, field IDs, original Parquet bytes and visible rows. Four native table histories show why same-commit rows and partition scope change the answer. Predict a filtered result, then reproduce all ten snapshots from the downloadable evidence bundle. The guide records the DuckDB reader limitations encountered during verification. Follow a delete →

Connected reading paths · September 2026

Follow one observation through shared controls and diagrams that map source positions to output, group state, file membership and selected bytes. DuckDB connects its filter to native join and Arrow variants. Engine join completion supplies the merge, and the emitted prefix supplies live output. Delta replay selects the version inspected below.

Iceberg’s evolution experiment now preserves the main story’s seventeen weather records and field identities. Lance shares exact/indexed query coordinates, carries the selected answer into storage accounting, and introduces manifests and footers before decoding values. Counterexamples sit beside the rules they challenge, with explicit returns to the running example.

Inspect the mechanisms · September 2026

DuckDB now exposes composed selections, nullable partial aggregates, real storage-layout pruning, estimated versus actual operator rows, duplicate join matches and Arrow string buffers. The engine guide adds collisions, outer-join counterexamples, build dependencies, skew and bounded sorting.

Delta makes writer conflicts, metadata/data retention and protocol requirements operable. Iceberg adds a real mixed-spec table, native residual predicates, field-name reuse, manifest status, snapshot reachability and retry validation. Lance now includes a reproducible IVF_FLAT index with measurable candidate recall and a browser-decoded footer traversal.

Every guide has a complete chapter navigator. Native evidence and illustrative state models are identified beside their controls. Execution · Operators · Delta · Iceberg · Lance

Follow one answer end to end · September 2026

The engine guide now tracks one observation through a visible predicate rewrite, join, ordering, and native Arrow result buffers. Lance connects selected neighbors to shared column-page buffers and fragment boundaries, then to original value bytes. Follow a query → · Follow selective access →

From a Lance neighbor to its bytes · September 2026

Compare every candidate’s squared distance and see why prefiltering changes top-k results. Follow a selected ID through its fragment and decode its two Float32 values from the original, hash-checked file. Trace a vector →

Inside the query engine · September 2026

Build station keys, probe the five warm observations, and merge two ordered streams. Inspect each slot of a real Arrow array, including the null whose value bytes must be ignored. Step through execution →

Follow the DuckDB values · September 2026

Inspect aligned input columns, predicate decisions and selected positions. Accumulate group counts and sums to see where the averages come from. Every threshold and input prefix is checked against native DuckDB. Follow the values →

Inside a Delta commit · September 2026

Follow one continuous investigation: why do 23 physical rows produce a 17-row table? Step through the original log lines, distinguish a reader's working state from complete snapshots, and follow a deleted observation through the retained and replacement files.

The expanded guide compares actual checkpoint and log sizes, exposes decoded checkpoint actions, and explains each file-pruning decision. Follow the Delta walkthrough →

Connected learning paths · September 2026

The homepage now connects storage, table membership, execution and selective access. A comparison page maps each technology's responsibilities and the limits of its demonstrated example. Compare the layers →

Lance, Explained · September 2026

Inspect a real manifest, fragment files and verified page buffers. Explore exact vector searches checked independently with NumPy, and download the complete reproduction bundle. Read the guide →

Query Engines, Explained · September 2026

Compare real DuckDB and DataFusion plans and exact matching results. Step through a conceptual join and inspect actual Arrow buffers. Read the guide →

Delta Lake, Explained · September 2026

Replay a real three-version transaction log, decode its classic checkpoint and compare active with historical files. Every version is independently verified with Spark Delta. Read the guide →

DuckDB, Explained · September 2026

Follow real Parquet scan plans, compare four recorded queries and inspect every operator. Open the matching sample and SQL in the explorer. Read the guide →

Local SQL with DuckDB · September 2026

The Parquet explorer now runs SELECT queries in your browser, with actual EXPLAIN plans, cancellation and complete CSV or typed JSON exports. The engine downloads when you first run SQL; your file stays local.

Exact integers, decimals and timestamps retain their database representation. Input, execution and export limits are visible beside the editor. Query a file →

Iceberg, Explained · September 2026

A real synthetic weather table shows how references select files. Explore three snapshots, a copy-on-write deletion, file pruning and a schema rename; inspect original JSON and Avro, then open the underlying Parquet files.

Every snapshot is independently verified with DuckDB. Read the guide →

Parquet preview

The first collection brings together Parquet, Explained, a local file explorer, your file's anatomy and the TypeScript reader's API overview.

  • Range reads and worker-based decoding in the explorer.
  • Mobile layout, keyboard navigation and reduced-motion support.
  • Explicit handling of dotted and nested column names.
  • Guide text available as HTML before JavaScript loads.