← Return to the Iceberg guide When a delete reaches a row.
The main guide follows file membership into original delete bytes. These native experiments continue with replacement keys, partition scope, compaction and cumulative deletion vectors.
Predict a query, then compare native results
Can the new key survive and satisfy a predicate?
Use the equality branch: the original eighteen temperatures are 5–22°C. One commit deletes key 3 and adds ID 3 at 30°C. Predict the result of SELECT observation_id WHERE temperature_c >= 20 ORDER BY observation_id. The delete and new file have sequence 2; the original data has sequence 1.
Your predicted IDs
Reproduce the evidence
Download the original tables, source data and pinned verifier. The README gives the Java 17 and uv commands. The verifier checks artifact hashes, makes a relocated private copy of the v2 absolute paths, then compares every field of every snapshot with Spark 3.5.6 / Iceberg 1.9.2 and DuckDB 1.4.4. It also checks predicates and Spark projections. Originals remain unchanged.
Reader limits matter: DuckDB’s narrow filtered equality-delete projection hits upstream issue 940. Its oracle query keeps the predicate and key columns; Spark separately checks the narrow projection. DuckDB 1.4.4 and 1.5.2 also removed the North key in an unrestricted scan with only a South-scoped delete. The verifier combines explicit North and South scans for that branch, checking every expected row; Spark checks the unrestricted scan. This is a reader limitation, not the format’s partition rule.
These are v2 position and equality deletes, committed with Iceberg’s Java API from PyArrow-written Parquet. Continue to the native key, partition and compaction experiments, the v3 Puffin bytes, and overlapping writer schedules for those mechanisms. The applicability explanation follows the pinned Iceberg specification.
Execute the edge cases
A key, a partition specification, and a rewrite all change applicability.
Native delete boundary
This explicit variation changes stations for IDs 3 and 4 to null and adds a second ID 3 at South, 30°C. The delete file contains (observation_id=3, station=null), with equality_ids=[1,2]. All key columns must match; a null delete value matches a null row value.
ID 3, null, 7°C
3=3 ✓ · null matches null ✓
Initially visibleID 4, null, 8°C
4=3 ✗ · null matches null ✓
SurvivesID 3, South, 30°C
3=3 ✓ · South matches null ✗
SurvivesThe table goes from nineteen to eighteen rows. Treating the key as OR would remove too much. Treating null equality like SQL’s ordinary = NULL would remove nothing. Spark independently reads the result and a temperature-only projection that must still apply both hidden key fields.
Native scan tasks: data files and attached deletes| Data file | Spec | Physical records | Deletes the reader must apply |
|---|
| A.parquet | 0 | 19 | None |
19 native result rows · IDs 1, 2, 3, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18
Snapshot 215217797410934907 · sequence 1 · Original metadata JSON
Inspect the native manifest entries and complete result
A real v3 Puffin file
The next delete must include the previous positions.
The same eighteen rows now live in one Iceberg v3 data file A. Apache Iceberg’s native DV writer deletes position 2 (ID 3), then writes a replacement vector containing positions 2 and 15 (IDs 3 and 16). A stays unchanged. The writer reads the prior vector, merges the next position, and atomically replaces its manifest entry.
Native vector version
pos 0ID 1Visible
pos 1ID 2Visible
pos 2ID 3Masked
pos 3ID 4Visible
pos 4ID 5Visible
pos 5ID 6Visible
pos 6ID 7Visible
pos 7ID 8Visible
pos 8ID 9Visible
pos 9ID 10Visible
pos 10ID 11Visible
pos 11ID 12Visible
pos 12ID 13Visible
pos 13ID 14Visible
pos 14ID 15Visible
pos 15ID 16Visible
pos 16ID 17Visible
pos 17ID 18Visible
Native Spark result: 17 rows. A snapshot may have at most one DV per referenced data file. Keeping only the newest position would make ID 3 reappear. The replacement DV must preserve earlier deleted positions.
The manifest locates this blob at offset 4, length 42, inside the original Puffin file. Its referenced data file is vectors/A.parquet.
PFA1 identifies the container. Its trailing little-endian footer length locates JSON blob metadata. The blob contains a big-endian length, Roaring magic, little-endian bitmap structure and big-endian CRC-32. The footer’s referenced-data-file and cardinality must agree with the manifest. The browser checks all of these, then reads the original unsigned 16-bit array entries. This bounded decoder supports these sparse containers, not every legal bitmap encoding.
The underlying Roaring framing is shared with Delta’s vector lesson; Puffin’s container and reference checks are separate. Puffin deletion-vector specification.