# Native Delta materialization and retention experiment

Spark 3.5.6 with Delta Spark 3.3.2 continues the native deletion-vector table at version 4.
REORG materializes its two masked observations. A second REORG is idempotent. VACUUM
removes obsolete files in this disposable copy and writes versions 6 and 7 for its audit
actions. Current rows remain intact, while a native historical read fails.

Download `/delta/lifecycle.zip` for the complete capture and recipe. Its `before`,
`after-reorg`, and `after-vacuum` directories are complete captured table directories.
Preserve their relative paths when extracting. `proof.json` lists every
artifact with size and SHA-256, full native records, active file actions and the error.
The proof is a recorded native experiment, not a browser table reader.
Hadoop's dot-prefixed checksum companions have hash-named download copies because static
servers may hide dotfiles. Their bytes are verified unchanged; the archive preserves their
original names and paths.

From the project root with Java 17 available:

```fish
uv run explorer/scripts/delta/native-lifecycle.py --verify
```

To generate a separate capture, pass a new output directory:

```fish
uv run explorer/scripts/delta/native-lifecycle.py --output /tmp/columnar-delta-cleanup-new
```

For standalone reproduction, extract the complete archive into a fresh directory, then run:

```fish
uv run reproduce.py --source ./before --output ./new-capture
uv run reproduce.py --source ./before --output ./new-capture --verify
```

The script always copies its source into a fresh temporary directory. Zero retention and
the disabled duration check apply only to that temporary copy; they are not production
retention settings. Existing output directories are never overwritten. Verification
repeats the native experiment, compares semantic outcomes, verifies captured file hashes,
and reads every captured current version with Spark and its physical Parquet IDs with
PyArrow. Native filenames and commit timestamps can differ between fresh runs.