Columnar / Storage and tables

Delta Lake

Delta Lake is an open table format that tracks a table through an ordered transaction log. The _delta_log folder sits beside the Parquet files. Commits add and remove file references; replaying them reconstructs a version, even while older files remain on disk. A checkpoint gives readers a shorter starting point. Databricks created Delta Lake; Spark and delta-rs produce and consume its tables.