Delta Lake is an open-source storage framework that enables building a Lakehouse architecture for Spark, Flink, Trino, Hive, Scala, Java, Rust, Python, & more!
Joined April 2019
- Tweets 2,438
- Following 67
- Followers 11,002
- Likes 1,097
1,193 Photos and videos
Jun 13
From 2017 to now, Delta Lake has grown to 40M downloads/month and powers daily processing of hundreds of exabytes.
At #DataAISummit, โThe Road to Delta 5.0โ will cover whatโs next:
๐น catalog-first Delta
๐น Data Source V2 modernization
๐น Delta Iceberg convergence
๐น Delta Kernel alignment (Java Rust)
Details: databricks.com/dataaisummit/โฆ
#DeltaLake #OpenSource
3
5
410
Jun 11
Back by popular demand! ๐ Delta Lake: The Definitive Guide book signing returns to Data AI Summit 2026.
If you are at Data AI Summit, come meet the authors, say hi to the Delta Lake community, and pick up a signed copy while supplies last.
๐๏ธ Tuesday, June 16
๐ 2:00โ2:30 PM
๐ Dev Lounge, Data AI Summit Expo, Moscone Center
Books tend to go quickly,ย so plan to stop by early. Hope to see you there! ๐
#DeltaLake #DataAISummit #OpenSource #DataEngineering @dennylee @newfront @Data_AI_Summit
2
3
27
2,664
Jun 8
Delta Lake Apache Iceberg co-evolved as parallel standards. That era is ending. ๐
๐น Iceberg v4 adaptive metadata tree single-file commits
๐น Delta Lake 5.0 adopts it as native content metadata ๐น One on-disk format, no translation layers
๐ Add it to your schedule: databricks.com/dataaisummit/โฆ
#DeltaLake #DataAISummit #ApacheIceberg
2
4
558
Jun 3
At Data AI Summit (June 15โ18), Scott Sandre will show how Catalog-Managed Tables support was designed in Delta Kernel. ๐
๐น A unified, catalog-agnostic API for connectors to build against
๐น Deep engine integrations for DuckDB, Delta-Spark, and Delta-Flink
๐น Catalog complexity kept out of Kernelโthe one right Delta abstraction
๐ Session details: databricks.com/dataaisummit/โฆ
#DeltaLake #DeltaKernel #DataAISummit
1
1
258
Jun 3
Data AI Summit session: Tyler Croy (delta-rs maintainer) will introduce Virtual Delta Tables and the associated open source code designed for multimodal inference.๐
๐๏ธ June 15-18
๐ San Francisco
๐ Session details: databricks.com/dataaisummit/โฆ
#DeltaLake #MultimodalAI #DataAndAISummit #Lakehouse
1
3
339
May 28
Headed to Data AI Summit (June 15โ18)? ๐๏ธ Don't miss Your Guide to Open Table Formats โ Delta, Iceberg, Best Practices, and What's Next!
Benjamin Mathew & Scott Sandre will cover Delta 5.0, Iceberg v4, Unified Delta Kernel (GA), and best practices across formats. Add it to your agenda. ๐
databricks.com/dataaisummit/โฆ
#DeltaLake #ApacheIceberg #DataAISummit
1
1
4
411
May 27
Delta Lake and Apache Icebergโข have converged on similar ideasโcolumnar metadata, manifest trees, deletion vectorsโbut two separate metadata structures still duplicate work.
At DAIS (June 15โ18, San Francisco): the next evolution of Delta Lake metadata. ๐
๐น Delta commits โ Iceberg v4โs adaptive metadata tree
๐น Tree-structured manifests Iceberg interoperability
๐น Transactional guarantees Delta users depend on โ preserved
๐ databricks.com/dataaisummit/โฆ
#DeltaLake #ApacheIceberg #DataandAISummit
3
6
393
May 26
DuckDB's Delta @unitycatalog_io extensions are no longer experimental ๐
๐ท INSERT via ATTACH (TYPE delta)
๐ท Time travel: AT (VERSION => n), VERSION, PIN_SNAPSHOT
๐ท Catalog Managed Tables Catalog Commits for concurrent writes
๐ท Incremental snapshot loading (nightly; v1.5.3 next)
Learn more: delta.io/blog/2026-05-06-delโฆ
#DeltaLake @duckdb
1
5
34
1,864
May 25
Headed to Data AI Summit? Don't miss New Foundations of Delta Lake with Kernel Spark's DSv2.
Delta on Spark DSv1 set the standard for the past 8 years. Rahul Potharaju & Tathagata Das will highlight the move to DSv2 and new foundations for the next decade.
๐๏ธJune 15-18
๐San Francisco
Add it to your agenda ๐
databricks.com/dataaisummit/โฆ
#DeltaLake #DataAISummit
2
8
568
May 22
.@ClickHouseDB is data lake ready. ๐
On the Delta Lake blog, the Clickhouse team shares how they integrated delta-kernel-rs: transaction logs, metadata, snapshots, Engine APIs, writes, schema evolution, time travel, and CDF (25.12).
๐ Read the post: delta.io/blog/2026-05-18-intโฆ
#DeltaLake #ClickHouse
7
573
May 12
๐ฃ Join us for the next Delta Lake community meetup on May 19!
Agenda:
๐น Issues & PR backlog
๐น Unified Kernel
๐นCommunity highlights
Register: luma.com/deltalake-0519
#DeltaLake #OpenSource #Rust #OpenLakehouse
1
2
248
Delta Lake retweeted
May 12
[Ep 1] Open Lakehouse AI: The Catalog Layer, Interoperability & AI-Agent Governance
Learn how composable lakehouses center the catalog for metadata, versioning, and commit coordination. ๐
๐ธ Interop: Iceberg REST, Unity Catalog (OSS), Spark, Delta Lake, Delta-RS, Iceberg.
๐ธ Governance: credential vending, row/col filters, column masks, audits, and why โno governance was the easiest governance.โ
๐ธ Agents as data customers (Temporal). Fine-grained access beats blanket credentials on 24/7 workloads.
๐ฅ Full episode: youtube.com/watch?v=dEFAkS7vโฆ
#OpenLakehouse #UnityCatalog @ApacheSpark @ApacheIceberg @DeltaLakeOSS
1
2
5
330
Apr 28
Delta Lake 4.2 pushes Delta Kernel further (new Flink connector, richer types) and hardens catalog-managed tables (atomic ops, SQL evolution, UniForm). ๐
๐น Kernel / Flink โ New Kernel Flink connector (experimental): catalog-managed, catalog-coordinated writes, exactly-once. Replaces legacy connector deprecated in Delta Lake 4.0.
๐น Schema (SQL) โ INSERT INTO โฆ BY NAME autoMerge adds columns in-commit. delta.stats.skipping.forceOptimizeStatsCollection โ data skipping on new columns without OPTIMIZE.
๐น Types โ Geospatial, Collation, Variant (shredding out of preview); Spark: full Variant schema conversion.
๐น Catalog โ RTAS / dynamic partition overwrite: one atomic commit. UniForm: Iceberg metadata at commit (reads immediate). HMS UniForm deprecated.
๐ Read more โ delta.io/blog/2026-04-17-delโฆ
#DeltaLake #OpenSource
7
590
Apr 21
Hey all! Thanks for joining us. ๐ Where are you tuning in from?
84
Apr 20
Delta Lake 4.2.0 is now available! ๐
Dive into the full release notes here: ๐github.com/delta-io/delta/reโฆ
Huge thanks to everyone in the Delta community who made this release possible! ๐
#DeltaLake #OpenSource #ApacheSpark #ApacheFlink
2
2
9
864
Apr 20
Here is a breakdown of whatโs new:
๐ [๐ฆ๐ฝ๐ฎ๐ฟ๐ธ] ๐จ๐ป๐ถ๐๐ ๐๐ฎ๐๐ฎ๐น๐ผ๐ด ๐ ๐ฎ๐ป๐ฎ๐ด๐ฒ๐ฑ ๐ง๐ฎ๐ฏ๐น๐ฒ ๐ฒ๐ป๐ต๐ฎ๐ป๐ฐ๐ฒ๐บ๐ฒ๐ป๐๐: REPLACE TABLE / RTAS and Dynamic Partition Overwrite support, automatic table schema/properties sync to catalog on table creation.
๐ [๐ฆ๐ฝ๐ฎ๐ฟ๐ธ] ๐๐ฒ๐น๐๐ฎ ๐ฆ๐ฝ๐ฎ๐ฟ๐ธ ๐ฉ๐ฎ ๐ฐ๐ผ๐ป๐ป๐ฒ๐ฐ๐๐ผ๐ฟ - ๐๐๐ฟ๐ฒ๐ฎ๐บ๐ถ๐ป๐ด ๐ฟ๐ฒ๐ฎ๐ฑ (๐ฒ๐
๐ฝ๐ฒ๐ฟ๐ถ๐บ๐ฒ๐ป๐๐ฎ๐น): enhance streaming read capabilities for catalog-managed table by supporting critical options like startingTimestamp and skipChangeCommits.
๐ [๐๐น๐ถ๐ป๐ธ] ๐ก๐ฒ๐ ๐๐ฒ๐ฟ๐ป๐ฒ๐น-๐ฏ๐ฎ๐๐ฒ๐ฑ ๐๐น๐ถ๐ป๐ธ ๐ฐ๐ผ๐ป๐ป๐ฒ๐ฐ๐๐ผ๐ฟ (๐ฒ๐
๐ฝ๐ฒ๐ฟ๐ถ๐บ๐ฒ๐ป๐๐ฎ๐น): a brand-new Kernel-based delta-flink connector that enables Apache Flink to read, write, and interact with catalog-managed Delta tables.
๐ [๐๐ฒ๐ฟ๐ป๐ฒ๐น] ๐๐ฒ๐ผ๐๐ฝ๐ฎ๐๐ถ๐ฎ๐น, ๐ฉ๐ฎ๐ฟ๐ถ๐ฎ๐ป๐ ๐๐, ๐ฎ๐ป๐ฑ ๐๐ผ๐น๐น๐ฎ๐๐ถ๐ผ๐ป๐ ๐๐ฎ๐ฏ๐น๐ฒ ๐ณ๐ฒ๐ฎ๐๐๐ฟ๐ฒ: Delta Kernel can now read and write tables using geometry/geography types with bounding-box data skipping, generally available Variant columns, and collated string types.
๐ [๐ฆ๐ฒ๐ฐ๐๐ฟ๐ถ๐๐] ๐ฉ๐ฒ๐ฟ๐๐ถ๐ผ๐ป ๐ต๐ฎ๐ฟ๐ฑ๐ฒ๐ป๐ถ๐ป๐ด ๐ณ๐ถ๐
: The Delta project has undergone a substantial hardening effort across multiple surface areas, including stronger validation and dependency security scanning to proactively reduce supply-chain risk.
1
191