๐๐ฒ๐น๐๐ฎ ๐๐ฒ๐ฟ๐ป๐ฒ๐น ๐ฃ๐ผ๐๐ฒ๐ฟ๐ ๐๐ผ๐ป๐ณ๐น๐๐ฒ๐ป๐ ๐ง๐ฎ๐ฏ๐น๐ฒ๐ณ๐น๐ผ๐ ๐
@confluentinc Tableflowโs Delta Lake support is built on Delta Kernel, the open source library developed by Databricks to make Delta Lake universally accessible across any processing engine. Delta Kernel provides Java and Rust APIs that let developers read, write, and commit to Delta tables directlyโwithout handling low-level Delta protocol details. This simplifies connector development and makes it easy for connectors to adopt the latest Delta innovations, such as catalog-managed commits, VARIANT data type, and type widening.
๐ Learn more:
confluent.io/blog/tableflow-โฆ
By leveraging Delta Kernel, Tableflow achieves high performance, strong compatibility, and open interoperability while simplifying the path from real-time streaming data to fully governed Delta tables in
@databricks Unity Catalog.
Specifically, Delta Kernel enables Tableflow to do the following:
โ
Maintain full atomicity, consistency, isolation, and durability (ACID) guarantees and Delta-compliant transaction logs for reliable table updates
โ
Write at scale, efficiently materializing Kafka topics into optimized Parquet data and metadata commits
โ
Stay future-proof with transparent adoption of new Delta Lake features and protocol versions
#opensource #deltalake #deltakernel #tableflow #confluent