Filter
Exclude
Time range
-
Near
๐——๐—ฒ๐—น๐˜๐—ฎ ๐—ž๐—ฒ๐—ฟ๐—ป๐—ฒ๐—น ๐—ฃ๐—ผ๐˜„๐—ฒ๐—ฟ๐˜€ ๐—–๐—ผ๐—ป๐—ณ๐—น๐˜‚๐—ฒ๐—ป๐˜ ๐—ง๐—ฎ๐—ฏ๐—น๐—ฒ๐—ณ๐—น๐—ผ๐˜„ ๐Ÿš€ @confluentinc Tableflowโ€™s Delta Lake support is built on Delta Kernel, the open source library developed by Databricks to make Delta Lake universally accessible across any processing engine. Delta Kernel provides Java and Rust APIs that let developers read, write, and commit to Delta tables directlyโ€”without handling low-level Delta protocol details. This simplifies connector development and makes it easy for connectors to adopt the latest Delta innovations, such as catalog-managed commits, VARIANT data type, and type widening. ๐Ÿ”— Learn more: confluent.io/blog/tableflow-โ€ฆ By leveraging Delta Kernel, Tableflow achieves high performance, strong compatibility, and open interoperability while simplifying the path from real-time streaming data to fully governed Delta tables in @databricks Unity Catalog. Specifically, Delta Kernel enables Tableflow to do the following: โœ… Maintain full atomicity, consistency, isolation, and durability (ACID) guarantees and Delta-compliant transaction logs for reliable table updates โœ… Write at scale, efficiently materializing Kafka topics into optimized Parquet data and metadata commits โœ… Stay future-proof with transparent adoption of new Delta Lake features and protocol versions #opensource #deltalake #deltakernel #tableflow #confluent
9
1,316
TD and @dennylee discuss Delta Kernel and how it simplifies building Delta connectors by providing simple, narrow programmatic APIs that hide all the complex details of the Delta protocol specification. ๐Ÿฆ€ youtube.com/watch?v=KVUMFv74โ€ฆ #deltakernel #dataaisummit #oss #deltalake
2
9
1,549
Watch Delta Kernel: Simplifying Building Connectors for Delta from #DataAISummit to learn more โžก lnkd.in/eWbkTA24 cc Tathagata Das, @dennylee #opensource #oss #deltakernel #linuxfoundation (3/3)
2
6
762