Tabular at Trino Fest - CDC patterns in Apache Iceberg

Mirroring transactional database tables in a data lake or warehouse is a popular use case for Apache Iceberg, but there are many approaches and pitfalls.

This talk details patterns and best practices for writing CDC change streams into Iceberg tables, and it covers the trade-offs between different approaches and features.

Timestamps:
0:00 Introduction
2:07 What is Iceberg and CDC?
5:36 Bank Example
11:14 Why is CDC difficult?
17:19 Storage Trade-offs
20:05 Change Log Pattern
21:54 MERGE Pattern
27:16 Link UPSERT Pattern
29:26 Hybrid Pattern