---

Видео на тему: DatabaseInternals - видео

TopK- Billion-Scale Hybrid Retrieval from the Ground Up (Marek Galovic)turbopuffer- Object Storage-native Database for Search (Simon Eskildsen)TonicDB- Databases without an OS? Meet QuinineHM (Filip Obradovic)YugabyteDB- Distributed PostgreSQL for Modern Apps (Hari Krishna Sunder)HorizonDB Совместная разработка PostgreSQL и Azure для облачных OLTP-систем (Адам Праут)Aurora DSQL- Serverless, Scalable, Global OLTP Database System (Marc Brooker)Redpanda Oxla or Why Your Hashmaps are Secretly Wrecking Your Performance (Akidau + Symanski)Push-Based Execution in DuckDB by Mark Raasveldt Выступающий: Марк Раасвелдт (CWI) Дата: 26.11.2021 Тема: Push-Based Execution in DuckDB Краткое содержание: DuckDB недавно перешла от исходной модели выполнения pull-based к модели выполнения push-based. В этом докладе мы обсудим, как работает новая push-based модель, причины, которые побудили к этому переходу, а также различные компромиссы и различия между этими двумя моделями. Биография: Марк Раасвелдт — сооснователь DuckDB Labs и постдокторант в группе Database Architectures института CWI. 00:00 Вступление 00:47 Recap how does database work 03:05 The exchange operator 04:10 Morsel driven parallelism 05:09 Pipeline breakers 06:50 Sink interface 07:43 Simplified hash join example 10:46 Push based execution 12:29 How do you make a push-based model 12:47 Operators 14:02 What does the operator interface look like 16:58 Finished flag 17:46 The source interface 18:56 Pipeline events 19:10 How do we implement unions? 20:34 Pipeline scheduling. 21:13 Split up pipelines into events. 22:47 Full/right outer joins 24:28 Sinks with expensive Finalize step 25:44 Scan sharing 27:48 Async I/O 28:24 Hybrid early/late materializationApache Fluss: A Streaming Storage for Real-Time Lakehouse (Jark Wu) 00:10 Intro 02:25 What is Apache Flink? 04:05 Stream processing Use cases: TikTok ral-time recommendation 05:40 FlinkSQL: "Incremental Materilized View" Compute engine 07:50 The Fact: No Such Suitable Storage 09:00 Fluss: a streaming table storage for Flink 14:05 What is Apache Fluss? 16:00 Fluss Overview 17:00 The Logical models in Fluss 17:56 Fluss Architecture 19:35 Fluss Table Sharding Scheme 21:30 Q&A 21:50 Write path and Durability of Log Tables 23:46 Write path and Durability of Primary-Key table 26:22 Q&A 27:39 Read path of Primary-Kay table 29:01 Log table is Columnar Stream 30:37 Log tablet is a columnar stream 33:20 Fluss Streaming Lakehouse 35:01 API abstraction 36:13 Q&A 37:00 Storage Unification 39:20 Lakehouse Tiering Service 41:55 Lifecycle Management of Different Tiers 43:32 Union Read: the read to Real-Time Lakehouse 44:44 Union Read for Streaming Quer 47:46 Quick introduction of Apache Paimon 48:43 Merge-On-Read of Batch Union Read 50:10 Deletion Vector of Batch Union Read (WIP) 53:00 Schema Evolution 54:02 Future Plan 54:37 Q&AFUT25-11 From Storage Formats to Open Governance- The Evolution to Apache Polaris (Prashant Singh)FUT25-10 Reconstructing History with XTDB (Jeremy Taylor + James Henderson)FUT25-05 Where We’re Going, We Don’t Need Rows- Columnar Data Connectivity with Apache Arrow ADBC (Ian Cook)STR25_15 Replication (Apache Kafka)STR25_14 Brokers (Apache Kafka)STR25_13 Partitions (Apache Kafka)STR25_12 Topics (Apache Kafka)STR25_11 Apache Kafka is back! (2025 Edition ft. Tim Berglund)STR25_09 Checkpoints and RecoverySTR25_07 Stateful Stream Processing with Flink SQLSTR25-06 Using Kafka with Flink