Sources, Sinks, and Operators: A Performance Deep Dive
5.6 هزار بار بازدید -
3 سال پیش
-
At Splunk we have built
At Splunk we have built a Flink streaming infrastructure and scaled it to petabytes of data per day and millions of events per second. Along the way, we've learned a lot about writing performant operations in the DataStream API and putting together high-throughput pipelines. This talk will cover our real-life experiences in scaling for this throughput and the best practices we've learned along the way. We'll talk about aggregation, built-in functions, user-defined functions, the async I/O API, as well as discoveries around GC, Java object management, state backends, and serialization.
0:00 Introduction
1.07 Environmental & Performance Testing
3:57 The basic challenges
10:17 Sources
21:22 Sinks
25:10 Putting it all together
0:00 Introduction
1.07 Environmental & Performance Testing
3:57 The basic challenges
10:17 Sources
21:22 Sinks
25:10 Putting it all together
3 سال پیش
در تاریخ 1400/08/10 منتشر شده
است.
5,645
بـار بازدید شده