Custom, Complex Windows at Scale using Apache Flink - Matt Zimmer (Netflix)
3.3 هزار بار بازدید -
7 سال پیش
-
Flink Forward Berlin, September 2017
Flink Forward Berlin, September 2017 #flinkforward
Matt Zimmer, Senior Software Engineer at Netflix
The windowing capabilities offered by most stream processing engines are limited to aligned windows of a fixed duration. However, many real-world event processing use cases don’t fit this rigid structure, resulting in awkward processing pipelines. There haven’t been good alternatives, until recently that is. Apache Flink offers a rich Window API that supports implementing unaligned windows of varying duration. In this talk, Matt Zimmer will discuss using this API at Netflix to aggregate events into windows customized along varying definitions of a session. He will talk about implementation details such as: * Handling out-of-order events * Limiting state build-up while aggregating a subset of events from an event stream * Periodically emitting early results * Creating windows bounded by a type of event Attendees will leave this talk with practical techniques and knowledge to implement their own custom windows in Apache Flink.
https://www.slideshare.net/FlinkForwa...
https://data-artisans.com/
Matt Zimmer, Senior Software Engineer at Netflix
The windowing capabilities offered by most stream processing engines are limited to aligned windows of a fixed duration. However, many real-world event processing use cases don’t fit this rigid structure, resulting in awkward processing pipelines. There haven’t been good alternatives, until recently that is. Apache Flink offers a rich Window API that supports implementing unaligned windows of varying duration. In this talk, Matt Zimmer will discuss using this API at Netflix to aggregate events into windows customized along varying definitions of a session. He will talk about implementation details such as: * Handling out-of-order events * Limiting state build-up while aggregating a subset of events from an event stream * Periodically emitting early results * Creating windows bounded by a type of event Attendees will leave this talk with practical techniques and knowledge to implement their own custom windows in Apache Flink.
https://www.slideshare.net/FlinkForwa...
https://data-artisans.com/
7 سال پیش
در تاریخ 1396/06/22 منتشر شده
است.
3,359
بـار بازدید شده