Data Architecture 101: Kappa (Real-Time Data)
3.8 هزار بار بازدید -
9 ماه پیش
-
►► See how I’ve helped
►► See how I’ve helped 10+ small data teams add more structure using modern tools, w/o making a mess (Free Workshop) → https://www.kahandatasolutions.com/si...
All things being equal, I think we'd all want access to source data in real time.
This is the holy-grail of data engineering and removes most delays to insights.
One architecture approach that makes this possible is known as the Kappa Architecture.
It focuses on real-time data loading & processing rather than waiting on batches.
No brainer, right?
Well, back here in the real-world, things aren't made equal.
Different architecture approaches require different levels of complexity (aka skill requirements).
Which impacts design & maintenance time.
Which all tends to mean higher overall cost of ownership.
All that to say, just because it's technically possible, doesn't always mean it's the best choice for your team.
Truthfully, I find it's usually not necessary.
BUT that doesn't mean you shouldn't be aware of it and be able to evaluate it.
So in this video we'll review at a high level what the Kappa architecture is about.
By the end, you'll understand the key points and be able to decide whether or not it makes sense for your team.
Enjoy!
Timestamps:
0:00 - Intro
0:23 - What is a Kappa Architecture
1:28 - General Considerations
2:53 - Example Architecture
Title & Tags:
Data Architecture 101: Kappa (Real-Time Data)
#kahandatasolutions #dataengineering #dataarchitecture
All things being equal, I think we'd all want access to source data in real time.
This is the holy-grail of data engineering and removes most delays to insights.
One architecture approach that makes this possible is known as the Kappa Architecture.
It focuses on real-time data loading & processing rather than waiting on batches.
No brainer, right?
Well, back here in the real-world, things aren't made equal.
Different architecture approaches require different levels of complexity (aka skill requirements).
Which impacts design & maintenance time.
Which all tends to mean higher overall cost of ownership.
All that to say, just because it's technically possible, doesn't always mean it's the best choice for your team.
Truthfully, I find it's usually not necessary.
BUT that doesn't mean you shouldn't be aware of it and be able to evaluate it.
So in this video we'll review at a high level what the Kappa architecture is about.
By the end, you'll understand the key points and be able to decide whether or not it makes sense for your team.
Enjoy!
Timestamps:
0:00 - Intro
0:23 - What is a Kappa Architecture
1:28 - General Considerations
2:53 - Example Architecture
Title & Tags:
Data Architecture 101: Kappa (Real-Time Data)
#kahandatasolutions #dataengineering #dataarchitecture
9 ماه پیش
در تاریخ 1402/08/10 منتشر شده
است.
3,802
بـار بازدید شده