Chapter #9 - How to design data pipeline on gcp (Google Cloud Platform) ?
100.6 هزار بار بازدید -
3 سال پیش
-
Chapter #9 - Designing data
Chapter #9 - Designing data pipeline solution on GCP
#datapipeline #googlecloud #gcp
What is Data Pipeline? - Chapter #8 - Cloud IAM Basics | ident...
**Google Cloud Platform Beginner Series**
Google Cloud Platform Beginner Series (2020) - Google Cloud Platform (GCP) - Beginne...
**Networking Basics Playlist**
Networking and Infra Concepts - Networking & Infra Concepts
** Other Popular Playlists**
Crunching Data Series (2020) - Learn - Data Engineering, Data Archit...
Latest technology tutorial (2020) -
What is a Data Vault ? | 3NF vs Dimen...
Hi Friends, I am Anshul Tiwari, and welcome to our youtube channel IT k Funde.
More about this video -
In this video, we will learn something different, instead of discussing any specific product on google cloud platform, we will learn how to design a data pipeline solution on google cloud platform.
This will help us in 2 ways -
1. This will helps us improve our design thinking
2. We will get an overview of all google cloud products that can be used in developing a data pipeline.
We will start with understanding a typical on premise data pipeline and enterprise data warehouse (edw) design. We will then take this same solution and try to implement it on google cloud platform.
On-Prem solution design contains below products -
#Streaming data - Apache Kafka
#ETL - Informatica, IBM Datastage, SAP Data Services
#EDW - Teradata
#Dashbaords & reporting - Tableau and SAP Business Objects
GCP cloud design contains below products -
#Streaming data - Cloud PubSub
#ETL - Cloud Data Fusion
#Storage - Google Cloud Storage
#EDW - Google Big Query
#Dashboards - Looker, Google Data Studio
#ML - AutoML
#Metadata Management - Data Catalog
PLEASE WATCH ALL THE VIDEOS IN ANY PLAYLIST FOR BETTER UNDERSTANDING !!
Credits & Resources for further study -
GCP resources documentation for detailed study -
#Cloud PubSub - https://cloud.google.com/pubsub/docs/...
#Cloud Data Fusion - https://cloud.google.com/data-fusion/...
#Google Cloud Storage - https://cloud.google.com/storage/docs...
#Google Big Query- https://cloud.google.com/bigquery/doc...
#Data Studio - https://cloud.google.com/bi-engine/do...
#Looker - https://docs.looker.com/
#AutoML - https://cloud.google.com/automl/docs
#Data Catalog - https://cloud.google.com/data-catalog...
images - pixabay.com
**Social Links**
Youtube - itkfunde
Facebook - Facebook: ITkFUNDE
Linkedin - LinkedIn: ansh9685
Twitter - Twitter: ansh9685
Instagram -Instagram: itkfunde
**About This Channel**
Friends ITkFUNDE channel wants to bring I.T. industry knowledge, information, career advice, and much more to every individual regardless of whether he or she belongs to I.T or not. This channel is for everyone interested in learning something new!
#datapipeline #googlecloud #gcp
What is Data Pipeline? - Chapter #8 - Cloud IAM Basics | ident...
**Google Cloud Platform Beginner Series**
Google Cloud Platform Beginner Series (2020) - Google Cloud Platform (GCP) - Beginne...
**Networking Basics Playlist**
Networking and Infra Concepts - Networking & Infra Concepts
** Other Popular Playlists**
Crunching Data Series (2020) - Learn - Data Engineering, Data Archit...
Latest technology tutorial (2020) -
What is a Data Vault ? | 3NF vs Dimen...
Hi Friends, I am Anshul Tiwari, and welcome to our youtube channel IT k Funde.
More about this video -
In this video, we will learn something different, instead of discussing any specific product on google cloud platform, we will learn how to design a data pipeline solution on google cloud platform.
This will help us in 2 ways -
1. This will helps us improve our design thinking
2. We will get an overview of all google cloud products that can be used in developing a data pipeline.
We will start with understanding a typical on premise data pipeline and enterprise data warehouse (edw) design. We will then take this same solution and try to implement it on google cloud platform.
On-Prem solution design contains below products -
#Streaming data - Apache Kafka
#ETL - Informatica, IBM Datastage, SAP Data Services
#EDW - Teradata
#Dashbaords & reporting - Tableau and SAP Business Objects
GCP cloud design contains below products -
#Streaming data - Cloud PubSub
#ETL - Cloud Data Fusion
#Storage - Google Cloud Storage
#EDW - Google Big Query
#Dashboards - Looker, Google Data Studio
#ML - AutoML
#Metadata Management - Data Catalog
PLEASE WATCH ALL THE VIDEOS IN ANY PLAYLIST FOR BETTER UNDERSTANDING !!
Credits & Resources for further study -
GCP resources documentation for detailed study -
#Cloud PubSub - https://cloud.google.com/pubsub/docs/...
#Cloud Data Fusion - https://cloud.google.com/data-fusion/...
#Google Cloud Storage - https://cloud.google.com/storage/docs...
#Google Big Query- https://cloud.google.com/bigquery/doc...
#Data Studio - https://cloud.google.com/bi-engine/do...
#Looker - https://docs.looker.com/
#AutoML - https://cloud.google.com/automl/docs
#Data Catalog - https://cloud.google.com/data-catalog...
images - pixabay.com
**Social Links**
Youtube - itkfunde
Facebook - Facebook: ITkFUNDE
Linkedin - LinkedIn: ansh9685
Twitter - Twitter: ansh9685
Instagram -Instagram: itkfunde
**About This Channel**
Friends ITkFUNDE channel wants to bring I.T. industry knowledge, information, career advice, and much more to every individual regardless of whether he or she belongs to I.T or not. This channel is for everyone interested in learning something new!
3 سال پیش
در تاریخ 1400/01/11 منتشر شده
است.
100,663
بـار بازدید شده