Logging in Azure Data Factory: Save logs into Database - Pipeline/Session Logs

In this video we see how to save the Azure Data Factory pipeline logs into a Azure SQL Database. Using stored procedures we store the pipeline output properties into a table where we can analyze them later on. You can find the stored procedures and DDL statements on my GitHub below. We also enable logging in a Data Factory copy activity and move the exported data from the Blob Storage into the Database. This way we store Session Logs too.

Follow me on social media:
LinkedIn: LinkedIn: apostolos-athanasiou-9a0baa119
GitHub: https://github.com/apostolos1927/Data...
Facebook: https://www.facebook.com/profile.php?...

00:00 - Intro
01:07 - DDL statements & Stored Procedures
15:58 - Saving pipeline logs into Database
17:13 - Pipeline properties for logging
23:07 - Session logging in copy activities
23:32 - Conclusion

• #education
• #azure_data_factory
• #logging
• #logs
• #data_factory_logging
• #logs_in_data_factory
• #logging_in_data_factory
• #pipeline_logs
• #pipeline_logs_in_data_factory
• #pipeline_and_session_logs_in_data_factory
• #data_factory_pipeline_logs
• #data_factory_session_logs
• #azure_data_factory_logs
• #azure_data_factory_logging
• #save_data_factory_logs
• #save_logs_in_database_from_data_factory
• #data_factory_logging_into_database
• #store_data_factory_logs
• #azure_data_factory_save_pipeline_logs
• #logs_logging