Build your first pipeline DAG | Apache airflow for beginners
#apacheairflow #airflowforbeginners #datapipeline #etl #etlpipeline #airflow2 Welcome to series Apache airflow for beginners. In this video, you'll learn how to build your first pipeline DAG in Apache Airflow. As a Data Engineer, you will be tasked to write a data aggregation and cleaning pipeline. We are going to build an ETL pipeline as part of Airflow DAG with a real-world example of Hotel Reservation Records. Read at https://maxcotec.com/learning/apache-airflow --------------------------------------------------- Get complete code from here; https://github.com/maxcotec/Apache-Airflow/blob/1787097721a8cec8999bdaee4c04a9f4bc0e1f71/DAG_ingestion_pipeline Watch next video Airflow Macros https://youtu.be/TxrGzLlsXdM Watch the previous video here: Run Airflow locally via Docker https://www.youtube.com/watch?v=TkvX1L__g3s&t=222s -------------------------------------------------------- Timeline 00:00 intro 00:13 definition 00:40 DAG building blocks 01:39 Example - ingestion pipeline overview 03:05 Lets Code DAG 07:53 Lets deploy DAG 09:29 Examine DAG output 10:25 Ingest more data 11:17 Outro -------------------------------------------------------- Learn Something New Below: https://maxcotec.com/blog/is-a-stundenglass-3d-print-compatible/ https://maxcotec.com/blog/pv-center-for-information-technology-excellence/ ? If this video was helpful to you, please don't forget to thumbs-up, ⌨️ leave a comment and ? share with your friends. ⭐ Support our work by hitting that subscribe button. ? hit bell icon to stay alert on more upcoming interesting stuff. Stay tuned !
#apacheairflow #airflowforbeginners #datapipeline #etl #etlpipeline #airflow2 Welcome to series Apache airflow for beginners. In this video, you'll learn how to build your first pipeline DAG in Apache Airflow. As a Data Engineer, you will be tasked to write a data aggregation and cleaning pipeline. We are going to build an ETL pipeline as part of Airflow DAG with a real-world example of Hotel Reservation Records. Read at https://maxcotec.com/learning/apache-airflow --------------------------------------------------- Get complete code from here; https://github.com/maxcotec/Apache-Airflow/blob/1787097721a8cec8999bdaee4c04a9f4bc0e1f71/DAG_ingestion_pipeline Watch next video Airflow Macros https://youtu.be/TxrGzLlsXdM Watch the previous video here: Run Airflow locally via Docker https://www.youtube.com/watch?v=TkvX1L__g3s&t=222s -------------------------------------------------------- Timeline 00:00 intro 00:13 definition 00:40 DAG building blocks 01:39 Example - ingestion pipeline overview 03:05 Lets Code DAG 07:53 Lets deploy DAG 09:29 Examine DAG output 10:25 Ingest more data 11:17 Outro -------------------------------------------------------- Learn Something New Below: https://maxcotec.com/blog/is-a-stundenglass-3d-print-compatible/ https://maxcotec.com/blog/pv-center-for-information-technology-excellence/ ? If this video was helpful to you, please don't forget to thumbs-up, ⌨️ leave a comment and ? share with your friends. ⭐ Support our work by hitting that subscribe button. ? hit bell icon to stay alert on more upcoming interesting stuff. Stay tuned !