We used it to manage processes for etl pipelines
July 05, 2022
We used it to manage processes for etl pipelines
Score 9 out of 10
Vetted Review
Verified User
Overall Satisfaction with Apache Airflow
We use Apache Airflow to streamline the data pipelines, create workflows according to the needs of the project and overall monitoring of the functionality itself. In addition, we are using Apache Airflow to solve the problem of retrieving data from Hive before creating the workflow in its entirety. It's also utilized for automation.
- In charge of the ETL processes.
- As there is no incoming or outgoing data, we may handle the scheduling of tasks as code and avoid the requirement for monitoring.
- There is no way to assess the processes because they do not keep the metadata.
- Python is currently the only language supported for creating programmed pipelines.
- They need to implement both event-based and time-based scheduling.
- Most of the ETL processes were automated, cutting down on human labor.
- Apache Airflow's user interface (UI) was very informative and straightforward.
- Since ETL processes were providing data via airflow, we were able to gain a deeper comprehension of the data at hand.
Multiple DAGs can be orchestrated simultaneously at varying times, and runs can be reproduced or replicated with relative ease. Overall, utilizing Apache Airflow is easier to use than other solutions now on the market. It is simple to integrate in Apache Airflow, and the workflow can be monitored and scheduling can be done quickly using Apache Airflow. We advocate using this tool for automating the data pipeline or process.
Do you think Apache Airflow delivers good value for the price?
Yes
Are you happy with Apache Airflow's feature set?
Yes
Did Apache Airflow live up to sales and marketing promises?
Yes
Did implementation of Apache Airflow go as expected?
Yes
Would you buy Apache Airflow again?
Yes