Apache Airflow vs. Dollar Universe Workload Automation

For a quick job scanning of status and deep-diving into job issues, details, and flows, AirFlow does a good job. No fuss, no muss. The low learning curve as the UI is very straightforward, and navigating it will be familiar after spending some time using it. Our requirements are pretty simple. Job scheduler, workflows, and monitoring. The jobs we run are >100, but still is a lot to review and troubleshoot when jobs don't run. So when managing large jobs, AirFlow dated UI can be a bit of a drawback.

Incentivized

It is suitable for the scenarios that require more automation of the workloads in heterogeneous environments in order to reduce human errors and standardize the workflow.

Incentivized

• In charge of the ETL processes.
• As there is no incoming or outgoing data, we may handle the scheduling of tasks as code and avoid the requirement for monitoring.

Incentivized

• Functional across multiple platforms
• Lightweight and reliable
• More scope for increasing automation

• they should bring in some time based scheduling too not only event based
• they do not store the metadata due to which we are not able to analyze the workflows
• they only support python as of now for scripted pipeline writing

Incentivized

• Maybe UI can be further improved to be more intuitive and user friendly.

Incentivized

It is a good technical choice for the enterprise to promote automation and integration, and switching from the traditional silo manual work to a new event-driven, fully automated workflow, allowing better visibility and resilience.

Incentivized

It is generally ok, meeting our requirements, and the support tickets are handled properly within the agreed SLA.

Incentivized

There are a number of reasons to choose Apache Airflow over other similar platforms- Integrations—ready-to-use operators allow you to integrate Airflow with cloud platforms (Google, AWS, Azure, etc) Apache Airflow helps with backups and other DevOps tasks, such as submitting a Spark job and storing the resulting data on a Hadoop cluster It has machine learning model training, such as triggering a Sage maker job.

Incentivized

Well versed tool like others in industry.

• A lot of helpful features out-of-the-box, such as the DAG visualizations and task trees
• Allowed us to implement complex data pipelines easily and at a relatively low cost

Incentivized

• Increased reliability
• Minimises operators
• Handy