Apache Airflow is an open source tool that can be used to programmatically author, schedule and monitor data pipelines using Python and SQL.
N/A
AutoSys Workload Automation
Score 7.2 out of 10
N/A
Broadcom offers AutoSys Workload Automation, a solution to enhance visibility and control of complex workloads across platforms, ERP systems, and the cloud. It helps to reduce the cost and complexity of managing mission critical business processes, ensuring consistent and reliable service delivery. It is based on the former CA AutoSys, acquired by Broadcom with CA Technologies.
In AutoSys Workload Automation, strong legacy presence, proven reliability, slightly simpler for basic scheduling if user already have in-house AutoSys Workload Automation expertise. so we already had investments in AutoSys Workload Automation, trained team, and large numbers …
Airflow is well-suited for data engineering pipelines, creating scheduled workflows, and working with various data sources. You can implement almost any kind of DAG for any use case using the different operators or enforce your operator using the Python operator with ease. The MLOps feature of Airflow can be enhanced to match MLFlow-like features, making Airflow the go-to solution for all workloads, from data science to data engineering.
In Informatica or Spark ETL jobs loading data into a Data Warehouse using AutoSys Workload Automation triggers ETL pipelines, monitors completion of jobs and also triggers the power bi report refresh. AutoSys Workload Automation can schedule jobs based on file arrival, and provide alerts if any data loads get failed. A small data team managing greater than 100 jobs. Buying and maintaining AutoSys Workload Automation is overkill in terms of cost, complexity and maintenance does not justify the value
Apache Airflow is one of the best Orchestration platforms and a go-to scheduler for teams building a data platform or pipelines.
Apache Airflow supports multiple operators, such as the Databricks, Spark, and Python operators. All of these provide us with functionality to implement any business logic.
Apache Airflow is highly scalable, and we can run a large number of DAGs with ease. It provided HA and replication for workers. Maintaining airflow deployments is very easy, even for smaller teams, and we also get lots of metrics for observability.
UI/Dashboard can be updated to be customisable, and jobs summary in groups of errors/failures/success, instead of each job, so that a summary of errors can be used as a starting point for reviewing them.
Navigation - It's a bit dated. Could do with more modern web navigation UX. i.e. sidebars navigation instead of browser back/forward.
Again core functional reorg in terms of UX. Navigation can be improved for core functions as well, instead of discovery.
In AutoSys Workload Automation, Workflow and job dependencies are shown in a static way, which makes user difficult to visualize more complex job chains or debug failures in a graphical view.
Latest schedulers (like Control-M, Airflow) allow more easy workflow design makes user to understand it better, but AutoSys Workload Automation relies heavily on JIL scripts and text-based job definitions.
AutoSys Workload Automation interface is very slow when searching or filtering over thousand of jobs
For its capability to connect with multicloud environments. Access Control management is something that we don't get in all the schedulers and orchestrators. But although it provides so many flexibility and options to due to python , some level of knowledge of python is needed to be able to build workflows.
AutoSys Workload Automation is a place to schedule, monitor, and manage thousands of jobs. Time, file arrival, dependencies are much very flexible. Once jobs are set up they run consistently with minimal intervention. AutoSys Workload Automation is powerful but very complex o understand. Mainly for beginners the interface is not user friendly. AutoSys Workload Automation is very useful in terms of job scheduling and automation. AutoSys Workload Automation is also useful for strong logging and reporting purpose. AutoSys Workload Automation has reduced a lot of human efforts and manual processing.
Multiple DAGs can be orchestrated simultaneously at varying times, and runs can be reproduced or replicated with relative ease. Overall, utilizing Apache Airflow is easier to use than other solutions now on the market. It is simple to integrate in Apache Airflow, and the workflow can be monitored and scheduling can be done quickly using Apache Airflow. We advocate using this tool for automating the data pipeline or process.
In AutoSys Workload Automation, strong legacy presence, proven reliability, slightly simpler for basic scheduling if user already have in-house AutoSys Workload Automation expertise. so we already had investments in AutoSys Workload Automation, trained team, and large numbers of jobs running thus making AutoSys Workload Automation more cost-effective to continue rather than migrate to use AutoSys Workload Automation over Control-M. All our workloads are heavily batch-driven (SAP, ETL) real-time pipelines. AutoSys Workload Automation provides the robustness and enterprise support that Airflow lacks without heavy internal engineering overhead.
Impact Depends on number of workflows. If there are lot of workflows then it has a better usecase as the implementation is justified as it needs resources , dedicated VMs, Database that has a cost
Critical job process consistently complete on time without delays
Significant reduction in team effort due to this resources for available for higher-value tasks. Human intervention is reduced due to this incident costs are also reduced.
Faster incident resolution, improved productivity, and lower downtime.
