Apache Airflow is an open source tool that can be used to programmatically author, schedule and monitor data pipelines using Python and SQL. Created at Airbnb as an open-source project in 2014, Airflow was brought into the Apache Software Foundation’s Incubator Program 2016 and announced as Top-Level Apache Project in 2019. It is used as a data orchestration solution, with over 140 integrations and community support.
N/A
Tidal by Redwood
Score 6.4 out of 10
N/A
Tidal Automation, from Redwood Software since the early 2023 acquisition, is an enterprise workload automation platform for automating and orchestrating cross-application, cross-platform workloads – in on-prem, cloud or hybrid environments – from one central point of control. Tidal is used to optimize mission-critical business processes, manage…
For a quick job scanning of status and deep-diving into job issues, details, and flows, AirFlow does a good job. No fuss, no muss. The low learning curve as the UI is very straightforward, and navigating it will be familiar after spending some time using it. Our requirements are pretty simple. Job scheduler, workflows, and monitoring. The jobs we run are >100, but still is a lot to review and troubleshoot when jobs don't run. So when managing large jobs, AirFlow dated UI can be a bit of a drawback.
On coming to well suited experience while error handling and effortless recovery processes are crucial, Tidal performs exceptionally well. It can identify and fix automated issues, reducing downtime and interruptions. A smaller automation solution may be more cost-effective if a business primarily utilizes a single platform or uses a small number of applications that do not require complex integration.
Tidal Automation allows us to automate and schedule/ perform various tasks in a easy and effective manner. It is highly interactive and effective allowing nearly 4k -5k jobs to run a day.
Tidal is designed to be easily shareable and collaborative allowing multiple users to work at a particular time making work effective.
Tidal allows us to ensure that every event which is triggered is exactly when it's supposed to be regardless of other activities which are going as well.
It is user-friendly to generate the reports required of a particular object and allows us to test the codes perfectly well before executing in live environment.
Still a bit slow when navigating. If you close a job you have to wait a few seconds to open another one. Even when you made no changes.
When viewing a job and make no changes, the "ok" button changes the last modified date as if you made a change. No big deal, but wastes time when troubleshooting a problem and looking into what jobs were changed last.
You can see the parameters column in the "job activity", but not in "Job definitions".
Can't search the parameters field in the filter.
Changing a variable name does not change it on the job. It still works because Tidal Automation uses the ID number. It just causes confusion when you see a variable on a job and can't find the variable under "Variables". On top of that, Tidal Automation does not show the ID column under "Variables" making it even more difficult to find the variable.
We are on the fence. The increased pricing for renewals is staggering. With new automation options like Microsoft's Power Automate and Event Driven Ansible on the field, there are other options now available.
Having provided consulting services for years on Tidal by Redwood, I recommend going with a solutions partner or consultant to deploy it. I believe there are sizing and tuning guidelines that should be followed for environments of scale. I believe they are not critical when first lighting up the product, but if you are not aware of them you will encounter performance degradation after a few thousand job objects are added.
There are a number of reasons to choose Apache Airflow over other similar platforms- Integrations—ready-to-use operators allow you to integrate Airflow with cloud platforms (Google, AWS, Azure, etc) Apache Airflow helps with backups and other DevOps tasks, such as submitting a Spark job and storing the resulting data on a Hadoop cluster It has machine learning model training, such as triggering a Sage maker job.
1. Tidal is good at processing large volume pf data and is cost effective. 2. Tidal can automate the scheduling of production objects, ensure that materials are delivered on time 3. Tidal Process large volumes of data which cannot be done everyday by running codes/scripts manually which does it with ease when required.
