Apache Airflow vs. Oracle Autonomous Transaction Processing

For a quick job scanning of status and deep-diving into job issues, details, and flows, AirFlow does a good job. No fuss, no muss. The low learning curve as the UI is very straightforward, and navigating it will be familiar after spending some time using it. Our requirements are pretty simple. Job scheduler, workflows, and monitoring. The jobs we run are >100, but still is a lot to review and troubleshoot when jobs don't run. So when managing large jobs, AirFlow dated UI can be a bit of a drawback.

Incentivized

If your company's value is not increased by maintaining databases, you should use the ATP.

• In charge of the ETL processes.
• As there is no incoming or outgoing data, we may handle the scheduling of tasks as code and avoid the requirement for monitoring.

Incentivized

• Quick and simple to deploy (less than 5 steps to create a new instance)
• No administrative tasks (backup, re-sizing) required

Incentivized

• they should bring in some time based scheduling too not only event based
• they do not store the metadata due to which we are not able to analyze the workflows
• they only support python as of now for scripted pipeline writing

Incentivized

• Connectivity between ATP and Oracle Kuberentes Engine cluster seems to drop randomly
• The ATP could come in an even smaller shape as the smallest shape is already quite big . But if must be more than always free, as that version does have connectivity limitations about 20 connections

You get almost all the features of Oracle Database, including the ability to deploy APEX applications, without having to worry about system, storage or database configuration and maintenance, backups, etc. Also, you can start for free and move up adding resources as you need.

Incentivized

There are a number of reasons to choose Apache Airflow over other similar platforms- Integrations—ready-to-use operators allow you to integrate Airflow with cloud platforms (Google, AWS, Azure, etc) Apache Airflow helps with backups and other DevOps tasks, such as submitting a Spark job and storing the resulting data on a Hadoop cluster It has machine learning model training, such as triggering a Sage maker job.

Incentivized

Both Azure SQL Database and Oracle Autonomous Transaction Processing offer a powerful and robust database platform. The advantage of Autonomous Transaction Processing is the ability to host APEX applications for free and without any special/complicated deployment or configuration process. Having a rapid development environment embedded for free was the main advantage for choosing Oracle as my cloud database provider.

Incentivized

• A lot of helpful features out-of-the-box, such as the DAG visualizations and task trees
• Allowed us to implement complex data pipelines easily and at a relatively low cost

Incentivized

• Approx 100 times faster spin up than manual setup of an Oracle DB
• 3h annual maintenance for big version releases (18c to 19c) compared to est. 500h annual maintenance with manual database updates