Apache Airflow vs. RabbitMQ

Overview
ProductRatingMost Used ByProduct SummaryStarting Price
Apache Airflow
Score 8.6 out of 10
N/A
Apache Airflow is an open source tool that can be used to programmatically author, schedule and monitor data pipelines using Python and SQL. Created at Airbnb as an open-source project in 2014, Airflow was brought into the Apache Software Foundation’s Incubator Program 2016 and announced as Top-Level Apache Project in 2019. It is used as a data orchestration solution, with over 140 integrations and community support.N/A
RabbitMQ
Score 8.5 out of 10
N/A
RabbitMQ, an open source message broker, is part of Pivotal Software, a VMware company acquired in 2019, and supports message queue, multiple messaging protocols, and more. RabbitMQ is available open source, however VMware also offers a range of commercial services for RabbitMQ; these are available as part of the Pivotal App Suite.N/A
Pricing
Apache AirflowRabbitMQ
Editions & Modules
No answers on this topic
No answers on this topic
Offerings
Pricing Offerings
Apache AirflowRabbitMQ
Free Trial
NoNo
Free/Freemium Version
YesNo
Premium Consulting/Integration Services
NoNo
Entry-level Setup FeeNo setup feeNo setup fee
Additional Details——
More Pricing Information
Features
Apache AirflowRabbitMQ
Workload Automation
Comparison of Workload Automation features of Product A and Product B
Apache Airflow
9.7
10 Ratings
16% above category average
RabbitMQ
-
Ratings
Multi-platform scheduling9.910 Ratings00 Ratings
Central monitoring9.910 Ratings00 Ratings
Logging9.910 Ratings00 Ratings
Alerts and notifications9.810 Ratings00 Ratings
Analysis and visualization9.910 Ratings00 Ratings
Application integration9.010 Ratings00 Ratings
Best Alternatives
Apache AirflowRabbitMQ
Small Businesses

No answers on this topic

No answers on this topic

Medium-sized Companies
ActiveBatch Workload Automation
ActiveBatch Workload Automation
Score 8.1 out of 10
Apache Kafka
Apache Kafka
Score 8.1 out of 10
Enterprises
Redwood RunMyJobs
Redwood RunMyJobs
Score 9.3 out of 10
Apache Kafka
Apache Kafka
Score 8.1 out of 10
All AlternativesView all alternativesView all alternatives
User Ratings
Apache AirflowRabbitMQ
Likelihood to Recommend
9.0
(10 ratings)
9.9
(11 ratings)
Usability
10.0
(1 ratings)
8.0
(1 ratings)
Support Rating
-
(0 ratings)
6.5
(4 ratings)
User Testimonials
Apache AirflowRabbitMQ
Likelihood to Recommend
Apache
For a quick job scanning of status and deep-diving into job issues, details, and flows, AirFlow does a good job. No fuss, no muss. The low learning curve as the UI is very straightforward, and navigating it will be familiar after spending some time using it. Our requirements are pretty simple. Job scheduler, workflows, and monitoring. The jobs we run are >100, but still is a lot to review and troubleshoot when jobs don't run. So when managing large jobs, AirFlow dated UI can be a bit of a drawback.
Read full review
Open Source
It is highly recommended that if you have microservices architecture and if you want to solve 2 phase commit issue, you should use RabbitMQ for communication between microservices. It is a quick and reliable mode of communication between microservices. It is also helpful if you want to implement a job and worker mechanism. You can push the jobs into RabbitMQ and that will be sent to the consumer. It is highly reliable so you won't miss any jobs and you can also implement a retry of jobs with the dead letter queue feature. It will be also helpful in time-consuming API. You can put time-consuming items into a queue so they will be processed later and your API will be quick.
Read full review
Pros
Apache
  • In charge of the ETL processes.
  • As there is no incoming or outgoing data, we may handle the scheduling of tasks as code and avoid the requirement for monitoring.
Read full review
Open Source
  • What RabbitMQ does well is what it's advertised to do. It is good at providing lots of high volume, high availability queue. We've seen it handle upwards of 10 million messages in its queues, spread out over 200 queues before its publish/consume rates dipped. So yeah, it can definitely handle a lot of messages and a lot of queues. Depending on the size of the machine RabbitMQ is running on, I'm sure it can handle more.
  • Decent number of plugins! Want a plugin that gives you an interface to view all the queues and see their publish/consume rates? Yes, there's one for that. Want a plugin to "shovel" messages from one queue to another in an emergency? Check. Want a plugin that does extra logging for all the messages received? Got you covered!
  • Lots of configuration possibilities. We've tuned over 100 settings over the past year to get the performance and reliability just right. This could be a downside though--it's pretty confusing and some settings were hard to understand.
Read full review
Cons
Apache
  • they should bring in some time based scheduling too not only event based
  • they do not store the metadata due to which we are not able to analyze the workflows
  • they only support python as of now for scripted pipeline writing
Read full review
Open Source
  • It breaks communication if we don't acknowledge early. In some cases our work items are time consuming that will take a time and in that scenario we are getting errors that RabbitMQ broke the channel. It will be good if RabbitMQ provides two acknowledgements, one is for that it has been received at client side and second ack is client is completed the processing part.
Read full review
Usability
Apache
Easy to learn Easy to use Robust workflow orchestration framework Good in dependent job management
Read full review
Open Source
RabbitMQ is very usable if you are a programmer or DevOps engineer. You can setup and configure a messaging system without any programmatic knowledge either through an admin console plugin or through a command-line interface. It's very easy to spin up additional consumers when volume is heavy and it's very easy to manage those consumers either through automated scripting or through their admin console. Because it's language agnostic it integrates with any system supporting AMQP.
Read full review
Support Rating
Apache
No answers on this topic
Open Source
RabbitMQ is more software than service so there's no real customer service to speak of unless you go with a provider such as CloudAMQP. So I'll just speak on CloudAMQP. Their customer support is only okay: they only do it over email. They frequently gloss over our support tickets and half answer them without delving deeply or investigating our issues. Their response times are pretty reasonable though.
Read full review
Alternatives Considered
Apache
There are a number of reasons to choose Apache Airflow over other similar platforms- Integrations—ready-to-use operators allow you to integrate Airflow with cloud platforms (Google, AWS, Azure, etc) Apache Airflow helps with backups and other DevOps tasks, such as submitting a Spark job and storing the resulting data on a Hadoop cluster It has machine learning model training, such as triggering a Sage maker job.
Read full review
Open Source
RabbitMQ has a few advantages over Azure Service Bus 1) RMQ handles substantially larger files - ASB tops out at 100MB, we use RabbitMQfor files over 200MB 2) RabbitMQ can be easily setup on prem - Azure Service Bus is cloud only 3) RabbitMQ exchanges are easier to configure over ASB subscriptions ASB has a few advantages too 1) Cloud based - just a few mouse clicks and you're up and running
Read full review
Return on Investment
Apache
  • A lot of helpful features out-of-the-box, such as the DAG visualizations and task trees
  • Allowed us to implement complex data pipelines easily and at a relatively low cost
Read full review
Open Source
  • Earlier we had a problem with missing work items with our own implementation but later using RabbitMQ is solved a problem. Now our job processing mechanism is highly reliable.
  • We also had a problem with scaling, processing 1k work items per second. RabbitMQ helped us to scale well with increasing work items.
Read full review
ScreenShots