Outdated but still reliable
October 04, 2019
Outdated but still reliable
Score 6 out of 10
Vetted Review
Verified User
Overall Satisfaction with RabbitMQ
We at Iterable use RabbitMQ for two key features: (1) keeping a queue of all the incoming messages that need to be processed with some semblance of order (2) scheduling work to be done later.
The reason we need to keep a queue is that when our traffic spikes, we can have up 1 million messages coming in that need to be processed in some form or fashion. To expect the backend service to support that is crazy. Instead, we dump them into RabbitMQ to give our backend service time to process them. As for scheduling work, the use case is to give our customers control over when they want some form of work to be done. What we do is we store their work request into a RabbitMQ delay queue with a set expiration. When it expires, it expires into the normal queues where our backend service will process them.
The reason we need to keep a queue is that when our traffic spikes, we can have up 1 million messages coming in that need to be processed in some form or fashion. To expect the backend service to support that is crazy. Instead, we dump them into RabbitMQ to give our backend service time to process them. As for scheduling work, the use case is to give our customers control over when they want some form of work to be done. What we do is we store their work request into a RabbitMQ delay queue with a set expiration. When it expires, it expires into the normal queues where our backend service will process them.
- What RabbitMQ does well is what it's advertised to do. It is good at providing lots of high volume, high availability queue. We've seen it handle upwards of 10 million messages in its queues, spread out over 200 queues before its publish/consume rates dipped. So yeah, it can definitely handle a lot of messages and a lot of queues. Depending on the size of the machine RabbitMQ is running on, I'm sure it can handle more.
- Decent number of plugins! Want a plugin that gives you an interface to view all the queues and see their publish/consume rates? Yes, there's one for that. Want a plugin to "shovel" messages from one queue to another in an emergency? Check. Want a plugin that does extra logging for all the messages received? Got you covered!
- Lots of configuration possibilities. We've tuned over 100 settings over the past year to get the performance and reliability just right. This could be a downside though--it's pretty confusing and some settings were hard to understand.
- So why did I only give it a 5? Well, there is almost no observability out of the box. Want to see what messages are in a queue? You have to take messages out and re-enqueue them, breaking the order and risking message loss. What to list all the queues? You have to install a janky plugin that only works half the time and takes 5 seconds to refresh.
- Delay queue expiration can cause RabbitMQ to grind to a halt. We've seen that when delay queues have over 1 million messages, they expire in an explosion! Or rather, in an ice-age. RabbitMQ tries to dump all the messages of an expired delay queue at once and it causes memory usage and CPU usage to spike and suddenly RabbitMQ stops accepting new messages into its other queues.
- Positive: we don't need to keep way too many backend machines around to deal with bursts because RabbitMQ can absorb and buffer bursts long enough to let an understaffed set of backend services to catch up on processing. Hard to put a number to it but we probably save $5k a month having fewer machines around.
- Negative: we've got many angry customers due to queues suddenly disappearing and dropping our messages when we try to publish to them afterward. Ideally, RabbitMQ should warn the user when queues expire due to inactivity but it doesn't, and due to our own bugs we've lost a lot of customer data as a result.
- Positive: makes decoupling the web and API services from the deeper backend services easier by providing queues as an interface. This allowed us to split up our teams and have them develop independently of each other, speeding up software development.
- Apache Kafka and Pulsar
Honestly, though we're still trying out Kafka and Pulsar, I'd go with them for message broker and as traffic buffers. We are only still using RabbitMQ because it's hard to transition off after writing tons of code custom-built for RabbitMQ. Kafka is better because it's way more durable and observable. You have less risk of data loss. You can replay messages easily and quickly. I like Pulsar more at a high level (haven't tried operating it myself yet) because it's more naturally scalable. You can easily divide up the work onto multiple Pulsar instances without having to re-architect your code, I think. RabbitMQ on the other hand is really hard to scale horizontally.
Do you think RabbitMQ delivers good value for the price?
Yes
Are you happy with RabbitMQ's feature set?
Yes
Did RabbitMQ live up to sales and marketing promises?
Yes
Did implementation of RabbitMQ go as expected?
Yes
Would you buy RabbitMQ again?
No