Qlik Replicate enables organizations to accelerate data real-time replication, ingestion and streaming via change data capture, across a wide range of heterogeneous databases, data warehouses and data lake platforms.
N/A
RabbitMQ
Score 9.0 out of 10
N/A
RabbitMQ, an open source message broker, is part of Pivotal Software, a VMware company acquired in 2019, and supports message queue, multiple messaging protocols, and more.
RabbitMQ is available open source, however VMware also offers a range of commercial services for RabbitMQ; these are available as part of the Pivotal App Suite.
Qlik Replicate works very well with relational data platforms, both on premise and in the cloud, for example Oracle, SQL Server, PostgreSQL, MySQL and others, it also works very well with DB2. If the data source is MongoDB, it is more complicated and currently there is no possibility of sending data to MongoDB.
It is highly recommended that if you have microservices architecture and if you want to solve 2 phase commit issue, you should use RabbitMQ for communication between microservices. It is a quick and reliable mode of communication between microservices. It is also helpful if you want to implement a job and worker mechanism. You can push the jobs into RabbitMQ and that will be sent to the consumer. It is highly reliable so you won't miss any jobs and you can also implement a retry of jobs with the dead letter queue feature. It will be also helpful in time-consuming API. You can put time-consuming items into a queue so they will be processed later and your API will be quick.
Replicate is extremely stable and does not generate a lot of alerts/failures/issues that take up time to troubleshoot.
It is very easy to add new source tables to a Replicate task so that we're always in sync with new data available from the CRM.
It's nice that Qlik Replicate also allows you to create a job to stop and then restart your tasks during maintenance windows that occur on both the source and target systems.
What RabbitMQ does well is what it's advertised to do. It is good at providing lots of high volume, high availability queue. We've seen it handle upwards of 10 million messages in its queues, spread out over 200 queues before its publish/consume rates dipped. So yeah, it can definitely handle a lot of messages and a lot of queues. Depending on the size of the machine RabbitMQ is running on, I'm sure it can handle more.
Decent number of plugins! Want a plugin that gives you an interface to view all the queues and see their publish/consume rates? Yes, there's one for that. Want a plugin to "shovel" messages from one queue to another in an emergency? Check. Want a plugin that does extra logging for all the messages received? Got you covered!
Lots of configuration possibilities. We've tuned over 100 settings over the past year to get the performance and reliability just right. This could be a downside though--it's pretty confusing and some settings were hard to understand.
The base Replicate web GUI is lacking. If you have dozens or more tasks, it's hard to get a sense of how they're performing. Enterprise Manager solves all of these problems but is a separate install.
The support portal is extremely difficult to navigate. It's hard to track down exactly what you're looking for.
It would be helpful to have better documentation and example queries for the tables in the Enterprise Manager analytics database.
Destination databases that don't support common DDL commands behave unpredictably. And the replication of schema changes isn't consistent.
It breaks communication if we don't acknowledge early. In some cases our work items are time consuming that will take a time and in that scenario we are getting errors that RabbitMQ broke the channel. It will be good if RabbitMQ provides two acknowledgements, one is for that it has been received at client side and second ack is client is completed the processing part.
The availability of the replicated data in disparate environments has is now crucial. Replacing a product like Qlik Replicate would require significant time, investments, and work. In addition, Qlik Replicate is reasonably reliable with few failures.
We now have greater business flexibility and scalability, and our big data integration projects have a quick rate of growth, which has been profitable for us. Independent of the sources involved, maintaining data consistency between sources is easy. One of my favorite features is the way it lets owners of the source system start and stop processes from updating their system windows.
RabbitMQ is very easy to configure for all supported languages (Python, Java, etc.). I have personally used it on Raspberry Pi devices via a Flask Python API as well as in Java applications. I was able to learn it quickly and now have full mastery of it. I highly recommend it for any IoT project.
The issue I've had is that Qlik does an awful job of keeping their customers informed when new versions of the software are available. We found that we were using a version that was no longer supported and could never get help. When it came time to get us upgraded so that we were on a current version, no one knew how to help get us to where we needed to be. We had to purchased professional services time and even then I was basically on my own to get everything built out and set up. Qlik needs to be more proactive with communicating about new releases and how to get your version upgraded in the most secure, safe way possible.
I gave it a 10 but we do not have a support contract with any company for RabbitMQ so there is no official support in that regard. However, there is a community and questions asked on StackOverflow or any other major question and answer site will usually get a response.
Follow the directions from the Qlik documentation. They are pretty straight forward and easy enough to follow. If you follow these, then you are not likely to have issues on implementation.
Great tool for data replication solution for Oracle/SQLServers/etc. Real easy to get it set up and start realizing business value. Getting the PoC accomplished in a short window. Product costing and easy to start small and scale as needed. It helped cover most of our ask compared to other solutions.
RabbitMQ has a few advantages over Azure Service Bus 1) RMQ handles substantially larger files - ASB tops out at 100MB, we use RabbitMQfor files over 200MB 2) RabbitMQ can be easily setup on prem - Azure Service Bus is cloud only 3) RabbitMQ exchanges are easier to configure over ASB subscriptions ASB has a few advantages too 1) Cloud based - just a few mouse clicks and you're up and running
Prior to using Qlik Replicate, we used an ETL solution to copy data from the Oracle ERP system to the Microsoft SQL Server BI system at a 15-minute interval. It was very tedious to maintain. Qlik Replicate is much easier to use and we replicate data near real-time now.
Positive: we don't need to keep way too many backend machines around to deal with bursts because RabbitMQ can absorb and buffer bursts long enough to let an understaffed set of backend services to catch up on processing. Hard to put a number to it but we probably save $5k a month having fewer machines around.
Negative: we've got many angry customers due to queues suddenly disappearing and dropping our messages when we try to publish to them afterward. Ideally, RabbitMQ should warn the user when queues expire due to inactivity but it doesn't, and due to our own bugs we've lost a lot of customer data as a result.
Positive: makes decoupling the web and API services from the deeper backend services easier by providing queues as an interface. This allowed us to split up our teams and have them develop independently of each other, speeding up software development.
