Apache CouchDB vs. Apache Kafka

Great for REST API development, if you want a small, fast server that will send and receive JSON structures, CouchDB is hard to beat. Not great for enterprise-level relational database querying (no kidding). While by definition, document-oriented databases are not relational, porting or migrating from relational, and using CouchDB as a backend is probably not a wise move as it's reliable, but It may not always be highly available.

Incentivized

Apache Kafka is well-suited for most data-streaming use cases. Amazon Kinesis and Azure EventHubs, unless you have a specific use case where using those cloud PaAS for your data lakes, once set up well, Apache Kafka will take care of everything else in the background. Azure EventHubs, is good for cross-cloud use cases, and Amazon Kinesis - I have no real-world experience. But I believe it is the same.

• It can replicate and sync with web browsers via PouchDB. This lets you keep a synced copy of your database on the client-side, which offers much faster data access than continuous HTTP requests would allow, and enables offline usage.
• Simple Map/Reduce support. The M/R system lets you process terabytes of documents in parallel, save the results, and only need to reprocess documents that have changed on subsequent updates. While not as powerful as Hadoop, it is an easy to use query system that's hard to screw up.
• Sharding and Clustering support. As of CouchDB 2.0, it supports clustering and sharding of documents between instances without needing a load balancer to determine where requests should go.
• Master to Master replication lets you clone, continuously backup, and listen for changes through the replication protocol, even over unreliable WAN links.

Incentivized

• Really easy to configure. I've used other message brokers such as RabbitMQ and compared to them, Kafka's configurations are very easy to understand and tweak.
• Very scalable: easily configured to run on multiple nodes allowing for ease of parallelism (assuming your queues/topics don't have to be consumed in the exact same order the messages were delivered)
• Not exactly a feature, but I trust Kafka will be around for at least another decade because active development has continued to be strong and there's a lot of financial backing from Confluent and LinkedIn, and probably many other companies who are using it (which, anecdotally, is many).

Incentivized

• NoSQL DB can become a challenge for seasoned RDBMS users.
• The map-reduce paradigm can be very demanding for first-time users.
• JSON format documents with Key-Value pairs are somewhat verbose and consume more storage.

Incentivized

• Sometimes it becomes difficult to monitor our Kafka deployments. We've been able to overcome it largely using AWS MSK, a managed service for Apache Kafka, but a separate monitoring dashboard would have been great.
• Simplify the process for local deployment of Kafka and provide a user interface to get visibility into the different topics and the messages being processed.
• Learning curve around creation of broker and topics could be simplified

Because our current solution S3 is working great and CouchDB was a nightmare. The worst is that at first, it seemed fine until we filled it with tons of data and then started to create views and actually delete.

Incentivized

Kafka is quickly becoming core product of the organization, indeed it is replacing older messaging systems. No better alternatives found yet

Incentivized

Couchdb is very simple to use and the features are also reduced but well implemented. In order to use it the way its designed, the ui is adequate and easy. Of course, there are some other task that can't be performed through the admin ui but the minimalistic design allows you to use external libraries to develop custom scripts

Incentivized

Apache Kafka is highly recommended to develop loosely coupled, real-time processing applications. Also, Apache Kafka provides property based configuration. Producer, Consumer and broker contain their own separate property file

Incentivized

Support for Apache Kafka (if willing to pay) is available from Confluent that includes the same time that created Kafka at Linkedin so they know this software in and out. Moreover, Apache Kafka is well known and best practices documents and deployment scenarios are easily available for download. For example, from eBay, Linkedin, Uber, and NYTimes.

Incentivized

it support is minimal also hw requirements. Also for development, we can have databases replicated everywhere and the replication is automagical. once you set up the security and the rules for replication, you are ready to go. The absence of a model let you build your app the way you want it

Incentivized

It has been 5+ years since we chose CouchDB. We looked an MongoDB, Cassandra, and probably some others. At the end of the day, the performance, power potential, and simplicity of CouchDB made it a simple choice for our needs. No one should use just because we did. As I said early, make sure you understand your problems, and find the right solution. Some random reading that might be useful: http://www.julianbrowne.com/article/viewer/brewers-cap-theorem https://www.couchbase.com/nosql-resources/why-nosql\ https://www.infoq.com/articles/cap-twelve-years-later-how-the-rules-have-changed

Incentivized

I used other messaging/queue solutions that are a lot more basic than Confluent Kafka, as well as another solution that is no longer in the market called Xively, which was bought and "buried" by Google. In comparison, these solutions offer way fewer functionalities and respond to other needs.

Incentivized

• It has saved us hours and hours of coding.
• It is has taught us a new way to look at things.
• It has taught us patience as the first few weeks with CouchDB were not pleasant. It was not easy to pick up like MongoDB.

Incentivized

• Positive: Get a quick and reliable pub/sub model implemented - data across components flows easily.
• Positive: it's scalable so we can develop small and scale for real-world scenarios
• Negative: it's easy to get into a confusing situation if you are not experienced yet or something strange has happened (rare, but it does). Troubleshooting such situations can take time and effort.

Incentivized