Apache Kafka vs. Firebase

For brokering messages, Confluent Kafka is well suited since it offers a managed solution ready to use. Scenarios where the solution is not very well suited are for example, where pricing is an issue. The solution costs quite a lot for basic usage (for example: for 3 clusters, pricing is above 100k$ a year).

Incentivized

Firebase should be your first choice if your platform is mobile first. Firebase's mobile platform support for client-side applications is second to none, and I cannot think of a comparable cross-platform toolkit. Firebase also integrates well with your server-side solution, meaning that you can plug Firebase into your existing app architecture with minimal effort.

Firebase lags behind on the desktop, however. Although macOS support is rapidly catching up, full Windows support is a glaring omission for most Firebase features. This means that if your platform targets Windows, you will need to implement the client functionality manually using Firebase's web APIs and wrappers, or look for another solution.

Incentivized

• Apache Kafka is able to handle a large number of I/Os (writes) using 3-4 cheap servers.
• It scales very well over large workloads and can handle extreme-scale deployments (eg. Linkedin with 300 billion user events each day).
• The same Kafka setup can be used as a messaging bus, storage system or a log aggregator making it easy to maintain as one system feeding multiple applications.

Incentivized

• Extremely robust. Has about any tool you can think of under one roof making it extremely useful as a backup platform for data analytics or small teams that need something quickly.
• Intuitive and easy UI/UX. Being made and owned by Google, you expect nothing less. Very easy to use for anyone that has any marketing or analytical experience especially in Google Analytics (which I just assume all marketers do).
• Safe, secure, and sturdy. Never need to worry about downtimes or misinformation as it's as clean and safe as it is being run by Google.
• FREE! What else is there to say. Unless you're an extremely large application handling hundreds of thousands to millions of users, this pay as you go plan will stay free.

Incentivized

• The Kafka Tool is a community-made Java application that looks and feels from the past century.
• Logging can be confusing. This certainly shows when we have to do troubleshooting.
• Hybrid scenarios - pub/sub, but there are services in and outside a Kubernetes cluster. Then there are a ~3 options, but only 2 (the harder ones) are production-safe.

Incentivized

• Firebase/Firestore has very limited support for querying more complicated items; for example, performing a simple string search is not possible.
• While upfront costs are low, costs can grow quickly if you're not careful about what you are being billed for.
• Dashboards have at times shown different information to what is billed, and support from Google is less than stellar and not as effective as that from Amazon or Microsoft.

Incentivized

Kafka has suited our use case very well so far. Going forward we are planning to expand our platform manifold so the load on Kafka and our reliance on Kafka is going to increase only.

Apache Kafka is highly recommended to develop loosely coupled, real-time processing applications. Also, Apache Kafka provides property based configuration. Producer, Consumer and broker contain their own separate property file

Incentivized

I don't use the Firebase UI much, but rather connect it to GA4. GA4 has a great event model but the GA4 UI and analysis capabilities are limited. It's harder to measure product usage type of engagement but if you have the time and resources to leverage the GA4 to BiqQuery export you'll have all the raw event data you'll need for deep analysis, segmentation, and audience activation.

Incentivized

Support for Apache Kafka (if willing to pay) is available from Confluent that includes the same time that created Kafka at Linkedin so they know this software in and out. Moreover, Apache Kafka is well known and best practices documents and deployment scenarios are easily available for download. For example, from eBay, Linkedin, Uber, and NYTimes.

Incentivized

Our analytics folks handled the majority of the communication when it came to customer service, but as far as I was aware, the support we got was pretty good. When we had an issue, we were able to reach out and get support in a timely fashion. Firebase was easy to reach and reasonably available to assist when needed.

Incentivized

Apache Kafka is built for scale. From high throughput and real-time data streaming, it has a strong advantage over RabbitMQ with its low latency. This put Apache Kafka at the forefront as the platform of choice for large datasets messaging and ensuring scalability when data scale up tremendously. RabbitMQ however has its strengths in traditional messaging. Routing and message delivery reliability are the bedrock of RabbitMQ and this is where RabbitMQ excels. In my previous workplace, RabbitMQ was of choice as reliability matters more than scale. In two words. Apache Kafka for scale, RabbitMQ for reliability. And for cloud deployment and large dataset messaging in what I am doing now, Apache Kafka is the default choice.

Before using Firebase, we exclusively used self hosted database services. Using Firebase has allowed us to reduce reliance on single points of failure and systems that are difficult to scale. Additionally, Firebase is much easier to set up and use than any sort of self hosted database. This simplicity has allowed us to try features that we might not have based on the amount of work they required in the past.

Incentivized

• Positive: bursts of traffic on special holidays are easy to handle because Kafka can absorb and buffer all the messages we need to process long enough to let an understaffed set of back-end services catch up on processing. Hard to put a number to it but we probably save $5k a month having fewer machines running.
• Positive: makes decoupling the web and API services from the deeper back-end services easier by providing topics as an interface. This allowed us to split up our teams and have them develop independently of each other, speeding up software development.
• Negative: our engineers have made mistakes such as accidentally dropping a few thousand messages due to the CLI being confusing to use, and as a result a customer lost some of their precious data. I'd say that was more our fault than Kafka's though.

Incentivized

• Firebase has been able to help us understand reliably, the drop-off in our user flows with their funnel feature. This has made it easy for us to be able to pinpoint weaknesses in our funnel and test and optimize with data as the dependent variable.
• From an economic standpoint, we don't pay for Firebase which is great, but as the saying goes "You get what you pay for" also holds true in this context. As we looked to grow and scale, we looked for a paid solution.
• From a developer resource standpoint, Firebase has been extremely easy to integrate into our app. Whether it be the event tracking, dynamic links or crash reporting we have not had to waste too much developer time thanks to their well-organized developer docs.

Incentivized