Apache Kafka is an open-source stream processing platform developed by the Apache Software Foundation written in Scala and Java. The Kafka event streaming platform is used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications.
N/A
Firebase
Score 8.4 out of 10
N/A
Google offers the Firebase suite of application development tools, available free or at cost for higher degree of usages, priced flexibly accorded to features needed. The suite includes A/B testing and Crashlytics, Cloud Messaging (FCM) and in-app messaging, cloud storage and NoSQL storage (Cloud Firestore and Firestore Realtime Database), and other features supporting developers with flexible mobile application development.
Apache Kafka is well-suited for most data-streaming use cases. Amazon Kinesis and Azure EventHubs, unless you have a specific use case where using those cloud PaAS for your data lakes, once set up well, Apache Kafka will take care of everything else in the background. Azure EventHubs, is good for cross-cloud use cases, and Amazon Kinesis - I have no real-world experience. But I believe it is the same.
Firebase should be your first choice if your platform is mobile first. Firebase's mobile platform support for client-side applications is second to none, and I cannot think of a comparable cross-platform toolkit. Firebase also integrates well with your server-side solution, meaning that you can plug Firebase into your existing app architecture with minimal effort.
Firebase lags behind on the desktop, however. Although macOS support is rapidly catching up, full Windows support is a glaring omission for most Firebase features. This means that if your platform targets Windows, you will need to implement the client functionality manually using Firebase's web APIs and wrappers, or look for another solution.
Really easy to configure. I've used other message brokers such as RabbitMQ and compared to them, Kafka's configurations are very easy to understand and tweak.
Very scalable: easily configured to run on multiple nodes allowing for ease of parallelism (assuming your queues/topics don't have to be consumed in the exact same order the messages were delivered)
Not exactly a feature, but I trust Kafka will be around for at least another decade because active development has continued to be strong and there's a lot of financial backing from Confluent and LinkedIn, and probably many other companies who are using it (which, anecdotally, is many).
Analytics wise, retention is extremely important to our app, therefore we take advantage of the cohort analysis to see the impact of our middle funnel (retargeting, push, email) efforts affect the percent of users that come back into the app. Firebase allows us to easily segment these this data and look at a running average based on certain dates.
When it comes to any mobile app, a deep linking strategy is essential to any apps success. With Firebase's Dynamic Links, we are able to share dynamic links (recognize user device) that are able to redirect to in-app content. These deep links allow users to share other deep-linked content with friends, that also have link preview assets.
Firebase allows users to effectively track events, funnels, and MAUs. With this simple event tracking feature, users can put organize these events into funnels of their main user flows (e.g., checkout flows, onboarding flows, etc.), and subsequently be able to understand where the drop-off is in the funnel and then prioritize areas of the funnel to fix. Also, MAU is important to be able to tell if you are bringing in new users and what's the active volume for each platform (Android, iOS).
Sometimes it becomes difficult to monitor our Kafka deployments. We've been able to overcome it largely using AWS MSK, a managed service for Apache Kafka, but a separate monitoring dashboard would have been great.
Simplify the process for local deployment of Kafka and provide a user interface to get visibility into the different topics and the messages being processed.
Learning curve around creation of broker and topics could be simplified
Attribution and specifically multi-touch attribution could be more robust such as Branch or Appsflyer but understand this isn't Firebases bread and butter.
More parameters. Firebase allows you to track tons of events (believe it's up to 50 or so) but the parameters of the events it only allows you to track 5 which is so messily and unbelievable. So you're able to get good high-level data but if you want to get granular with the events and actions are taken on your app to get real data insight you either have to go with a paid data analytics platform or bring on someone that's an expert in SQL to go through Big Query.
City-specific data instead of just country-specific data would have been a huge plus as well.
Apache Kafka is highly recommended to develop loosely coupled, real-time processing applications. Also, Apache Kafka provides property based configuration. Producer, Consumer and broker contain their own separate property file
It is simple to use overall, the console's main menu is divided into Develop, Quality, Analytics and Grow - which have further subdivisions by their set of features and tools. Develop and Quality are relevant for product and tech. Analytics is relevant for product, analytics and Grow is relevant for marketing. This makes the overall use very easy.
Support for Apache Kafka (if willing to pay) is available from Confluent that includes the same time that created Kafka at Linkedin so they know this software in and out. Moreover, Apache Kafka is well known and best practices documents and deployment scenarios are easily available for download. For example, from eBay, Linkedin, Uber, and NYTimes.
Our analytics folks handled the majority of the communication when it came to customer service, but as far as I was aware, the support we got was pretty good. When we had an issue, we were able to reach out and get support in a timely fashion. Firebase was easy to reach and reasonably available to assist when needed.
I used other messaging/queue solutions that are a lot more basic than Confluent Kafka, as well as another solution that is no longer in the market called Xively, which was bought and "buried" by Google. In comparison, these solutions offer way fewer functionalities and respond to other needs.
Before using Firebase, we exclusively used self hosted database services. Using Firebase has allowed us to reduce reliance on single points of failure and systems that are difficult to scale. Additionally, Firebase is much easier to set up and use than any sort of self hosted database. This simplicity has allowed us to try features that we might not have based on the amount of work they required in the past.
Positive: Get a quick and reliable pub/sub model implemented - data across components flows easily.
Positive: it's scalable so we can develop small and scale for real-world scenarios
Negative: it's easy to get into a confusing situation if you are not experienced yet or something strange has happened (rare, but it does). Troubleshooting such situations can take time and effort.
Makes building real-time interfaces easy to do at scale with no backend involvement.
Very low pricing for small companies and green-fields projects.
Lack of support for more complicated queries needs to be managed by users and often forces strange architecture choices for data to enable it to be easily accessed.
