Apache Kafka

Apache Kafka is really the bedrock of all things streaming and data processing. I cannot imagine if there is any other product that does it better. My last 2 companies used it, and my current one does so as well. If you want your data stream to be organized and sent, Apache Kafka has become the tool of choice. I have dabbled in Azure EventHubs as well, if you are into opensource data streaming, Apache Kafka will take you where you need to be for data lakes and the amount of data that is streamed for the cybersecurity industry that my company is in. Without Apache Kafka, there is no way that my company products can handle the volume of data that we process for our customers.

We use Apache Kafka as an event bus for all our async activities & Micro Service Communication, like sending emails, SMS, and notifications between services and consumers and for event & data processing.

We use Apache Kafka as message broker between our two client facing applications. We used ActiveMQ before but it had shortfalls of high availability and clustering. Kafka solved it on both fronts and gives a good business continuity.

We use Apache Kafka to stream order information across systems. An order may go through certain updates through its lifecycle. These updates need to be communicated to the systems in near real time and we rely on Kafka for this.Our business use case is to take these orders up with the insurance companies for approval and thus the order information need to be up to date. Kafka has been excellent at doing this so far.

Currently consulting and implementing for a bank, we use a cloud-native Kafka solution (Confluent Kafka) for brokering. The solution is well documented, and liked by the developers but lacks certain technical aspects to improve usability and administration.