Apache Kafka, the F1 of messaging
Updated March 01, 2018
Apache Kafka, the F1 of messaging
Score 9 out of 10
Vetted Review
Verified User
Overall Satisfaction with Apache Kafka
Apache Kafka is becoming the new standard for messaging at our organization. Originally we limited the use to big data environments and projects but as the technology is becoming more mature we think it will eventually replace classical messaging software.
Pros
- High volume/performance throughput environments
- Low latency projects
- Multiple consumers for the same data, reprocessing, long-lasting information
Cons
- Still a bit inmature, some clients have required recoding in the last few versions
- New feaures coming very fast, several upgrades a year may be required
- Not many commercial companies provide support
- Easier deployment and horizontal scalability
- Messaging cost reduction
- Developments require adaptation and some paradigm shift to interoperate with Kafka
- IBM MQ and TIBCO EMS
Kafka is faster and more scalable, also "free" as opensource (albeit we deploy using a commercial distribution). Infrastructure tends to be cheaper. On the other hand, projects must adapt to Kafka APIs that sometimes change and BAU increases until a major 1.x version comes out and adds stability to the product.
Using Apache Kafka
20 - Kafka is core for several business/technical functions:
- Data streaming: ingest data into Datalake, process information near real-time
- Log processing: able to hold logs from all the company applications both to process and as transport them to a final storage (like timeseries DB, Elasticsearch and so on)
- Reliable messaging now that exactly-one-delivery semantics have been implemented
and so on
- Data streaming: ingest data into Datalake, process information near real-time
- Log processing: able to hold logs from all the company applications both to process and as transport them to a final storage (like timeseries DB, Elasticsearch and so on)
- Reliable messaging now that exactly-one-delivery semantics have been implemented
and so on
Developers with deep knowledge of stream processing, otherwise your organization will not use all the capabilities.
Operators with DevOps skills. Kafka, even in version 1.0, is still a bit inmature and lacks of proper adminitration tools (apart from those from 3rd party like Cloudera, Hortonworks, Confluent, Lenses and so on) so hands on scripts and detailled monitoring of the platform is a must.
Operators with DevOps skills. Kafka, even in version 1.0, is still a bit inmature and lacks of proper adminitration tools (apart from those from 3rd party like Cloudera, Hortonworks, Confluent, Lenses and so on) so hands on scripts and detailled monitoring of the platform is a must.
- Application technical log processing
- Realtime transaction analysis
- Messaging as a Service for PaaS and CaaS applications
- Several data hubs: technical, business, social...
Comments
Please log in to join the conversation