Apache Kafka is an open-source stream processing platform developed by the Apache Software Foundation written in Scala and Java. The Kafka event streaming platform is used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications.
N/A
Red Hat OpenShift
Score 9.2 out of 10
N/A
OpenShift is Red Hat's Cloud Computing Platform as a Service (PaaS) offering. OpenShift is an application platform in the cloud where application developers and teams can build, test, deploy, and run their applications.
Apache Kafka is well-suited for most data-streaming use cases. Amazon Kinesis and Azure EventHubs, unless you have a specific use case where using those cloud PaAS for your data lakes, once set up well, Apache Kafka will take care of everything else in the background. Azure EventHubs, is good for cross-cloud use cases, and Amazon Kinesis - I have no real-world experience. But I believe it is the same.
Red Hat OpenShift, despite its complexity and overhead, remains the most complete and enterprise-ready Kubernetes platform available. It excels in research projects like ours, where we need robust CI/CD, GPU scheduling, and tight integration with tools like Jupyter, OpenDataHub, and Quiskit. Its security, scalability, and operator ecosystem make it ideal for experimental and production-grade AI workloads. However, for simpler general hosting tasks—such as serving static websites or lightweight backend services—we find traditional VMs, Docker, or LXD more practical and resource-efficient. Red Hat OpenShift shines in complex, container-native workflows, but can be overkill for basic infrastructure needs.
Really easy to configure. I've used other message brokers such as RabbitMQ and compared to them, Kafka's configurations are very easy to understand and tweak.
Very scalable: easily configured to run on multiple nodes allowing for ease of parallelism (assuming your queues/topics don't have to be consumed in the exact same order the messages were delivered)
Not exactly a feature, but I trust Kafka will be around for at least another decade because active development has continued to be strong and there's a lot of financial backing from Confluent and LinkedIn, and probably many other companies who are using it (which, anecdotally, is many).
We had a few microservices that dealt with notifications and alerts. We used OpenShift to deploy these microservices, which handle and deliver notifications using publish-subscribe models.
We had to expose an API to consumers via MTLS, which was implemented using Server secret integration in OpenShift. We were then able to deploy the APIs on OpenShift with API security.
We integrated Splunk with OpenShift to view the logs of our applications and gain real-time insights into usage, as well as provide high availability.
Sometimes it becomes difficult to monitor our Kafka deployments. We've been able to overcome it largely using AWS MSK, a managed service for Apache Kafka, but a separate monitoring dashboard would have been great.
Simplify the process for local deployment of Kafka and provide a user interface to get visibility into the different topics and the messages being processed.
Learning curve around creation of broker and topics could be simplified
I wouldn't necessarily say there is look everyday technology transform. I can see a trend wherein Red Hat OpenShift is adopting all the new technology trends and helping their customers align with their priorities and the emerging technology trends. I wouldn't call out various scope for development every day. There is scope for development. It is all how the organizations adopt it and how they deliver it to their customers. I don't want to call out there is scope for development. It's happening. It is a never ending process.
At the moment, I don't have anything to call out. We are experiencing Red Hat OpenShift and we can see every day they're coming up with new features as and when they come up with new features, we want to experience it more and more. We are looking for opportunities wherein this can be leveraged to help our users and partners.
OpenShift is really easy of use through its management console. OpenShift gives a very large flexibility through many inbuilt functionalities, all gathered in the same place (it's a very convenient tool to learn DevOps technics hands on) OpenShift is an ideal integrated development / deployment platform for containers
Apache Kafka is highly recommended to develop loosely coupled, real-time processing applications. Also, Apache Kafka provides property based configuration. Producer, Consumer and broker contain their own separate property file
The virtualization part takes some getting used to it you are coming from a more traditional hypervisor. Customization options are not intuitive to these users. The process should be more clear. Perhaps a guide to Openshift Virtualization for users of RHV, VMware, etc. would ease this transition into the new platform
Redhat openshift is generally reliable and available platform, it ensures high availability for most the situations. in fact the product where we put openshift in a box, we ensure that the availability is also happening at node and network level and also at storage level, so some of the factors that are outside of Openshift realm are also working in HA manner.
Overall, this platform is beneficial. The only downsides we have encountered have been with pods that occasionally hang. This results in resources being dedicated to dead or zombie pods. Over time, these wasted resources occasionally cause us issues, and we have had difficulty monitoring these pods. However, this issue does not overshadow the benefits we get from Openshift.
Support for Apache Kafka (if willing to pay) is available from Confluent that includes the same time that created Kafka at Linkedin so they know this software in and out. Moreover, Apache Kafka is well known and best practices documents and deployment scenarios are easily available for download. For example, from eBay, Linkedin, Uber, and NYTimes.
Every time we need to get support all the Red Hat team move forward looking to solve the problem. Sometimes this was not easy and requires the scalation to product team, and we always get a response. Most of the minor issues were solved with the information from access.redhat.com
I was not involved in the in person training, so i can not answer this question, but the team in my org worked directly with Openshift and able to get the in person training done easily, i did not hear problem or complain in this space, so i hope things happen seamlessly without any issue.
We went thru the training material on RH webesite, i think its very descriptive and the handson lab sesssions are very useful. It would be good to create more short duration videos covering one single aspect of openshift, this wll keep the interest and also it breaks down the complexity to reasonable chunks.
I used other messaging/queue solutions that are a lot more basic than Confluent Kafka, as well as another solution that is no longer in the market called Xively, which was bought and "buried" by Google. In comparison, these solutions offer way fewer functionalities and respond to other needs.
The Tanzu Platform seemed overly complicated, and the frequent changes to the portfolio as well as the messaging made us uneasy. We also decided it would not be wise to tie our application platform to a specific infrastructure provider, as Tanzu cannot be deployed on anything other than vSphere. SUSE Rancher seemed good overall, but ultimately felt closer to a DIY approach versus the comprehensive package that Red Hat OpenShift provides.
It's easy to understand what are being billed and what's included in each type of subscription. Same with the support (Std or Premium) you know exactly what to expect when you need to use it. The "core" unit approach on the subscription made really simple to scale and carry the workloads from one site to another.
This is a great platform to deployment container applications designed for multiple use cases. Its reasonably scalable platform, that can host multiple instances of applications, which can seamlessly handle the node and pod failure, if they are configured properly. There should be some scalability best practices guide would be very useful
Positive: Get a quick and reliable pub/sub model implemented - data across components flows easily.
Positive: it's scalable so we can develop small and scale for real-world scenarios
Negative: it's easy to get into a confusing situation if you are not experienced yet or something strange has happened (rare, but it does). Troubleshooting such situations can take time and effort.
All of the above. Red Hat OpenShift going into a developer-type setting can be stood up very quickly. There's a very short period to have developers onboard to it and they're able to become productive much faster than a grow your own type solution.
