Apache Kafka is an open-source stream processing platform developed by the Apache Software Foundation written in Scala and Java. The Kafka event streaming platform is used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications.
N/A
NGINX
Score 9.1 out of 10
Mid-Size Companies (51-1,000 employees)
NGINX, a business unit of F5 Networks, powers over 65% of the world's busiest websites and web applications. NGINX started out as an open source web server and reverse proxy, built to be faster and more efficient than Apache. Over the years, NGINX has built a suite of infrastructure software products o tackle some of the biggest challenges in managing high-transaction applications. NGINX offers a suite of products to form the core of what organizations need to create…
I selected Nginx because it is easy to learn, use, and maintain. I almost never have to troubleshoot it ever since I deploy it. It just meets my need for a personal learning experience. It works well with PHP and MySQL on Linux. That is why I chose it at first.
Apache Kafka is well-suited for most data-streaming use cases. Amazon Kinesis and Azure EventHubs, unless you have a specific use case where using those cloud PaAS for your data lakes, once set up well, Apache Kafka will take care of everything else in the background. Azure EventHubs, is good for cross-cloud use cases, and Amazon Kinesis - I have no real-world experience. But I believe it is the same.
[NGINX] is very well suited for high performance. I have seen it used on servers with 1k current connections with no issues. Despite seeing it used in many environments I've never seen software developers use it over apache, express, IIS in local dev environments so it may be more difficult to setup. I've also seen it used to load balance again without issues.
Really easy to configure. I've used other message brokers such as RabbitMQ and compared to them, Kafka's configurations are very easy to understand and tweak.
Very scalable: easily configured to run on multiple nodes allowing for ease of parallelism (assuming your queues/topics don't have to be consumed in the exact same order the messages were delivered)
Not exactly a feature, but I trust Kafka will be around for at least another decade because active development has continued to be strong and there's a lot of financial backing from Confluent and LinkedIn, and probably many other companies who are using it (which, anecdotally, is many).
Sometimes it becomes difficult to monitor our Kafka deployments. We've been able to overcome it largely using AWS MSK, a managed service for Apache Kafka, but a separate monitoring dashboard would have been great.
Simplify the process for local deployment of Kafka and provide a user interface to get visibility into the different topics and the messages being processed.
Learning curve around creation of broker and topics could be simplified
Customer support can be strangely condescending, perhaps it's a language issue?
I find it a little weird how the release versions used for Nginx+ aren't the same as for open source version. It can be very confusing to determine the cross-compatibility of modules, etc., because of this.
It seems like some (most?) modules on their own site are ancient and no longer supported, so their documentation in this area needs work.
It's difficult to navigate between nginx.com commercial site and customer support. They need to be integrated together.
I'd love to see more work done on nginx+ monitoring without requiring logging every request. I understand that many statistics can only be derived from logs, but plenty should work without that. Logging is not an option in many environments.
Apache Kafka is highly recommended to develop loosely coupled, real-time processing applications. Also, Apache Kafka provides property based configuration. Producer, Consumer and broker contain their own separate property file
Front end proxy and reverse proxy of Nginx is always useful. I always prefer to Nginx in overall usability when you have application server and database or multiple application servers and single database i.e. clustered application. Nginx provides really good features and flexibility which helps the system administrator in case of troubleshooting and also from the administration perspective. Also, Nginx doesn't delay any request because of internal performance issues.
Support for Apache Kafka (if willing to pay) is available from Confluent that includes the same time that created Kafka at Linkedin so they know this software in and out. Moreover, Apache Kafka is well known and best practices documents and deployment scenarios are easily available for download. For example, from eBay, Linkedin, Uber, and NYTimes.
Community support is great, and they've also had a presence at conferences. Overall, there is no shortage of documentation and community support. We're currently using it to serve up some WordPress sites, and configuring NGINX for this purpose is well documented.
I used other messaging/queue solutions that are a lot more basic than Confluent Kafka, as well as another solution that is no longer in the market called Xively, which was bought and "buried" by Google. In comparison, these solutions offer way fewer functionalities and respond to other needs.
We have used Traffic, Apache, Google Cloud Load Balancing and other managed cloud-based load balancers. When it comes to scale and customization nothing beats Nginx. We selected Nginx over the others because
we have a large number of services and we can manage a single Nginx instance for all of them
we have high impact services and Nginx never breaks a sweat under load
individual services have special considerations and Nginx lets us configure each one uniquely
Positive: Get a quick and reliable pub/sub model implemented - data across components flows easily.
Positive: it's scalable so we can develop small and scale for real-world scenarios
Negative: it's easy to get into a confusing situation if you are not experienced yet or something strange has happened (rare, but it does). Troubleshooting such situations can take time and effort.
Nginx has decreased the burden of web server administration and maintenance, and we are spending less time on server issues than when we were using Apache.
Nginx has allowed more people in our company to get involved with configuring things on the web server, so there's no longer a single point of failure ("the Apache guy").
Nginx has given us the ability to handle a larger number of requests without scaling up in hardware quite so quickly.
