Apache Kafka is an open-source stream processing platform developed by the Apache Software Foundation written in Scala and Java. The Kafka event streaming platform is used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications.
N/A
New Relic
Score 8.1 out of 10
N/A
New Relic is a SaaS-based web and mobile application performance management provider for the cloud and the datacenter. They provide code-level diagnostics for dedicated infrastructures, the cloud, or hybrid environments and real time monitoring.
$0
No credit card required; 100 GB free ingest per month, 1 free full user + unlimited basic users, 8 days retention, 100 Synthetics Checks
Pricing
Apache Kafka
New Relic
Editions & Modules
No answers on this topic
Free (Forever)
$0
No credit card required; 100 GB free ingest per month, 1 free full user + unlimited basic users, 8 days retention, 100 Synthetics Checks
Telemetry Data Platform
$0.25
per month per extra GB data ingest (after first free 100GB per month)
Incident Intelligence
$0.50
per month per event (after first 1000 free events per month)
Standard
$99
per month per full user (after first free full user - unlimited free basic users)
Apache Kafka is well-suited for most data-streaming use cases. Amazon Kinesis and Azure EventHubs, unless you have a specific use case where using those cloud PaAS for your data lakes, once set up well, Apache Kafka will take care of everything else in the background. Azure EventHubs, is good for cross-cloud use cases, and Amazon Kinesis - I have no real-world experience. But I believe it is the same.
New - relic is well suited if you want to analyse the performance of your services and you want to improve it. Integration with multiple services with same account gives a clear picture of flow of your APIs if you have micro-service architecture. New-relic is less appropriate when you want to do logging of your system. As it does not emits every single calls
Really easy to configure. I've used other message brokers such as RabbitMQ and compared to them, Kafka's configurations are very easy to understand and tweak.
Very scalable: easily configured to run on multiple nodes allowing for ease of parallelism (assuming your queues/topics don't have to be consumed in the exact same order the messages were delivered)
Not exactly a feature, but I trust Kafka will be around for at least another decade because active development has continued to be strong and there's a lot of financial backing from Confluent and LinkedIn, and probably many other companies who are using it (which, anecdotally, is many).
gives us an monitoring of all our underlying servers and also we can configure some alerts upon them like CPU and memory alerts.
Kubernetes cluster monitoring with new relic for EKS gives us and minute details of our cluster utilisation like node usage, pods memory request and limits
Network traceability for each and every request with response time analysis is great we can trace which component is responsible for generating response delay
log managements of the logs the infrastructure is generating we can view logs through there only
Sometimes it becomes difficult to monitor our Kafka deployments. We've been able to overcome it largely using AWS MSK, a managed service for Apache Kafka, but a separate monitoring dashboard would have been great.
Simplify the process for local deployment of Kafka and provide a user interface to get visibility into the different topics and the messages being processed.
Learning curve around creation of broker and topics could be simplified
I would like to see sort of simulator inside the user interface, that way we can send requests directly from it to test some configuration instead of setting up a test environment in our end.
It would be nice if the data ingestion can be filtered by APM's. That way we can know which application is ingested most data.
It would be nice if we could ingest logs (apache, system logs, and other logs) and correlate them with the APM.
The only issue that we have had with New Relic is that the price might be a little expensive for smaller companies. The amount of data you store in New Relic impacts the cost, and can get away from you if you don't work closely with the vendor. Overall though the application is top notch.
Apache Kafka is highly recommended to develop loosely coupled, real-time processing applications. Also, Apache Kafka provides property based configuration. Producer, Consumer and broker contain their own separate property file
As an engineer, New Relic has been very quick and easy for me to pick up/install/use. It has been less easy for some of the less technical-minded folks in our organization and their UI still is inconsistent multiple years after refactoring their platform to be New Relic One.
Support for Apache Kafka (if willing to pay) is available from Confluent that includes the same time that created Kafka at Linkedin so they know this software in and out. Moreover, Apache Kafka is well known and best practices documents and deployment scenarios are easily available for download. For example, from eBay, Linkedin, Uber, and NYTimes.
The support team has been really helpful and resolved most of the issues on time. However, for a couple of issues, several follow-ups were needed to elicit a reasonable response. The issue was deeply technical and could have been investigated only by their Architects, and bringing them into the ticket took longer than needed
It's better to start by implementing New Relic in one project and test everything. Try to follow best recommended practices and read all the official documentation. Everything seems well tested. Then, start by installing agents to the rest of your projects and keep a close look to all logs and metrics New Relic gives you.
I used other messaging/queue solutions that are a lot more basic than Confluent Kafka, as well as another solution that is no longer in the market called Xively, which was bought and "buried" by Google. In comparison, these solutions offer way fewer functionalities and respond to other needs.
New Relic is the most full-featured offering that we've found, and is incredibly easy to start using with a PHP app. The New Relic agent is installed as a PHP extension so it is able to monitor and track the performance of any PHP app being run by the web server. Other tools required the installation and setup of a PHP dependency at the application level.
Positive: Get a quick and reliable pub/sub model implemented - data across components flows easily.
Positive: it's scalable so we can develop small and scale for real-world scenarios
Negative: it's easy to get into a confusing situation if you are not experienced yet or something strange has happened (rare, but it does). Troubleshooting such situations can take time and effort.
