Databricks offers the Databricks Lakehouse Platform (formerly the Unified Analytics Platform), a data science platform and Apache Spark cluster manager. The Databricks Unified Data Service provides a platform for data pipelines, data lakes, and data platforms.
$0.07
Per DBU
Splunk Enterprise
Score 8.6 out of 10
N/A
Splunk is software for searching, monitoring, and analyzing machine-generated big data, via a web-style interface. It captures, indexes and correlates real-time data in a searchable repository from which it can generate graphs, reports, alerts, dashboards and visualizations.
We have also used ELK (Elastic Logstash Kibana) with some benefits, but Splunk is way better than ELK. We also use AWS CloudWatch for Lambdas that are written in AWS. However CloudWatch is not a replacement for Splunk.
Medium to Large data throughput shops will benefit the most from Databricks Spark processing. Smaller use cases may find the barrier to entry a bit too high for casual use cases. Some of the overhead to kicking off a Spark compute job can actually lead to your workloads taking longer, but past a certain point the performance returns cannot be beat.
It's well suited for what I do, which is network security operations. And that's for anything from troubleshooting incidents, troubleshooting performance, troubleshooting for the purpose of a compliance and auditing. It's not best suited for users who are new in terms of they're new to the product and they have expectations that probably Splunk cannot meet.
We are using Splunk extensively in our projects and we have recently upgraded to Splunk version 6.0 which is quite efficient and giving expected results. We keep track of updates and new features Splunk introduces periodically and try to introduce those features in our day to day activities for improvement in our reporting system and other tasks.
Because it is an amazing platform for designing experiments and delivering a deep dive analysis that requires execution of highly complex queries, as well as it allows to share the information and insights across the company with their shared workspaces, while keeping it secured.
in terms of graph generation and interaction it could improve their UI and UX
You can literally throw in a single word into Splunk and it will pull back all instances of that word across all of your logs for the time span you select (provided you have permission to see that data). We have several users who have taken a few of the free courses from Splunk that are able to pull data out of it everyday with little help at all.
One of the best customer and technology support that I have ever experienced in my career. You pay for what you get and you get the Rolls Royce. It reminds me of the customer support of SAS in the 2000s when the tools were reaching some limits and their engineer wanted to know more about what we were doing, long before "data science" was even a name. Databricks truly embraces the partnership with their customer and help them on any given challenge.
Splunk maintains a well resourced support system that has been consistent since we purchased the product. They help out in a timely manner and provide expert level information as needed. We typically open cases online and communicate when possible via e-mail and are able to resolve most issues with that method.
The online course was simple clear and described the main capabilities of the solution. There is also an initial module that can be done for free so anyone can familiarize themselves with the functionality of this solution. On the other hand, however, there could be more free online courses. Maybe even with a certificate, this would broaden the group of people who are familiar with the platform while increasing familiarity with the solution itself.
The most important differentiating factor for Databricks Lakehouse Platform from these other platforms is support for ACID transactions and the time travel feature. Also, native integration with managed MLflow is a plus. EMR, Cloudera, and Hortonworks are not as optimized when it comes to Spark Job Execution. Other platforms need to be self-managed, which is another huge hassle.
I didn't get to fully evaluate Logstash as our corporation was already using Logstash, but both seemed like viable solutions to the problem that we were having. I wanted to evaluate Logstash some more, both did seem like they would work for the business needs that we had, we went with splunk as many teams were already using it.
I don't have any numbers to share but Splunk has positively served as a 24/7 monitoring tool that has saved hours of work by self-detecting, saving statistics and alerting problems in the system or from external interfaces as soon as they happen.
Splunk dashboards does a solid job in collecting, analyzing data and creating reports that contain an entire day's activity and then automatically sent out to the business.
Splunk is very easy to learn and very useful to any program or business application.
