Databricks Lakehouse Platform vs. Splunk Enterprise

If you need a managed big data megastore, which has native integration with highly optimized Apache Spark Engine and native integration with MLflow, go for Databricks Lakehouse Platform. The Databricks Lakehouse Platform is a breeze to use and analytics capabilities are supported out of the box. You will find it a bit difficult to manage code in notebooks but you will get used to it soon.

Incentivized

Pros: Splunk is very well suited if you have multiple log sources of related data. All of them can be correlated and tasks can be automated based on the requirement. Other than alerts, Splunk can also run a specific script of your choice, based on some defined conditions. Cons: If you have a few logs but a large number of log sources, Splunk can be very expensive.

Incentivized

• Process raw data in One Lake (S3) env to relational tables and views
• Share notebooks with our business analysts so that they can use the queries and generate value out of the data
• Try out PySpark and Spark SQL queries on raw data before using them in our Spark jobs
• Modern day ETL operations made easy using Databricks. Provide access mechanism for different set of customers

Incentivized

• Real-time + Scheduled alerts - i-e you can set up alerts which are actively monitoring your logs
• Pretty good response time for search results. With our key/value logging, Splunk makes it blazing fast to query the data.
• Dashboards provide insights into historical data
• Love how Splunk indexes all of the data and provides keys to search on

Incentivized

• Connect my local code in Visual code to my Databricks Lakehouse Platform cluster so I can run the code on the cluster. The old databricks-connect approach has many bugs and is hard to set up. The new Databricks Lakehouse Platform extension on Visual Code, doesn't allow the developers to debug their code line by line (only we can run the code).
• Maybe have a specific Databricks Lakehouse Platform IDE that can be used by Databricks Lakehouse Platform users to develop locally.
• Visualization in MLFLOW experiment can be enhanced

Incentivized

• At times some queries can run slowly if indices are not on a portion of the query you use.
• Setup time initially can be difficult if your logs aren't stored in common locations or in a common way to write the log.
• Ability to ingest logs from different locations without having to change code to put logs in a certain place (pro and con).
• Searches can be a bit more difficult to look through if your log isn't pulled in a manner that is easy to read through splunk.

Incentivized

We are using Splunk extensively in our projects and we have recently upgraded to Splunk version 6.0 which is quite efficient and giving expected results. We keep track of updates and new features Splunk introduces periodically and try to introduce those features in our day to day activities for improvement in our reporting system and other tasks.

Because it is an amazing platform for designing experiments and delivering a deep dive analysis that requires execution of highly complex queries, as well as it allows to share the information and insights across the company with their shared workspaces, while keeping it secured.

in terms of graph generation and interaction it could improve their UI and UX

Incentivized

You can literally throw in a single word into Splunk and it will pull back all instances of that word across all of your logs for the time span you select (provided you have permission to see that data). We have several users who have taken a few of the free courses from Splunk that are able to pull data out of it everyday with little help at all.

Incentivized

When properly setup and configured, Splunk is extremely reliable.

Incentivized

One of the best customer and technology support that I have ever experienced in my career. You pay for what you get and you get the Rolls Royce. It reminds me of the customer support of SAS in the 2000s when the tools were reaching some limits and their engineer wanted to know more about what we were doing, long before "data science" was even a name. Databricks truly embraces the partnership with their customer and help them on any given challenge.

Splunk maintains a well resourced support system that has been consistent since we purchased the product. They help out in a timely manner and provide expert level information as needed. We typically open cases online and communicate when possible via e-mail and are able to resolve most issues with that method.

Incentivized

The online course was simple clear and described the main capabilities of the solution. There is also an initial module that can be done for free so anyone can familiarize themselves with the functionality of this solution. On the other hand, however, there could be more free online courses. Maybe even with a certificate, this would broaden the group of people who are familiar with the platform while increasing familiarity with the solution itself.

Smooth without too many major issues.

Incentivized

Compared to Synapse & Snowflake, Databricks provides a much better development experience, and deeper configuration capabilities. It works out-of-the-box but still allows you intricate customisation of the environment. I find Databricks very flexible and resilient at the same time while Synapse and Snowflake feel more limited in terms of configuration and connectivity to external tools.

I wanted to learn a new language that I can quickly master and implement. Splunk is easy, fun to use and best of all, it can be developed in hours not days or weeks. Splunk is fundamentally a programming language that is minimal but yet powerful enough to collect, analyze and visualize data.

Incentivized

Splunk can scale in to the petabyte per day range which of course is awesome

• The ability to spin up a BIG Data platform with little infrastructure overhead allows us to focus on business value not admin
• DB has the ability to terminate/time out instances which helps manage cost.
• The ability to quickly access typical hard to build data scenarios easily is a strength.

Incentivized

• Overall very positive. It has provided visibility to what is going on within our network.
• One drawback is the time it takes to get up to speed with the application, but this is up to the user, and Splunk education is excellent.
• In my field, IT Security, there are few other friends to have in your back pocket better than Splunk. They are just that good.

Incentivized