Apache Hive vs. Splunk Enterprise

Software work execution is on a large scale, it is good to use for new projects or organizational changes, data lineage mapping has always been dubious but this one has had good results. You can store and synchronize data from different departments, the storage process can be manual but it is best automated.

Incentivized

Pros: Splunk is very well suited if you have multiple log sources of related data. All of them can be correlated and tasks can be automated based on the requirement. Other than alerts, Splunk can also run a specific script of your choice, based on some defined conditions. Cons: If you have a few logs but a large number of log sources, Splunk can be very expensive.

Incentivized

• Apache Hive allows use to write expressive solutions to complex problems thanks to its SQL-like syntax.
• Relatively easy to set up and start using.
• Very little ramp-up to start using the actual product, documentation is very thorough, there is an active community, and the code base is constantly being improved.

Incentivized

• Real-time + Scheduled alerts - i-e you can set up alerts which are actively monitoring your logs
• Pretty good response time for search results. With our key/value logging, Splunk makes it blazing fast to query the data.
• Dashboards provide insights into historical data
• Love how Splunk indexes all of the data and provides keys to search on

Incentivized

• Some queries, particularly complex joins, are still quite slow and can take hours
• Previous jobs and queries are not stored sometimes
• Switching to Impala can sometimes be time-consuming (i.e. the system hangs, or is slow to respond).
• Sometimes, directories and tables don't load properly which causes confusion

Incentivized

• At times some queries can run slowly if indices are not on a portion of the query you use.
• Setup time initially can be difficult if your logs aren't stored in common locations or in a common way to write the log.
• Ability to ingest logs from different locations without having to change code to put logs in a certain place (pro and con).
• Searches can be a bit more difficult to look through if your log isn't pulled in a manner that is easy to read through splunk.

Incentivized

Since I do not know the second data warehouse solution that integrate with HDFS as well as Hive.

We are using Splunk extensively in our projects and we have recently upgraded to Splunk version 6.0 which is quite efficient and giving expected results. We keep track of updates and new features Splunk introduces periodically and try to introduce those features in our day to day activities for improvement in our reporting system and other tasks.

Hive is a very good big data analysis and ad-hoc query platform, which supports scaling also. The BI processes can be easily integrated with Hadoop via the Hive. It can deal with a much larger data set that traditional RDBMS can not. It is a "must-have" component of the big data domain.

Incentivized

You can literally throw in a single word into Splunk and it will pull back all instances of that word across all of your logs for the time span you select (provided you have permission to see that data). We have several users who have taken a few of the free courses from Splunk that are able to pull data out of it everyday with little help at all.

Incentivized

When properly setup and configured, Splunk is extremely reliable.

Incentivized

Apache Hive is a FOSS project and its open source. We need not definitely comment on anything about the support of open source and its developer community. But, it has got tremendous developer support, awesome documentation. I would justify the fact that much support can be gathered from the community backup.

Incentivized

Splunk maintains a well resourced support system that has been consistent since we purchased the product. They help out in a timely manner and provide expert level information as needed. We typically open cases online and communicate when possible via e-mail and are able to resolve most issues with that method.

Incentivized

The online course was simple clear and described the main capabilities of the solution. There is also an initial module that can be done for free so anyone can familiarize themselves with the functionality of this solution. On the other hand, however, there could be more free online courses. Maybe even with a certificate, this would broaden the group of people who are familiar with the platform while increasing familiarity with the solution itself.

Smooth without too many major issues.

Incentivized

Besides Hive, I have used Google BigQuery, which is costly but have very high computation speed. Amazon Redshift is the another product, I used in my recent organisation. Both Redshift and BigQuery are managed solution whereas Hive needs to be managed

Incentivized

I wanted to learn a new language that I can quickly master and implement. Splunk is easy, fun to use and best of all, it can be developed in hours not days or weeks. Splunk is fundamentally a programming language that is minimal but yet powerful enough to collect, analyze and visualize data.

Incentivized

Splunk can scale in to the petabyte per day range which of course is awesome

• Apache hive is secured and scalable solution that helps in increasing the overall organization productivity.
• Apache hive can handle and process large amount of data in a sufficient time manner.
• It simplifies writing SQL queries, hence helping the organization as most companies use SQL for all query jobs.

Incentivized

• Overall very positive. It has provided visibility to what is going on within our network.
• One drawback is the time it takes to get up to speed with the application, but this is up to the user, and Splunk education is excellent.
• In my field, IT Security, there are few other friends to have in your back pocket better than Splunk. They are just that good.

Incentivized