Hadoop: A Robust Big Data Platform
Hadoop is being used to solve big data modeling problems in our firm. The corporate analytics team uses Hadoop to perform functions like …
Hadoop
Apache Hadoop
Overview
Recent Reviews
Reviewer Pros & Cons
View all pros & cons
Kunal Sonalkar
Data Research Analyst
Southwest Florida Water Management District (Higher Education, 5001-10,000 employees)
Video Reviews
Leaving a video review helps other professionals like you evaluate products. Be the first one in your network to record a review of Hadoop, and make your voice heard!
Pricing
View all pricingN/A
Unavailable
What is Hadoop?
Hadoop is an open source software from Apache, supporting distributed processing and data storage. Hadoop is popular for its scalability, reliability, and functionality available across commoditized hardware.
Entry-level set up fee?
- No setup fee
Offerings
- Free Trial
- Free/Freemium Version
- Premium Consulting / Integration Services
Would you like us to let the vendor know that you want pricing?
8 people want pricing too
Features Scorecard
No scorecards have been submitted for this product yet.Start a Scorecard.
Product Details
What is Hadoop?
Hadoop is an open source software from Apache, supporting distributed processing and data storage. Hadoop is popular for its scalability, reliability, and functionality available across commoditized hardware.
Hadoop Video
What is Hadoop?
Hadoop Integrations
- Sematext Infrastructure Monitoring (formerly Sematext SPM)
Hadoop Technical Details
| Operating Systems | Unspecified |
|---|---|
| Mobile Application | No |
Comparisons
View all alternatives
Frequently Asked Questions
What is Hadoop?
Hadoop is an open source software from Apache, supporting distributed processing and data storage. Hadoop is popular for its scalability, reliability, and functionality available across commoditized hardware.
What is Hadoop's best feature?
Reviewers rate Data Sources highest, with a score of 8.7.
Who uses Hadoop?
The most common users of Hadoop are from Enterprises (1,001+ employees) and the Information Technology & Services industry.
Reviews and Ratings
(267)
Reviews
(1-25 of 37)- Popular Filters
Companies can't remove reviews or game the system. Here's why
April 11, 2022
Hadoop: A Robust Big Data Platform
Hadoop is being used to solve big data modeling problems in our firm. The corporate analytics team uses Hadoop to perform functions like data manipulation, information retrieval, data mapping, and statistical modeling. The business problem which it solves is the limitation of CSV/Excel files to handle more than a million rows. Hadoop allows you to process big data and also has connectivity with platforms like R Studio where you can deploy mathematical models.
- Capability to collaborate with R Studio. Most of the statistical algorithms can be deployed.
- Handling Big Data issues like storage, information retrieval, data manipulation, etc.
- Redundant tasks like data wrangling, data processing, and cleaning are more efficient in Hadoop as the processing times are faster.
- Hadoop requires intensive computational platforms like a minimum of 8GB memory and i5 processor. Sometimes the hardware does become a hindrance.
- If we can connect Hadoop to Salesforce, it would be a tremendous functionality as most CRM data comes from that channel.
- It will be good to have some Geo Coding features if someone wants to opt for spatial data analysis using latitudes and longitudes.
August 17, 2021
Great enterprise tool for handling large data
Apache Hadoop is one of the most effective and efficient software which has been storing and processing an extremely colossal amount of data in my company for a long time now. The software Hadoop is primarily used for data collection of large amounts, storage as well as for analytics. From my experience, I have to say that Hadoop is extremely useful and has a reliable plus valid purpose.
- The various modules sometimes are pretty challenging to learn but at the same time, it has made Hadoop easy to implement and perform.
- Hadoop comprises a thoughtful file system which is called as Hadoop Distributed File System that beautifully processes all components and programs.
- Hadoop is also very easy to install so this is also a great aspect of Hadoop as sometimes the installation process is so tricky that the user loses interest.
- Customer support is quick.
- As much as I really appreciate Hadoop there are certain cons attached to it as well. I personally think that Hadoop should work attentively towards their interactive querying platforms which in my opinion is quite slow as compared to other players available in the market.
- Apart from that, a con that I have noticed is that there are many modules that exist in Hadoop so due to the higher number of modules it becomes difficult and time-consuming to learn and ace all of them.
July 21, 2021
Good tool for unstructured data
Apache Hadoop is an open-source software library that is designed for the collection, storage, and analysis of large amounts of data sets. Apache Hadoop’s architecture comprises components that include a distributed file system. This is mostly used for massive data collection, analytics, and storage. Also, having consistent data can be integrated across other platforms and have one single source of truth.
- Apache Hadoop has made managing large amounts of data quite easy.
- The system contains a file system known as HDFS (Hadoop Distributed File System) which processes components and programs.
- The parallel processing tool of this software is also a good aspect of Apache Hadoop.
- It keeps interesting and reliable features and functions.
- Apache Hadoop also has a store of very big data files in machines with high levels of availability.
- I personally feel that Apache Hadoop is slower as compared to other interactive querying platforms. Queries can take up to hours sometimes which can be frustrating and discouraging sometimes.
- Also, there are so many modules of Apache Hadoop so it takes so much more time to learn all of them. Other than that, optimization is somewhat a challenge in Apache Hadoop.
We use Apache Hadoop to store and process large amounts of data (petabytes per day) across thousands of data pipelines. Hadoop works reliably for this purpose. Data scientists at the company also use it for interactive querying for analytics and modeling purposes.
- Storing large amounts of data
- Processing large amounts of data via a familiar SQL interface
- Slower than other interactive querying engines. Queries take minutes at least and up to hours sometimes
- Tuning the settings to be able to run certain queries can require a lot of domain knowledge
January 16, 2021
Apache Hadoop Can Save on the Headaches
[Apache Hadoop] is being handled as it is (mostly) intended. For large, unstructured data management from our data flows to include logging and reports extract, transform and load. We are using it at a medium scale in an on-prem server delivery with Cloudera as the management platform. While I firmly believe cloudera makes it a bit easier to manage, it obfuscates issues at times.
- Handles large amounts of unstructured data well, for business level purposes
- Is a good catchall because of this design, i.e. what does not fit into our vertical tables fits here.
- Decent for large ETL pipelines and logging free-for-alls because of this, also.
- Many, many modules and because of Apache open source, takes time to learn
- Integration is not always seamless between the disparate pieces nor are all the pieces required.
- Optimization can be challenging (see PSTL design)
September 21, 2020
Hadoop -- Great Value for What You Pay
It's used organization-wide for older data that's not used as frequently. We use Teradata to warehouse our more recent data, but for data we don't access as often, it's migrated to Hadoop. It addresses the problem of securely storing data without paying the fortune that most warehouses charge for premium cloud storage.
- Accessible
- Inexpensive
- User friendly
- Much slower than more premium platforms
- Doesn't connect with other data warehouses
- Not mainstream -- somewhat more, "hacky" of a solution
September 20, 2020
Fault Tolerance and High Availablility Made Easy with Hadoop
We are using it within my department to process large sets of data that can't be processed in a timely fashion on a single computer or node. The various modules provided with Hadoop make it easy for us to implement map-reduce and perform parallel processing on large sets of data. We have approximately 40TB of data that we run various algorithms against as we try to use the data to solve business problems and prevent fraudulent transactions.
- Map-reduce
- Parallel processing
- Handles node failures
- HDFS: distributed file system
- More connectors
- Query optimization
- Job scheduling
December 07, 2019
Hadoop provides enterprise performance and scale at a low cost
We leverage Hadoop for several of our Tier 1 applications. We use Hadoop for our enterprise data lake where all of the data that our company takes in from our members is stored and then a lot of our applications use that as their master datasource. We also leverage Hadoop clusters for Paxata and Data Scientist analytics workloads. Basically, anything that requires a scale-out approach, we put on Hadoop.
- Scale.
- Stability.
- Reliability.
- There are a lot of Hadoop-specific services and applications under the hood that you have to learn how to administer across the Hadoop cluster.
- Enterprise-class support doesn't live up to other third-party vendors.
June 05, 2019
Hadoop vs. Alternatives
It is being used at our Fortune 500 clients. It is great for storage, but it is not well understood by the business. The challenge is that it requires very sophisticated data scientists to use properly and in parallel, but the data scientists turn the data on its head, causing IT execution issues. This has forced IT to restructure data in a denormalized form so the business users can actually be productive. This is a big trend in organizations.
- Great for inexpensive storage, when originally introduced.
- Distributed processing
- Industry standard
- Network fabric needs to be more sophisticated.
- Need centralized storage.
- The three copy of data should have been in the original design, not years later.
- Consider deploying Spectrum Scale in these environments.
May 16, 2018
Hadoop Review
It is massively being used in our organization for data storage, data backup, and machine learning analytics. Managing vast amounts of data has become quite easy since the arrival of the Hadoop environment. Our department is on verge of moving towards Spark instead of MapReduce, but for now, Hadoop is being used extensively for MapReduce purposes.
- Hadoop Distributed Systems is reliable.
- High scalability
- Open Sources, Low Cost, Large Communities
- Compatibility with Windows Systems
- Security needs more focus
- Hadoop lack in real time processing
March 28, 2018
Great Option for Unstructured Data
- Used for Massive data collection, storage, and analytics
- Used for MapReduce processes, Hive tables, Spark job input, and for backing up data
- Storing Retail Catalog & Session data to enable omnichannel experience for customers, and a 360-degree customer insight
- Having a consistent data store that can be integrated across other platforms, and have one single source of truth.
- HDFS is reliable and solid, and in my experience with it, there are very few problems using it
- Enterprise support from different vendors makes it easier to 'sell' inside an enterprise
- It provides High Scalability and Redundancy
- Horizontal scaling and distributed architecture
- Less organizational support system. Bugs need to be fixed and outside help take a long time to push updates
- Not for small data sets
- Data security needs to be ramped up
- Failure in NameNode has no replication which takes a lot of time to recover
January 04, 2018
Hadoop is pretty Badass
Apache Hadoop is a cost effective solution for storing and managing vast amounts of data efficiently. It is dependable and works even when various clusters fail. The Hadoop Distributed File System (HDFS) also goes a long way in helping in storing data. MapReduce and Tez, with the help of Hive of course, processes large amounts of data in a lesser time frame than expected. This helps our data warehouse to be updated with lesser resources rather than reading, processing and updating data in a relational data base.
- It is cost effective.
- It is highly scalable.
- Failure tolerant.
- Hadoop does not fit all needs.
- Converting data into a single format takes time.
- Need to take additional security measures to secure data.
December 13, 2017
Hadoop: Highly available, scalable and cost effective for big data storage and processing.
Currently, there are two directorates using Hadoop for processing a vast amount of data from various data sources in my organization. Hadoop helps us tackle our problem of maintaining and processing a huge amount of data efficiently. High availability, scalability and cost efficiency are the main considerations for implementing Hadoop as one of the core solutions in our big-data infrastructure.
- Scalability is one of the main reasons we decided to use Hadoop. Storage and processing power can be seamlessly increased by simply adding more nodes.
- Replication on Hadoop's distributed file system (HDFS) ensures robustness of data being stored which ensures high-availability of data.
- Using commodity hardware as a node in a Hadoop cluster can reduce cost and eliminates dependency on particular proprietary technology.
- User and access management are still challenging to implement in Hadoop, deploying a kerberized secured cluster is quite a challenge itself.
- Multiple application versioning on a single cluster would be a nice to have feature.
- Processing a large number of small files also becomes a problem on a very large cluster with hundreds of nodes.
October 24, 2017
Hadoop for Justifying Business Decisions with Hard Data
Hadoop has been an amazing development in the world of Big Data. Where relational databases fall short with regard to tuning and performance, Hadoop rises to the occasion and allows for massive customization leveraging the different tools and modules. We use Hadoop to input raw data and add layers of consolidation or analysis to make business decisions about disparate datapoints.
- Hadoop can take loads of data quickly and performs well under load.
- Hadoop is customizable so that nearly any business objective can be justified with the right combination of data and reports.
- Hadoop has a lot of great resources, both informal like the community and formal like the supported modules and training.
- Hadoop is not a relational database, but it has the ability to add modules to run sql-like queries like Impala and Hive.
- Hadoop is open source and has many modules. It can be difficult without context to know which modules to leverage.
September 22, 2017
Hadoop review 2346
Hadoop is used to build a data lake where all enterprise data for my entire company can be stored. With data centralization and standardization we use it to build analytical solutions for our company. There are many other uses for the data - for example monitoring performance via KPIs, etc.
- Massive data processing
- Fault tolerance
- Speed to market
- Data visualization
- Data history
- Random access
August 24, 2017
Hadoop for Big Data
[It was used] As a proof of concept to analyze a huge amount of data. We were building a product to analyze huge data and eventually sell that product to a utility.
- Highly Scalable Architecture
- Low cost
- Can be used in a Cloud Environment
- Can be run on commodity Hardware
- Open Source
- Its open source but there are companies like hortonworks, Cloudera etc., which give enterprise support
- Lots of scripting still needed
- Some tools in the hadoop eco system overlap
June 03, 2016
A newbie's look at Hadoop
We needed a robust/redundant system to run multiple simultaneous jobs for our ETL pipeline, this needed distributed storage space, integration with Windows AD user accounts and the ability to expand when needed with little to no downtime.
We are using Cloudera 5.6 to orchestrate the install (along with puppet) and manage the hadoop cluster.
We are using Cloudera 5.6 to orchestrate the install (along with puppet) and manage the hadoop cluster.
- The distributed replicated HDFS filesystem allows for fault tolerance and the ability to use low cost JBOD arrays for data storage.
- Yarn with MapReduce2 gives us a job slot scheduler to fully utilize available compute resources while providing HA and resource management.
- The hadoop ecosystem allows for the use of many different technologies all using the same compute resources so that your spark, samza, camus, pig and oozie jobs can happily co-exist on the same infrastructure.
- Without Cloudera as a management interface the hadoop components are much harder to manage to ensure consistency across a cluster.
- The calculations of hardware resources to job slots/resource management can be quite an exercise in finding that "sweet spot" with your applications, a more transparent way of figuring this out would be welcome.
- A lot of the roles and management pieces are written in java, which from an administration perspective can have there own issues with garbage collection and memory management.
May 26, 2016
Experience with Hadoop by a novice user.
Hadoop is not used as a norm in my organization. I just use it personally to complete my job faster. It is implemented in the research computing cluster to be used by faculty and students. It completes jobs faster by parallelizing the tasks using MapReduce framework. This gives me considerable speed in the tasks I perform.
- Provides a reliable distributed storage to store and retrieve data. I am able to store data without having to worry that a node failing might cause the loss of data.
- Parallelizes the task with MapReduce and helps complete the task faster. The ease of use of MapReduce makes it possible to write code in a simple way to make it run on different slaves in the cluster.
- With the massive user base, it is not hard to find documentation or help relating to any problem in the area. Therefore, I rarely had any instances where I had to look for a solution for a really long time.
- I would have hoped for a simpler interface if possible, so that the initial effort that had to be spent would have been much less. I often see others who are starting to use hadoop are finding it hard to learn.
- I'm not sure if it is a problem with the organization and the modules they provide, but sometimes I wish there were more modules available to be used.
The company I worked at used Hadoop clusters for processing huge datasets. They had several nodes for both production and per-production nodes. It allowed distributed processing of data across several clusters with an easy to use software model. It is used by the Systems and IT department at my company.
- HDFS provides a very robust and fast data storage system.
- Hadoop works well with generic "commodity" hardware negating the need for expensive enterprise grade hardware.
- It is mostly unaffected by system and hardware failures of nodes and is self-sustained.
- While its open source nature provides a lot of benefits, there are multiple stability issues that arise due to it.
- Limited support for interactive analytics.
February 23, 2016
Hadoop quick review
We have Hadoop pre-prod and prod clusters. Production clusters are comprised of 200 nodes. And we have realtime clusters as well. All the data will be moved to Hadoop. We use Hadoop to do machine learning and data warehousing.
- Machine Learning Model, when SAS can not process 3 of years data. Hadoop is good tool to build the model.
- Data warehousing is also another good use case. Using Teradata is expensive.
- A lot of people are not from a programming background which makes Hue very important for end users when starting the Hadoop journey. Making Hue more user friendly and functional will be helpful for end users who don't much of a programming background.
February 16, 2016
Apache Hadoop is the best open source product I used.
My present company uses Hadoop and associated technology to create a data pipeline using open source tools. Apart from that we also consult for projects which could potentially use Hadoop. Apart from that, I also work as a consultant for HDP. We actively help in installation and setup of hadoop clusters.
- Hadoop is open source and with a wide community already present, the usage is much easy for individuals, startups and MNCs alike.
- Hadoop works well for commodity hardware and that makes it easier to avoid pricey clusters.
- Hadoop takes parallel programming to next level and helps processing of multi terabytes (even petabytes) of data easier.
- While Hadoop MR parallelizes jobs involving Big Data, it is slow for smaller data sets
- OLAP (analytics)is easier, however, OLTP (transactions) is a problem in most cases.
- People using Hadoop have to keep in mind that small proof of concepts may not scale as expected.
February 14, 2016
Hadoop an awesome tool for large scale batch processing.
I have been working with Hadoop since last year. It is very user friendly. Hadoop was used by the data center management team. It allows distributed processing of huge amount of data sets across clusters of computers using simple programming models.
- It is robust in the sense that any big data applications will continue to run even when individual servers fail.
- Enormous data can be easily sorted.
- It can be improved in terms of security.
- Since it is open source, stability issues must be improved.
December 09, 2015
Hadoop - best data optimization for the Enterprise
My organization uses Apache Hadoop for log analysis/data mining of data fetched from different practices in the US, Canada and India. It uses this data for showing analytical graphs and the progress of our software in those regions. Data from the practices is optimized and consumed by the customer applications. It provides faster performance and ease for data usage.
- Hadoop is a very cost effective storage solution for businesses’ exploding data sets.
- Hadoop can store and distribute very large data sets across hundreds of servers that operate, therefore it is a highly scalable storage platform.
- Hadoop can process terabytes of data in minutes and faster as compared to other data processors.
- Hadoop File System can store all types of data, structured and unstructured, in nodes across many servers
- For now, Hadoop is doing great and is very productive.
December 04, 2015
A very generic Hadoop review
We utilize Hadoop primarily as a large data staging area for disparate corporate data. Select data is aggregated and moved downstream to a more formal data warehouse. Some data analytics is also performed directly against the Hadoop stored data. The direct analytics is done primarily with Apache Spark utilizing Scala and Python.
- No requirement for schema on write.
- Ability to scale to massive amounts of data.
- Open platform provides multiple options and customizations to fit your exact needs.
- The platform is still maturing and can be confusing to research and use. Basic tasks can still be manual and are not always user friendly.
December 01, 2015
Hadoop - Effective tool for large scale distributed processing.
I have used Hadoop for building business feeds for a telecom client. The major purpose for using Hadoop was to tackle the problem of gaining insights into the ever growing number of business data. We leveraged the map reduce programming model to churn more than 30 gigabytes of data per day into actionable and aggregated data which was further leveraged by campaign teams to design and shape marketing and by product teams to envision new customer experiences.
- Hadoop is an excellent framework for building distributed, fault tolerant data processing systems which leverage HDFS which is optimized for low latency storage and high throughput performance.
- Hadoop Map reduce is a powerful programming model and can be leveraged directly either via use of Java programming language or by data flow languages like Apache Pig.
- Hadoop has a reach eco system of companion tools which enable easy integration for ingesting large amounts of data efficiently from various sources. For example Apache Flume can act as data bus which can use HDFS as a sink and integrates effectively with disparate data sources.
- Hadoop can also be leveraged to build complex data processing and machine learning workflows, due to availability of Apache Mahout, which uses the map reduce model of Hadoop to run complex algorithms.
- Hadoop is a batch oriented processing framework, it lacks real time or stream processing.
- Hadoop's HDFS file system is not a POSIX compliant file system and does not work well with small files, especially smaller than the default block size.
- Hadoop cannot be used for running interactive jobs or analytics.