Apache Flume vs. Apache Pig

Overview
ProductRatingMost Used ByProduct SummaryStarting Price
Apache Flume
Score 7.1 out of 10
N/A
Apache Flume is a product enabling the flow of logs and other data into a Hadoop environment.N/A
Apache Pig
Score 8.4 out of 10
N/A
Apache Pig is a programming tool for creating MapReduce programs used in Hadoop.N/A
Pricing
Apache FlumeApache Pig
Editions & Modules
No answers on this topic
No answers on this topic
Offerings
Pricing Offerings
Apache FlumeApache Pig
Free Trial
NoNo
Free/Freemium Version
NoNo
Premium Consulting/Integration Services
NoNo
Entry-level Setup FeeNo setup feeNo setup fee
Additional Details——
More Pricing Information
Best Alternatives
Apache FlumeApache Pig
Small Businesses

No answers on this topic

No answers on this topic

Medium-sized Companies
Cloudera Manager
Cloudera Manager
Score 9.7 out of 10
Cloudera Manager
Cloudera Manager
Score 9.7 out of 10
Enterprises
IBM Analytics Engine
IBM Analytics Engine
Score 8.9 out of 10
IBM Analytics Engine
IBM Analytics Engine
Score 8.9 out of 10
All AlternativesView all alternativesView all alternatives
User Ratings
Apache FlumeApache Pig
Likelihood to Recommend
8.0
(2 ratings)
8.1
(9 ratings)
Usability
-
(0 ratings)
10.0
(1 ratings)
Support Rating
5.0
(1 ratings)
6.0
(1 ratings)
User Testimonials
Apache FlumeApache Pig
Likelihood to Recommend
Apache
Apache Flume is well suited when the use case is log data ingestion and aggregate only, for example for compliance of configuration management. It is not well suited where you need a general-purpose real-time data ingestion pipeline that can receive log data and other forms of data streams (eg IoT, messages).
Read full review
Apache
Apache Pig is best suited for ETL-based data processes. It is good in performance in handling and analyzing a large amount of data. it gives faster results than any other similar tool. It is easy to implement and any user with some initial training or some prior SQL knowledge can work on it. Apache Pig is proud to have a large community base globally.
Read full review
Pros
Apache
  • Multiple sources of data (sources) and destinations (sinks) that allows you to move data form and to any relevant data storage
  • It is very easy to setup and run
  • Very open to personalization, you can create filters, enrichment, new sources and destinations
Read full review
Apache
  • Its performance, ease of use, and simplicity in learning and deployment.
  • Using this tool, we can quickly analyze large amounts of data.
  • It's adequate for map-reducing large datasets and fully abstracted MapReduce.
Read full review
Cons
Apache
  • It is very specific for log data ingestion so it is pretty hard to use for anything else besides log data
  • Data replication is not built in and needs to be added on top of Apache Flume (not a hard job to do though)
Read full review
Apache
  • UDFS Python errors are not interpretable. Developer struggles for a very very long time if he/she gets these errors.
  • Being in early stage, it still has a small community for help in related matters.
  • It needs a lot of improvements yet. Only recently they added datetime module for time series, which is a very basic requirement.
Read full review
Usability
Apache
No answers on this topic
Apache
It is quick, fast and easy to implement Apache Pig which makes is quite popular to be used.
Read full review
Support Rating
Apache
Apache Flume is open-source so support is limited. Never the less, it has great documentation and best practices documents from their end-users so it is not hard to use, setup and configure.
Read full review
Apache
The documentation is adequate. I'm not sure how large of an external community there is for support.
Read full review
Alternatives Considered
Apache
Apache Flume is a very good solution when your project is not very complex at transformation and enrichment, and good if you have an external management suite like Cloudera, Hortonworks, etc. But it is not a real EAI or ETL like AB Initio or Attunity so
you need to know exactly what you want. On the other hand being an opensource project give Apache a lot of room to personalize thanks to its plug-able architecture and has a very nice performance having a very low CPU and Memory footprint, a single server can do the job on many occasions, as opposed to the multi-server architecture of paid products.
Read full review
Apache
Apache Pig might help to start things faster at first and it was one of the best tool years back but it lacks important features that are needed in the data engineering world right now. Pig also has a steeper learning curve since it uses a proprietary language compared to Spark which can be coded with Python, Java.
Read full review
Return on Investment
Apache
  • Flume has simplified a lot many of our ingest procedures, easier to deploy and integrate than a classical EAI, reducing the time to market
  • But opposed to EAIs if the project starts to grow in complexity Apache Flume project may not be as suitable
Read full review
Apache
  • Higher learning curve than other similar technologies so on-boarding new engineers or change ownership of Apache Pig code tends to be a bit of a headache
  • Once the language is learned and understood it can be relatively straightforward to write simple Pig scripts so development can go relatively quickly with a skilled team
  • As distributed technologies grow and improve, overall Apache Pig feels left in the dust and is more legacy code to support than something to actively develop with.
Read full review
ScreenShots