Apache Pig is a programming tool for creating MapReduce programs used in Hadoop.
$0
OpenText Vertica
Score 10.0 out of 10
N/A
The Vertica Analytics Platform supplies enterprise data warehouses with big data analytics capabilities and modernization. Vertica is owned and supported by OpenText.
Apache Pig is best suited for ETL-based data processes. It is good in performance in handling and analyzing a large amount of data. it gives faster results than any other similar tool. It is easy to implement and any user with some initial training or some prior SQL knowledge can work on it. Apache Pig is proud to have a large community base globally.
Vertica as a data warehouse to deliver analytics in-house and even to your client base on scale is not rivaled anywhere in the market. Frankly, in my experience it is not even close to equaled. Because it is such a powerful data warehouse, some people attempt to use it as a transactional database. It certainly is not one of those. Individual row inserts are slow and do not perform well. Deletes are a whole other story. RDBMS it is definitely not. OLAP it rocks.
Could use some work on better integrating with cloud providers and open source technologies. For AWS you will find an AMI in the marketplace and recently a connector for loading data from S3 directly was created. With last release, integration with Kafka was added that can help.
Managing large workloads (concurrent queries) is a bit challenging.
Having a way to provide an estimate on the duration for currently executing queries / etc. can be helpful. Vertica provides some counters for the query execution engine that are helpful but some may find confusing.
Unloading data over JDBC is very slow. We've had to come up with alternatives based on vsql, etc. Not a very clean, official on how to unload data.
I haven't had any recent opportunity to reach out to Vertica support. From what I remember, I believe whenever I reached out to them the experience was smooth.
Apache Pig might help to start things faster at first and it was one of the best tool years back but it lacks important features that are needed in the data engineering world right now. Pig also has a steeper learning curve since it uses a proprietary language compared to Spark which can be coded with Python, Java.
Vertica performs well when the query has good stats and is tuned well. Options for GUI clients are ugly and outdated. IO optimized: it's a columnar store with no indexing structures to maintain like traditional databases. The indexing is achieved by storing the data sorted on disk, which itself is run transparently as a background process.
Higher learning curve than other similar technologies so on-boarding new engineers or change ownership of Apache Pig code tends to be a bit of a headache
Once the language is learned and understood it can be relatively straightforward to write simple Pig scripts so development can go relatively quickly with a skilled team
As distributed technologies grow and improve, overall Apache Pig feels left in the dust and is more legacy code to support than something to actively develop with.
