ClickHouse vs. Informatica Cloud Data Quality vs. OpenText Vertica

The most important thing when using ClickHouse is to be clear that the scenarios in which you want to use it really are the right ones. Many users think that when a database is very fast for a specific use case, it can be extrapolated to other contexts (most of the time different) in which a previous analysis has not been carried out.

ClickHouse is an analytical database, as such, it should be used for such purposes, where the information is stored correctly, the data volumes are really large and the queries to be performed are not the typical traditional queries on several columns with multiple aggregations. ClickHouse is not the solution for this.

On the other hand, if your case is not one of the above, it is quite possible that ClickHouse can help you. Where ClickHouse shines is when you are looking for aggregation over a particular column in large volumes of data.

Incentivized

For effective data collaboration, systematic verification of customer information, and address, among others, Informatica Data Quality is a fruitful application to consider. Besides, Informatica Data Quality controls quality through a cleansing process, giving the company a professional outline of candid data profiling and reputable analytics. Finally, Informatica Data Quality allows the simplistic navigation of content, with a dashboard that supports predictability.

Incentivized

Vertica as a data warehouse to deliver analytics in-house and even to your client base on scale is not rivaled anywhere in the market. Frankly, in my experience it is not even close to equaled. Because it is such a powerful data warehouse, some people attempt to use it as a transactional database. It certainly is not one of those. Individual row inserts are slow and do not perform well. Deletes are a whole other story. RDBMS it is definitely not. OLAP it rocks.

Incentivized

• Their MergeTree table engine provide impressive performance for data insert in bulk
• Not only data insert but also the way MergeTree engine uses Primary Keys to sort the data and perform data skipping based on the granules its also their secret for ridiculous fast queries
• Data compression its also great
• They provide especial table engines that allow you to read data directly from other sources like S3
• Since its written with C++ you have very granular data types and especial ones like enum, LowCardinality and etc, they save you a lot of storage since are stored as integer values
• ClickHouse functions besides the ones that respect ANSI Standards are also awesome and useful

Incentivized

• The matching algorithms in IDQ are very powerful if you understand the different types that they offer (e.g., Hamming Distance, Jaro, Bigram, etc..). We had to play around with it to see which best suit our own needs of identifying and eliminating duplicate customers. Setting up the whole process (e.g., creating the KeyGenerator Transformation, setting up the matching threshold, etc..) can be somewhat time consuming and a challenge if you don't first standardize your data.
• The integration with PowerCenter is great if you have both. You can either import your mappings directly to PowerCenter or to an XML file. The only downside is that some of the transformations are unique to IDQ, so you are not really able to edit them once in PowerCenter.
• The standardizer transformation was key in helping us standardize our customer data (e.g., names, addresses, etc..). It was helpful due to having create a reference table containing the standardized value and the associated unstandardized values. What was great was that if you used Informatica Analyst, a business analyst could login and correct any of the values.

• Extremely fast query performance - Vertica is one of the fastest query engines out there.
• Scales to TBs - Scales reasonably well up to 10-20 nodes and 10 - 100s of TB of data.
• Easy to Use - Fairly easy to user, we made quite some headway with just 1 person running it for a while.

Incentivized

• Avro data manipulation
• Kafka consistency
• DDL operations errors (by replica configuration)

Incentivized

• Several partnerships diminishing the value of technologies
• Unable to get list of objects from Repository (like sources & targets) that don't have any dependency
• Scheduling: The built-in scheduling tool has many constraints such as handling Unix/VB scripts etc. Most enterprises use third party tools for this.

• Could use some work on better integrating with cloud providers and open source technologies. For AWS you will find an AMI in the marketplace and recently a connector for loading data from S3 directly was created. With last release, integration with Kafka was added that can help.
• Managing large workloads (concurrent queries) is a bit challenging.
• Having a way to provide an estimate on the duration for currently executing queries / etc. can be helpful. Vertica provides some counters for the query execution engine that are helpful but some may find confusing.
• Unloading data over JDBC is very slow. We've had to come up with alternatives based on vsql, etc. Not a very clean, official on how to unload data.

Incentivized

As pointed out earlier, due all the robust features IDQ has, our use f the product is successful and stable. IDQ is being used in multiple sources (from CRM application and in batch mode). As this is an iterative process, we are looking to improve our system efficiency using IDQ.

Easy to use not only for developers but also business users

Incentivized

The application works well except an occasional error out while using the system. It usually gets fixed when restarting the Infa server

Incentivized

Performance works just fine. It was able to load 200+ business terms, 150+ DQ automation, etc. very well.

Incentivized

I haven't had any recent opportunity to reach out to Vertica support. From what I remember, I believe whenever I reached out to them the experience was smooth.

Incentivized

ClickHouse outperforms, especially in costs, since its compression/indexing engines are so smart, and even with very low computing power, you can already perform huge analyses of the data.

Incentivized

IDQ is used by a department at my organisation to ensure and enhance the data quality.
The usage was started with address standardization and now it had been brought to altogether a next level of quality check where it fixes duplicates, junk characters, standardize the names, streets, product descriptions.
In the past we had issues mainly with duplicate customers and products and this were affecting the sales projection and estimates.

Vertica performs well when the query has good stats and is tuned well. Options for GUI clients are ugly and outdated. IO optimized: it's a columnar store with no indexing structures to maintain like traditional databases. The indexing is achieved by storing the data sorted on disk, which itself is run transparently as a background process.

Incentivized

Scalability works as expected and it is truly an enterprise system.

Incentivized

• Queries that used to take more than 2 minutes now take less than 1 second
• Possibility to analyze use cases in real time (before was impossible)
• The applications are more complete and the users decisions are better

Incentivized

• Integration with tools like PowerCenter helped faster delivery of product, and at the same time conversion
• Reduce overall project cost due to bad data , bad quality, exceptions identified nearing go-live and post production
• Employee efficiency is increased exponentially due to more automated, customized tool

• Positive impact on ROI by being able to get customer insights in real-time.
• Positive ROI through reduced time to set-up and maintain Vertica instances.

Incentivized