Beautiful UI and UX to compete with any ETL, but at a cost!
October 06, 2020
Beautiful UI and UX to compete with any ETL, but at a cost!
Score 3 out of 10
Vetted Review
Verified User
Overall Satisfaction with Matillion
We still have an old-fashioned process in our monolith environment to extract-transform-load from a SQL server data store. Currently the monolith is the main product of the CM brand and is still being heavily maintained. The way we are moving forward is to "unlock" the data from its current storage in order to have better analytics using event streaming. For the time being, the only way we are able to extract large amounts of data from our relational SQL data store is using the ETL process through Matillion into our data warehouse solution in Redshift.
- The easy-to-use GUI makes it easier for our team to pass on the knowledge and upskill engineers on our ETL processes.
- The feature set is rich with many options to allow us to try different ways to transform our data without having to code.
- Many different integration points allow us to plug straight into services like SQS to help us communicate with our own internal services.
- Matillion does not scale well. It has a hard limit on the hardware / EC2 instances it can use. Most of the time that does not provide enough parallel processing for the millions of records we want to transform.
- It is expensive considering the infrastructure cost is added to Redshift costs, so the overall value for analytics is something we are constantly challenging.
- Constant Java heap space errors, again this is because of hard limits on EC2 instance hosting.
- Considering our specific use case is to provide analytics "for free" to our existing customers of all tiers, the added investment is hard to measure since this is an internal cost in order to keep up with our competitors. We are still trying to quantify customer retention directly related to the new analytics we provide because of what Matillion has helped us achieve.
From the moment we bought it on AWS marketplace to connecting to our monolith database it took a matter of hours. Deciding the Redshift table structure we needed for storage actually took longer than the actual setup time! That is a definite plus for anyone who wants to get up an running as soon as possible.
It is much easier to use in terms of GUI capabilities. The only reason we would use an ETL tool other than our own manually written SQL scripts, is to be able to allow other engineers to use it without having one domain expert stuck on the inner working of complex scripts. So the GUI is essential for that, otherwise we wouldn't bother considering an off the shelf product.