Item: Matillion
Rating: 7
Author: Jay Archer

Use Cases and Deployment Scope

We are using Matillion to centralize our data to a new data warehouse. Our primary use case is pulling data from a relational database (MySQL). I've been able to implement three different flavors of data sourcing, the two primary types being a full table load and incremental data loading of changed data into Slowly Changing Dimension and Fact tables in the DW. The performance of Matillion combined with Snowflake is astoundingly fast. We are also able to hit APIs to Zendesk and Hubspot easily to round out our integration with other SaaS vendors in our stack.

Pros and Cons

Push down query performance with Snowflake.
The ability to hit any API using Python.
A robust offering of pre-built connectors to databases, APIs, and other SaaS vendors.

No user community site for experienced developers to share their patterns and help grow the dev community
Documentation can get stale or be changed without notice.
Several aspects of the product are not user-friendly, and if implemented by an experienced product/UX person it would make the product easily 2x to 3x better.
No ability to vote on what features are in the pipeline.

Key Insights

Do you think Matillion delivers good value for the price?

Are you happy with Matillion's feature set?

Did Matillion live up to sales and marketing promises?

Yes

Did implementation of Matillion go as expected?

Would you buy Matillion again?

Return on Investment

Able to react to changing data quality issues caused upstream
Ability to handle massive scaling issues with ease
Ability to connect to more SaaS vendors as our business needs change

Usability

I would like to give it a 9.5 or so. Only a few aspects of the usability need some work. These are trivial things to fix, but really annoying to an experienced user.

Time to Value

My learning curve was long. Matillion is simply not like any other ETL / ELT tool out there, because Snowflake isn't like other cloud databases out there. I had to first learn a different pattern of data movement and error checking. Next, I had to learn which components to use and when. There aren't any good in-depth how-to guides out there. There aren't really detailed instructions on how to implement a specific pattern and what challenges you are going to face and how to handle them. There isn't a site for users and developers to hang out and ask questions, there is only the support ticket queue and some very knowledgeable and helpful Matillion engineers to answer your questions.

Scalability

I've only had a handful of performance issues that are clearly in the realm of Matillion's logic. The vast majority of features and functions perform as if they are operating directly in the instance with the database with no overhead.

Alternatives Considered

Fivetran, Stitch, Talend Data Integration, Trifacta and Alooma

If you take the time to build your own with Matillion, you will end up with a vastly better solution than Stitch and Fivetran. We ran Stitch and Fivetran side by side connected from our source DB to both RedShift and Snowflake and documented the performance results as well as the multitude of data issues we had during the evaluation period. Those tools are great for plug-and-play if your data is simple and small. Our data and our source database are not simple, and huge and Stitch and Fivetran are just not great at handling variety. Now that Matillion is in place, the data loads that took Stitch seven days can be completed in under an hour. We have control over metadata and our DW data model, and can customize which columns from upstream we care about. The sky is the limit now that we have built out our solution. We are customizing, growing and expanding our capabilities every day and there is absolutely nothing standing in our way with Matillion. If we had chosen any other vendor tool (we evaluated many more that aren't listed by TrustRadius) we would be stuck. We would be hacking away at data in the DW instead of handling data in a well thought out manner as part of the pipeline.

Other Software Used

Looker, Snowflake, Jira Software, Atlassian Confluence, Amazon Elastic Compute Cloud (EC2)

Likelihood to Recommend

Great if you need a visual, customized, powerful data engineering and data integration platform that can do pretty much anything. I have yet to hit a situation that I can't solve one way or another. Not great if you only have time for a plug-and-play solution. This is not Stitch or FiveTran, but if you invest the time to learn and use Matillion you can end up with a vastly better solution to your data needs.

Time Saved

yes

Cost Savings

we did not move from a legacy system, this was a brand new ELT system for us

Business Outcomes

We were able to connect about 70% of our data sources

Onboarding

a full year to get fully implemented. The onboarding was immensely painful given the documentation was lite and incorrect in many places. Many patterns of use I had to figure out for myself. At the time of implementation there was no developer community and the only company contact was through the support desk which was always rushed and abbreviated

custom in house ELT logic
ability to customize endlessly
lots of pre-built connectors to data sources

Innovative Uses

incremental and full-refresh loads
you can make an ingestion process so generic it can be used on any table in the sql data source (mysql)

Future Planned Uses

unique extract techniques not supported by the big ETL vendors

Likelihood to Renew

ability to use Matillion in a multi-dev multi-project environment is too painful and too error prone

Build your own customized version of a Stitch and FiveTran-like data replication process, but better, faster and able to evolve with your needs

Overall Satisfaction with Matillion