Item: Matillion
Rating: 8
Author: Caleb Dinsmore

Use Cases and Deployment Scope

We are using it as the backbone for various projects we do for clients who possess large amounts of semi-structured data that they currently cannot use effectively. We ingest and transform their data using Matillion, storing it in Snowflake and building reports on it using Tableau. These projects are what we primarily use Matillion for. We're looking into using it across our organization to help manage data like our financials and other organizational metadata that we use to make business decisions.

Pros and Cons

Very intuitive/easy to learn and teach
Great documentation
Easy to spin up and get going

Version control is minimal. Tough to form a process for collaborative development around it.
More thorough documentation around best practices specifically would be good.
Sometimes the tool freezes up when a large number of processes are running.

Return on Investment

Quite positive. It's much easier to train new hires on the system.
Less maintenance required than running a DIY setup like Kylo (open source).

Usability

It's good but could be better. The interface takes a bit to get used to, but thankfully Matillion provides good materials (documentation and video tutorials) that streamline learning it. Sometimes different jobs hang eternally, which impacts development (usually fixed with a reboot). Otherwise, it's a great tool that is fairly easy-to-use and learn.

Time to Value

During our evaluation of different products on the market, I was able to spin up an instance, connect it to Snowflake, and create jobs that accomplished a specific business need in one day. Once we formally adopted it, it's taken us several weeks/a couple of months to convert our existing ETL processes into a Matillion workflow, simply because of how many existing processes we had. It is very easy to get running with it and start being productive.

Scalability

Its scalability is one of its greatest strengths over a DIY solution like Kylo. Since it's an orchestration tool and doesn't do the heavy lifting itself, you only need to scale it when you need to increase the number of concurrent users (users editing jobs simultaneously). The processing power is handled by something like Snowflake.

Alternatives Considered

Google BigQuery, Fivetran, Alooma, Amazon Redshift and Talend Open Studio

Our organization hires present college students in an apprenticeship role, so it was important to us that whatever tool we use to be easy to train on. We also preferred that it integrate well with Snowflake since we had decided that we wanted to use that as our data storage solution. We needed this solution to not only operate as a means to get data into Snowflake but also as an ELT/ETL tool that could perform transformations on that data and load it into schemas within our warehouse. Because of Matillion's customer support and ease of use, we were won over.

Other Software Used

Snowflake, PostgreSQL, Elasticsearch

Likelihood to Recommend

Well Suited For:

If you are wanting to use a data warehouse solution like Snowflake but need more flexibility/power for getting data into Snowflake (and manipulating it when it's there).
If your data scientists/analysts aren't especially tech-savvy. Its interface is easier for less technically-inclined users to understand.

Less Appropriate When:

You aren't using Snowflake, BigQuery, Redshift, etc.
The quantity/complexity of the data you're working with is minimal/not very difficult to manage.
You want to transform your data before loading (ETL vs ELT).

Matillion Review

Overall Satisfaction with Matillion