Item: Matillion
Rating: 10
Author: Travis Schaugaard

Use Cases and Deployment Scope

Matillion is being used by Xyngular to ETL data from multiple environments into an AWS Redshift database. We also use it to pull data files from FTP and load them into the database. It also allows us to be very flexible on the scheduling of the jobs.

Pros and Cons

Loading an FTP file into the database.
Transforming data before loading it into the database.
Flexible scheduling of jobs.
Straight data copy from one database to another.

When I make changes to a job and add fields to what is being pulled, I have to drop the entire list and repopulate it.
Honestly, the first thing is the one part I have had issues with.

Return on Investment

Matillion has provided a way for us to have near real-time viewing of metrics.
Matillion has provided a way for us to do complex transformations on our data in a pipeline.

Usability

I like the way Matillion is set up and in most cases it's very easy to figure out what I need to do and where to find it. Sometimes it takes a little longer to figure out the solution.

Time to Value

We were able to get up and running with Matillion very quickly. It took no more than a day or two.

Scalability

We have had no issues with it scaling to do whatever has been needed. However, most of the biggest jobs that we are running only have a few tens of millions of records. As that gets bigger, we'll see how well Matillion handles it.

Alternatives Considered

Matillion is definitely a more powerful tool than AWS pipelines.

Other Software Used

GitHub

Likelihood to Recommend

We also use AWS pipelines to pull data from one environment to another. It works great for a straight data copy. However, it doesn't allow much transforming of the data. Matillion provides a much better way to make transformations on the data prior to loading it into the database.

Matillion has met the needs of our data warehousing and ETL better than we had exptected

Overall Satisfaction with Matillion