Item: Matillion
Rating: 7
Author: Jason Schwartz

Use Cases and Deployment Scope

Loading S3 data lake files to Redshift
Calling APIs and massaging data in Python and loading to redshift
Automatic migration of files in S3
Migrating data from Oracle to Redshift

Pros and Cons

Excellent support team
Consistent releases with great documentation
UI is easy to learn and navigate
Job scheduling is straightforward
I like the environment variables and how you can export a variable to a shared job

I think a non-CLI approach to installing and uninstalling python libraries would be nice. I mean, it isn't difficult to install a python library via Linux or CLI, but I imagine most companies don't feel comfortable allowing Matillion users to go on a virtual server and installing it themselves. Requirement.txt file for installing libraries would be simple, and maybe that could also be used to uninstall libraries as well......or maybe the library gets automatically downloaded if it is imported into a python script but the library doesn't exist.
Python Component is lacking very much in terms of UI. It would be unrealistic for me to suggest Matillion build its own EDI, but it would be nice if a python component could connect to a local IDE. Right now, if you want to write any decent length python code, you are going to be stuck copying and pasting your code from your local IDE.
It is also very difficult to debug in Matillion because you can't set breakpoints. Local IDE integration can resolve that.
I would like to have more templates to copy from for certain simplistic scenarios. For instance, a template for a job that fails which sends an AWS SNS with the Job name, component it failed on, and the error message. It wasn't as simple as I thought it would be to figure out and having to use a shared job for such circumstances can be painful because you have to export a bunch of variables.
There should be a drop down list for Global Matillion variables as it is difficult to remember at times.

Key Insights

Do you think Matillion delivers good value for the price?

Not sure

Are you happy with Matillion's feature set?

Yes

Did Matillion live up to sales and marketing promises?

I wasn't involved with the selection/purchase process

Did implementation of Matillion go as expected?

I wasn't involved with the implementation phase

Would you buy Matillion again?

Return on Investment

Definitely has resulted in fewer hours spent on data preparation
It is a very reliable platform that I can't ever recall failing on us
The more complicated jobs can look very messy and difficult on the eyes

Matillion provides an abundance of low code features, especially for SQL that can simplify what a component does (vs just using a plain sql script component). The API components aren't as fluid, and in my opinion, is better suited with python. I can't blame Matillion on this one as so many rules and parameters need to be accounted for.

Time to Value

I felt comfortable with Matillion about a few weeks after using it. After a year I feel very well versed in Matillion and its capabilities.

Scalability

It covers AWs, Snowflake, Google Cloud, and a bunch of other services. List is endless really.

Alternatives Considered

AWS Lambda and AWS Glue

Matillion has better documentation, is easier to pick up on, and is better supported. Glue doe shave pyspark though and might handle Big Data more effectively.

Other Software Used

AWS Glue, AWS Lambda, Domo

Likelihood to Recommend

Matillion is very strong when it comes to its SQL components and plays very nicely with loading files from S3 to Redshift. It is less desirable for those who want to use a lot of python though, at that point their experience would be limiting.

Low Code and Python component review

Overall Satisfaction with Matillion