Item: Matillion
Rating: 9
Author: Antonello Supino

Use Cases and Deployment Scope

We use Matillion for Redshift as to orchestrate data pipelines as our main data integration tool. We are pleased by the usability and reliability along with multi-user interface, git integration, and lots of plugins. Matillion has helped us to have strong data pipelines that we can monitor to run hundreds of jobs a day. Support is also proactive. It's overall a great ELT tool.

Pros and Cons

Visual flow of ETL
Integrations with GIT
Multi-user editing

Shared components portability
Having other databases other than Redshift as targets (such as MySQL)

Return on Investment

Positive ROI

Usability

The interface that can be edited by multiple users at the same time makes it very usable. It's not a script based tool but it can be so it is great for all levels of professionals. The interface is responsive and there is no need to install desktop clients which is great. Sometimes after upgrades, we have to manually clear browser cache which is a bit annoying.

Time to Value

We got Matillion from the AWS marketplace, and we got up and running within less than a week. Starting the server was simple, then we had to do a bit of configuration and integrations with our servers adding the credentials, git repos etc. The experience was really good as Matillion blends perfectly within the AWS ecosystem.

Scalability

With Matillion, we push most of the computational load down to redshift so it's not an issue to scale up. Perhaps the instance sizes available may not fit if the load becomes too high, however, we did not face any of this issue. We are using multiple projects with multiple users and all is going well.

Alternatives Considered

Informatica Data Engineering Streaming, Jenkins and Pentaho

Pentaho was used initially but the support was terrible. It was taking years to fix bugs and the scheduler was buggy, and it took an eternity to get the issue with the misfired job on the quartz Java scheduler to become made reliable. Informatica is a great tool that was considered, but Matillion was chosen as it is younger, cheaper and specific to Redshift. Jenkins is another tool we used. It is very reliable and free, but it was so difficult to create a simple and clear workflow to understand.

Other Software Used

Amazon Redshift, Periscope Data, Amazon Athena, AWS Serverless Application Repository

Likelihood to Recommend

Matillion is great when you want to integrate the data into a single DWH such as Redshift or Snowflake.
Also the task history is very clear along with all the detailed reporting on each single operator.

When you want to export data back to MySQL or to other data sources it is quite difficult, that's an area of improvement.

Robust ELT with Matillion for Redshift

Overall Satisfaction with Matillion