Item: Matillion
Rating: 6
Author: Verified User

Use Cases and Deployment Scope

We're using Matillion to mainly load data from three places into our RedShift clusters. The first one is from a s3, the second is syncing data from our production DB.

Pros and Cons

Managing Schedule
Intuitive UI
Easily integrates with the rest of AWS
Create different versions is easy

The deployment process is quite manual; need to export and import, create a new version. Would be nice if there's a repo for continuous integration
The Python script module is very limited. We try to use it to parse data on a file with 500 records, and it constantly crashes. It does not have the capabilities to run Python programs
In the Mongo module, the field must exist in the source system. Working with NoSQL DB, some fields might not exist just yet, and essentially we'll have to create everything downstream once the field appears in the source system.

Return on Investment

Accelerated development of ETL data pipeline.
Easy onboarding for new users.
Easily create scheduled jobs to maintain a RedShift environment.

Usability

It was easy to set up and create a simple job at first, but when it comes to more complex processes, it has its limitations. Ultimately, we went with writing our Python code on Lambda instead.

Time to Value

We were able to set up within an hour. I set up my first job (loading a CSV file from S3 to RedShift) by lunch.

Scalability

Its limitations on complex data manipulation using the Python module is a big blocker for us. We thought it would allow us to expand on Matillion's core feature, but it ended up hindering us.

Alternatives Considered

RedPoint Data Management & Quality, AWS Lambda, Microsoft SQL Server, PostgreSQL, MySQL, SAS DataFlux, SAS Data Management Platform and SAS Data Integration Studio

Overall from an ease of use and set up standpoint, it stacks up better than most of the other tools I've used. However, Matillion doesn't quite expand and scale up as well as the other solutions.

Other Software Used

RedPoint Data Management & Quality, SQL Server Business Intelligence Manager, SQL Server Integration Services, AWS Lambda

For organizations that have limited resource with AWS Infrastructure/Lambda knowledge, this is a great tool that works right out of the box. This tool is great for batch jobs and easily works well with AWS. However, for most processes, it could be replaced easily with a simple Lambda.

Easily set up your ETL/ELT pipeline with Matillion

Overall Satisfaction with Matillion