Item: Matillion
Rating: 9
Author: Callum O'Connor

Use Cases and Deployment Scope

We needed a way of empowering people with data in a no-code way. Data is a bottleneck and everyone needs it, but how do we give them this data and enable them to use it effectively? We launched ‘Data as a Product’ - A package of tools and resources that exposes raw data to people and allows them to manipulate it with ease. Matillion is the key tool in this package that allows people to manipulate data at the speed of thought with no code. Matillion’s no-code design, intuitive interface, and collaboration centric architecture allowed us to get people set up and transforming data in a matter of minutes with very little support from the Data team.

Pros and Cons

We leveraged Matillion’s no-code principals to make data manipulation easy for our internal customers. People who don't know how to use SQL no longer need to. Everything in Matillion is self-explained with no or little coding.
We connected Matillion to our data warehouse to allow people to read raw data, transform it, then write results back to their sandbox databases. The drag and drop component design allowed customers to create complex data models at the speed of thought without any risk to production data.
With sharing capabilities between projects enabled, everyone was able to help each other when questions arose which instilled a strong sense of collaboration and community.

The new DPC version of Matillion uses Git principals like Commit, Push, Merge etc. This is perfectly fine for engineers, but for our use-case it means our customers will need to understand a basic level of Git. It would be great if they had an auto-commit-push setting which does it all for them.

Return on Investment

Our embedded data analysts (data analysts that sit in a team outside of the Data team) all now use Matillion to create proof of concepts (POCs). This allows them to debug logic at a component level and quickly explore ideas without investing lots of time and effort.
Since the soft-announcement of ‘Data as a product’ (a beta launch) and demoing Matillion to some of our internal customers we’ve had a huge number of requests from people to get their hands on this new method of self serving data. We’ve yet to release the full product and make a company wide announcement, but early estimates show we can expect around 10-15% of the company to be onboarded and using Matillion as part of Data as a Product. Given the Data team only accounts for around 2% for the company's employees, that’s a huge increase in the number of people using and manipulating raw data!

Usability

The drag and drop component design allowed customers to create complex data models at the speed of thought. The prompts/errors are useful in terms of getting components to work, and you can see exactly where along the pipeline/transformation process something breaks. Branching off the pipeline and dip-checking/sampling the data as it goes is also super helpful for debugging. From an engineering perspective, we’re able to take complex SQL models, re-create and visualise them in Matillion, refactor the design, then use the SQL output from Matillion to create new data model.

Time to Value

Our initial build was METL (older version of Matillion). This meant we had to create an EC2 instance on AWS. This initial setup took maybe a few days. but once we spun up the mahine and got it all working, we connected Matillion to our data warehouse in a matter of minutes. The new DPC Matillion is all SaaS so was a lot quicker to set up, maybe a few hours, a day max.

Scalability

There are more and more connections to external sources being introduced each quarter on DPC which is great. And because a lot of the compute work is done on the data warehouse itself it means we can control costs easier. The DPC pricing model is also a lot more scalable compared to METL; where we're charged for the run-time of a pipeline, rather than number of users.

Alternatives Considered

dbt is great for engineers and those comfortable with coding. Matillion is the low-code alternative with a huge emphasis on collaboration with ease. You don't need to checkout a branch, clone, pull, merge etc. in order to help your colleague with a data pipeline. The simple drag'n'drop design allows for easy collaboration and understanding.

Key Insights

Do you think Matillion delivers good value for the price?

Yes

Are you happy with Matillion's feature set?

Yes

Did Matillion live up to sales and marketing promises?

Yes

Did implementation of Matillion go as expected?

Yes

Would you buy Matillion again?

Yes

Other Software Used

dbt, Apache Airflow, Snowflake

Likelihood to Recommend

Matillion lowers the bar for entry into the world of data transformation. it's low-code design, simple drag'n'drop visualisation and SQL code generator makes data transformation easy and educational for those starting off in the world of data. For refactoring, we’re able to take existing complex SQL models, re-create and visualise them in Matillion, change the design, then use the SQL output from Matillion to create new data models in our repos.
Matillion has completely changed the way we serve data to our internal customers at the company; we've exported capability and empowerment rather than allowing ourselves to get swamped with tickets and requests for every possible data question. It's a great tool for getting people to self-serve their own answers to questions about their data.

Low-code solution to transforming data at the speed of thought

Overall Satisfaction with Matillion