Matillion is a productivity platform for data teams. Matillion's Data Productivity Cloud helps data teams – coders and non-coders alike – to move, transform, and orchestrate data pipelines with the goal of empowering teams to deliver quality data at a speed and scale that matches the business’s data ambitions. The vendor states enterprises including Cisco,
N/A
Pricing
Apache Spark
Matillion
Editions & Modules
No answers on this topic
No answers on this topic
Offerings
Pricing Offerings
Apache Spark
Matillion
Free Trial
No
Yes
Free/Freemium Version
No
No
Premium Consulting/Integration Services
No
Yes
Entry-level Setup Fee
No setup fee
No setup fee
Additional Details
—
Billed directly via cloud marketplace on an hourly basis, with annual subscriptions available depending on the customer's cloud data warehouse provider.
It is much easier to use in terms of GUI capabilities. The only reason we would use an ETL tool other than our own manually written SQL scripts, is to be able to allow other engineers to use it without having one domain expert stuck on the inner working of complex scripts. So …
Well suited: To most of the local run of datasets and non-prod systems - scalability is not a problem at all. Including data from multiple types of data sources is an added advantage. MLlib is a decently nice built-in library that can be used for most of the ML tasks. Less appropriate: We had to work on a RecSys where the music dataset that we used was around 300+Gb in size. We faced memory-based issues. Few times we also got memory errors. Also the MLlib library does not have support for advanced analytics and deep-learning frameworks support. Understanding the internals of the working of Apache Spark for beginners is highly not possible.
It is very well suited for ETL on the cloud. Whenever there is something that can be accomplished with no code or little code, Matillion is a good tool. However, if your pipeline requires a lot of customizations, Matillion should be avoided.
We leveraged Matillion’s no-code principals to make data manipulation easy for our internal customers. People who don't know how to use SQL no longer need to. Everything in Matillion is self-explained with no or little coding.
We connected Matillion to our data warehouse to allow people to read raw data, transform it, then write results back to their sandbox databases. The drag and drop component design allowed customers to create complex data models at the speed of thought without any risk to production data.
With sharing capabilities between projects enabled, everyone was able to help each other when questions arose which instilled a strong sense of collaboration and community.
I think a non-CLI approach to installing and uninstalling python libraries would be nice. I mean, it isn't difficult to install a python library via Linux or CLI, but I imagine most companies don't feel comfortable allowing Matillion users to go on a virtual server and installing it themselves. Requirement.txt file for installing libraries would be simple, and maybe that could also be used to uninstall libraries as well......or maybe the library gets automatically downloaded if it is imported into a python script but the library doesn't exist.
Python Component is lacking very much in terms of UI. It would be unrealistic for me to suggest Matillion build its own EDI, but it would be nice if a python component could connect to a local IDE. Right now, if you want to write any decent length python code, you are going to be stuck copying and pasting your code from your local IDE.
It is also very difficult to debug in Matillion because you can't set breakpoints. Local IDE integration can resolve that.
I would like to have more templates to copy from for certain simplistic scenarios. For instance, a template for a job that fails which sends an AWS SNS with the Job name, component it failed on, and the error message. It wasn't as simple as I thought it would be to figure out and having to use a shared job for such circumstances can be painful because you have to export a bunch of variables.
There should be a drop down list for Global Matillion variables as it is difficult to remember at times.
With the current experience of Matillion, we are likely to renew with the current feature option but will also look for improvement in various areas including scalability and dependability. 1. Connectors: It offers various connectors option but isn't full proof which we will be looking forward as we grow. 2. Scalability: As usage increase, we want Matillion system to be more stable.
The only thing I dislike about spark's usability is the learning curve, there are many actions and transformations, however, its wide-range of uses for ETL processing, facility to integrate and it's multi-language support make this library a powerhouse for your data science solutions. It has especially aided us with its lightning-fast processing times.
It has been easy to train new employees who don't have previous experience with Matillion. It is quite self-explanatory. There are quite a few things that can be done with Python, however, we have not really looked into this feature much but likely will do in the future. Mostly, it is drag and drop of components and environments can be set up so easy to set up connections as well.
1. It integrates very well with scala or python. 2. It's very easy to understand SQL interoperability. 3. Apache is way faster than the other competitive technologies. 4. The support from the Apache community is very huge for Spark. 5. Execution times are faster as compared to others. 6. There are a large number of forums available for Apache Spark. 7. The code availability for Apache Spark is simpler and easy to gain access to. 8. Many organizations use Apache Spark, so many solutions are available for existing applications.
Overall, I've found Matillion to be responsive and considerate. I feel like they value us as a customer even when I know they have customers who spend more on the product than we do. That speaks to a motive higher than money. They want to make a good product and a good experience for their customers. If I have any complaint, it's that support sometimes feels community-oriented. It isn't always immediately clear to me that my support requests are going to a support engineer and not to the community at large. Usually, though, after a bit of conversation, it's clear that Matillion is watching and responding. And responses are generally quick in coming.
All the above systems work quite well on big data transformations whereas Spark really shines with its bigger API support and its ability to read from and write to multiple data sources. Using Spark one can easily switch between declarative versus imperative versus functional type programming easily based on the situation. Also it doesn't need special data ingestion or indexing pre-processing like Presto. Combining it with Jupyter Notebooks (https://github.com/jupyter-incubator/sparkmagic), one can develop the Spark code in an interactive manner in Scala or Python
Matillion ran circles around Stitch and Striim both in functionality, setup, and performance. There was no real comparison. Fivetran massively outperforms Matillion in pretty much every facet of the production from setup, maintenance, visibility, and usability. It already has the ability to connect any data source to a destination regardless of database type. Why we chose Matillion over Fivetran is that, for our current needs, Matillion provides us with the functionality that we need and a much more competitive price for a smaller company.
I have been able to connect Matillion to AWS Aurora Databases, MySQL databases, Rest APIs, Files in AWS S3, etc. Being able to load all of that disparate data into one datalake has made data mining and reporting a lot simpler. I wish everything could be implemented as easily as Matillion.
Faster turn around on feature development, we have seen a noticeable improvement in our agile development since using Spark.
Easy adoption, having multiple departments use the same underlying technology even if the use cases are very different allows for more commonality amongst applications which definitely makes the operations team happy.
Performance, we have been able to make some applications run over 20x faster since switching to Spark. This has saved us time, headaches, and operating costs.
