Dataiku vs. dbt

Dataiku is an awesome tool for data scientists. It really makes our lives easier. It is also really good for non technical users to see and follow along with the process. I do think that people can fall into the trap of using it without any knowledge at all because so much is automated, but I dont think that is the fault of Dataiku.

Incentivized

The prerequisite is that you have a supported database/data warehouse and have already found a way to ingest your raw data. Then dbt is very well suited to manage your transformation logic if the people using it are familiar with SQL. If you want to benefit from bringing engineering practices to data, dbt is a great fit. It can bring CI/CD practices, version control, automated testing, documentation generation, etc. It is not so well suited if the people managing the transformation logic do not like to code (in SQL) but prefer graphical user interfaces.

Incentivized

• Allows users to collaborate and monitor individual tasks
• Caters to both types of analysts, coders and non-coders, alike
• Integrate graphs and plots with visualization tools such as Tableau

Incentivized

• dbt supports version control through GIT, this allows teams to collaborate and track the data transformation logic.
• dbt allows us to build data models which helps to break complex transformation logic into simple and smaller logic.
• dbt is completely based on SQL which allows data analyst and data engineers to build the transformation logic.
• dbt can be easily integrated with snowflake.

Incentivized

• The integrated windows of frontend and backend in web applications make it cumbersome for the developer.
• When dealing with multiple data flows, it becomes really confusing, though they have introduced a feature (Zones) to cater to this issue.
• Bundling, exporting, and importing projects sometimes create issues related to code environment. If the code environment is not available, at least the schema of the flow we should be able to import should be.

Incentivized

• Field-level lineage (currently at table level)
• Documentation inheritance - if a field is documented the downstream field of the same name could inherit the doc info
• Adding python model support (in beta now)

Incentivized

The user experience is very good. Everything feels intuitive and "flows" (sorry excuse the pun) so nicely, and the customization level is also appropriate to the tool. Even as a newer data scientist, it felt easy to use and the explanations/tutorials were very good. The documentation is also at a good level

Incentivized

dbt is very easy to use. Basically if you can write SQL, you will be able to use dbt to get what you need done. Of course more advanced users with more technical skills can do more things.

Incentivized

The open source user community is friendly, helpful, and responsive, at times even outdoing commercial software vendors. Documentation is also top notch, and usually resolves issues without the need for human interactions. Great product design, with a focus on user experience, also makes platform use intuitive, thus reducing the need for explicit support.

Incentivized

Anaconda is mainly used by professional data scientists who have profound knowledge of Python coding, mainly used for building some new algorithm block or some optimization, then the module will be integrated into the Dataiku pipeline/workflow. While Dataiku can be used by even other kinds of users.

Incentivized

I actually don't know what the alternative to dbt is. I'm sure one must exist other than more 'roll your own' options like Apache Airflow, say, bu tin terms of super easy managed/cloud data transforms, dbt really does seem to be THE tool to use. It's $50/month per dev, BUT there's a FREE version for 1 dev seat with no read-only access for anyone else, so you can always start with that and then buy yourself a seat later.

Incentivized

• Customer satisfaction
• Timely project delivery

Incentivized

• Simplified our BI layer for faster load times
• Increased the quality of data reaching our end users
• Makes complex transformations manageable

Incentivized