dbt vs. Fivetran

If you can load your data first into your warehouse, dbt is excellent. It does the T(ransformation) part of ELT brilliantly but does not do the E(xtract) or L(oad) part. If you know SQL or your development team knows SQL, it's a framework and extension around that. So, it's easy to learn and easy to hire people with that technical skill (as opposed to specific Informatica, SnapLogic, etc. experience). dbt uses plain text files and integrates with GitHub. You can easily see the changes made between versions. In GUI-based UIs it was always hard to tell what someone had changed. Each "model" is essentially a "SELECT" statement. You never need to do a "CREATE TABLE" or "CREATE VIEW" - it's all done for you, leaving you to work on the business logic. Instead of saying "FROM specific_db.schema.table" you indicate "FROM ref('my_other_model')". It creates an internal dependency diagram you can view in a DAG. When you deploy, the dependencies work like magic in your various environments. They also have great documentation, an active slack community, training, and support. I like the enhancements they have been making and I believe they are headed in a good direction.

Incentivized

Mostly Fivetran can be useful for working with risk reduction operations during agile data analysis processes. I.e., in the short term the heaviest data movement operations would be safe. If you need to create an automated infrastructure for the data, the ability to create data list transformations in SQL is useful for keeping the work integrable or with schema changes. For situations that require a lot of speed: setting up the Fivetran platform is very easy, as you only need to authenticate the sources of the data to start working, and this is excellent for covering fast storage operations.

Incentivized

• user experience makes it easy to work with SQL and version control
• customer success team and the dbt (data build tool) community help establish best practices
• thorough and clear documentation

• Easily connects to source data using delivered connectors
• Transforms data into standard models and schemas
• Has very good documentation to help quickly setup connectors

Incentivized

• Slow load times of the dbt cloud environment (they're working on it via a new UI though)
• More out-of-the-box solutions for managing procedures, functions, etc would be nice to have, but honestly, it's pretty easy to figure out how to adapt dbt macros

Incentivized

• More detailed logging
• More flexible choices for time range over which records are synced
• More options for masking and excluding sensitive data

Incentivized

Just need to input connection info.

Incentivized

It runs pretty well and gets our data from point A to point cluster quickly enough. Honestly, it's not something I think about unless it breaks and that's pretty rare.

Incentivized

Most ETL pipeline products have a T layer, but dbt just does it better. The transformation is on steroids compared to the others. Also, just allows much more Adhoc solutions for very specific projects. Those ETL tools are probably better on the T part if you don't need too many transforms - also dbt is pretty much free dependent on how you work it, also extremely scalable.

Incentivized

Fivetran came well with the connectors' availability and updates with the source changes. We had an idea on data requirements in our case which helped us to work out on cost implication and take a decision for Fivetran as a data provider for our organization. These were 2 places where Fivetran out-performed, other vendors.

Incentivized

• Simplified our BI layer for faster load times
• Increased the quality of data reaching our end users
• Makes complex transformations manageable

Incentivized

• Development cost have reduced for each connector
• The pay-per-use model is still not out their which requires lot of overhead cost

Incentivized