dbt vs. IBM StreamSets

dbt (Data Build Tool) is best suited for doing the data transformation. dbt is just a transformation tool and it is not suitable for building a data pipeline which requires extraction of data and loading. dbt is well suited for SQL based transformation logic and it is less appropriate when transformation logic requires python.

Incentivized

When you are dealing with a data warehouse and want to find an easy way to integrate applications and expose data in real-time, then IBM StreamSets is the best tool to go for. I'm using it for the same purpose in my applications. This tool will be well-suited for someone with a proper technical background. Though IBM StreamSets UI is mostly drag and drop, advanced configurations require technical expertise or support to do the initial setup.

Incentivized

• user experience makes it easy to work with SQL and version control
• customer success team and the dbt (data build tool) community help establish best practices
• thorough and clear documentation

• It makes building data pipelines quite super intuitive even for non coders.
• Ir also handles real time data ingestion effortlessly so I always have up to date information for my reports.
• It's great at monitoring data quality as well.

Incentivized

• Slow load times of the dbt cloud environment (they're working on it via a new UI though)
• More out-of-the-box solutions for managing procedures, functions, etc would be nice to have, but honestly, it's pretty easy to figure out how to adapt dbt macros

Incentivized

• Where the person's skillsets in data analysis is not of an expert.
• Data monitoring and analysis.
• Customer data for better customer acquisition

Incentivized

dbt is very easy to use. Basically if you can write SQL, you will be able to use dbt to get what you need done. Of course more advanced users with more technical skills can do more things.

Incentivized

because i think that overall the solution is having a positive impact on the business, it allows multiple benefits in simplification of the tasks and is capable of doing multiple process that are usually done by a combination of man and systems, reducing the time and effort required to have the data.

Incentivized

Matillion is graphical versus dbt, which is SQL code-based (that, of course, is a matter of personal preference and not an objective advantage). The integrated testing, documentation generation, lineage, etc., were additional criteria that led us to choose dbt.

Incentivized

At Unify Logistics, we chose IBM StreamSets over Fivetran for its flexibility in handling complex, real time data pipelines across hybrid environments. While Fivetran offers simplicity and fast setup, StreamSets provides deeper customization, better data drift handling, and stronger support for dynamic logistics workflows.

Incentivized

• In 3 months we re-wrote the data warehouse (15-20 sources) in dbt with 3 developers.
• We are using it continually for the past year with no issues.
• Sorry, I don't have ROI numbers but the impact was huge.

Incentivized

• Huge reduction in maintenance as the StreamSets pipelines are resilient.
• Eliminated the need for other data ingestion tools, saving us hundreds of thousands annually.

Incentivized