HCL DataConnect (formerly Actian DataConnect) is a hybrid integration solution that enables users to integrate anything, anywhere, anytime. It can be deployed in on-premise, cloud, and/or hybrid environments and offers quick onboarding. With its emphasis data quality, data reuse, adaptability and self-service integration, DataConnect is designed to support rapid onboarding and deliver the fast time to value.
N/A
dbt
Score 9.1 out of 10
N/A
dbt is an SQL development environment, developed by Fishtown Analytics, now known as dbt Labs. The vendor states that with dbt, analysts take ownership of the entire analytics engineering workflow, from writing data transformation code to deployment and documentation. dbt Core is distributed under the Apache 2.0 license, and paid Teams and Enterprise editions are available.
Sr Integration Engineer - Dual Certified in Dell Boomi Developer 1 & 2, and Actian DataConnect
Chose Actian DataConnect
As I stated earlier, DataConnect makes a head-to-head run against the others. It has advantages and disadvantages compared to the competitors. The choice really comes down to pricing and specific requirement needs for the project you are planning. Questions such as …
Data Integrator was initially a part of a package deal for us with our new Student Information System. It was chosen prior to my arrival at the institution. Since I discovered we have SQL Server licenses, we have decided to use SSIS instead since I have a background in the MSBI …
dbt is very flexible and can fit into most data pipelines. This is a pro for most organizations that aren't fully bought into one platform (Google Cloud, etc.)
Matillion is graphical versus dbt, which is SQL code-based (that, of course, is a matter of personal preference and not an objective advantage). The integrated testing, documentation generation, lineage, etc., were additional criteria that led us to choose dbt.
I actually don't know what the alternative to dbt is. I'm sure one must exist other than more 'roll your own' options like Apache Airflow, say, bu tin terms of super easy managed/cloud data transforms, dbt really does seem to be THE tool to use. It's $50/month per dev, BUT …
Snaplogic is great at the Extraction and Load processes of ETL. It can pull data from anywhere, even behind firewalls. So if you need to get data from various APIs, databases, files, S3, SFTP, etc it is easy to do so. However, it requires special knowledge in order to build …
Most ETL pipeline products have a T layer, but dbt just does it better. The transformation is on steroids compared to the others. Also, just allows much more Adhoc solutions for very specific projects. Those ETL tools are probably better on the T part if you don't need too many …
Airflow can accomplish the same work as dbt (data build tool), however, dbt's (data build tool) development workflow and UI can open up data transformation and modeling work to non-data engineering teams. Looker might also be able to define data models via LookML with a …
I wouldn't recommend Data Integrator over SSIS, but it may be a better tool than other ETL offerings. Having a background in the Microsoft Stack, I am hesitant to recommend any other products especially in a Microsoft-centric company. If this was the only option to use, I think it does a good job at integrating systems after taking time to learn the approach. "Maps" in Data Integrator is definitely different than "process flow" philosophy in SSIS.
dbt (Data Build Tool) is best suited for doing the data transformation. dbt is just a transformation tool and it is not suitable for building a data pipeline which requires extraction of data and loading. dbt is well suited for SQL based transformation logic and it is less appropriate when transformation logic requires python.
Slow load times of the dbt cloud environment (they're working on it via a new UI though)
More out-of-the-box solutions for managing procedures, functions, etc would be nice to have, but honestly, it's pretty easy to figure out how to adapt dbt macros
dbt is very easy to use. Basically if you can write SQL, you will be able to use dbt to get what you need done. Of course more advanced users with more technical skills can do more things.
As I stated earlier, DataConnect makes a head-to-head run against the others. It has advantages and disadvantages compared to the competitors. The choice really comes down to pricing and specific requirement needs for the project you are planning. Questions such as On-premise or Cloud, Security, and Specific Connectors available will drive your decision.
Matillion is graphical versus dbt, which is SQL code-based (that, of course, is a matter of personal preference and not an objective advantage). The integrated testing, documentation generation, lineage, etc., were additional criteria that led us to choose dbt.