Likelihood to Recommend One of AWS Glue's most notable features that aid in the creation and transformation of data is its data catalog. Support, scheduling, and the automation of the data schema recognition make it superior to its competitors aside from that. It also integrates perfectly with other AWS tools. The main restriction may be integrated with systems outside of the AWS environment. It functions flawlessly with the current AWS services but not with other goods. Another potential restriction that comes to mind is that glue operates on a spark, which means the engineer needs to be conversant in the language.
Read full review If you can load your data first into your warehouse, dbt is excellent. It does the T(ransformation) part of ELT brilliantly but does not do the E(xtract) or L(oad) part. If you know SQL or your development team knows SQL, it's a framework and extension around that. So, it's easy to learn and easy to hire people with that technical skill (as opposed to specific Informatica,
SnapLogic , etc. experience). dbt uses plain text files and integrates with GitHub. You can easily see the changes made between versions. In GUI-based UIs it was always hard to tell what someone had changed. Each "model" is essentially a "SELECT" statement. You never need to do a "CREATE TABLE" or "CREATE VIEW" - it's all done for you, leaving you to work on the business logic. Instead of saying "FROM specific_db.schema.table" you indicate "FROM ref('my_other_model')". It creates an internal dependency diagram you can view in a DAG. When you deploy, the dependencies work like magic in your various environments. They also have great documentation, an active slack community, training, and support. I like the enhancements they have been making and I believe they are headed in a good direction.
Read full review Pros It is extremely fast, easy, and self-intuitive. Though it is a suite of services, it requires pretty less time to get control over it. As it is a managed service, one need not take care of a lot of underlying details. The identification of data schema, code generation, customization, and orchestration of the different job components allows the developers to focus on the core business problem without worrying about infrastructure issues. It is a pay-as-you-go service. So, there is no need to provide any capacity in advance. So, it makes scheduling much easier. Read full review user experience makes it easy to work with SQL and version control customer success team and the dbt (data build tool) community help establish best practices thorough and clear documentation Read full review Cons In-Stream schema registries feature people can not use this more efficiently in Connections feature they can add more connectors as well The crucial problem with AWS Glue is that it only works with AWS. Read full review Slow load times of the dbt cloud environment (they're working on it via a new UI though) More out-of-the-box solutions for managing procedures, functions, etc would be nice to have, but honestly, it's pretty easy to figure out how to adapt dbt macros Read full review Support Rating Amazon responds in good time once the ticket has been generated but needs to generate tickets frequent because very few sample codes are available, and it's not cover all the scenarios.
Read full review Alternatives Considered AWS Glue is a fully managed ETL service that automates many ETL tasks, making it easier to set AWS Glue simplifies ETL through a visual interface and automated code generation.
Read full review Most ETL pipeline products have a T layer, but dbt just does it better. The transformation is on steroids compared to the others. Also, just allows much more Adhoc solutions for very specific projects. Those ETL tools are probably better on the T part if you don't need too many transforms - also dbt is pretty much free dependent on how you work it, also extremely scalable.
Read full review Return on Investment It had a positive impact on the way we build our data lake. It is the single source of truth for data structure (schemas/tables/views). Read full review Simplified our BI layer for faster load times Increased the quality of data reaching our end users Makes complex transformations manageable Read full review ScreenShots