AWS Glue vs. dbt vs. SAP Master Data Governance

One of AWS Glue's most notable features that aid in the creation and transformation of data is its data catalog. Support, scheduling, and the automation of the data schema recognition make it superior to its competitors aside from that. It also integrates perfectly with other AWS tools. The main restriction may be integrated with systems outside of the AWS environment. It functions flawlessly with the current AWS services but not with other goods. Another potential restriction that comes to mind is that glue operates on a spark, which means the engineer needs to be conversant in the language.

Incentivized

The prerequisite is that you have a supported database/data warehouse and have already found a way to ingest your raw data. Then dbt is very well suited to manage your transformation logic if the people using it are familiar with SQL. If you want to benefit from bringing engineering practices to data, dbt is a great fit. It can bring CI/CD practices, version control, automated testing, documentation generation, etc. It is not so well suited if the people managing the transformation logic do not like to code (in SQL) but prefer graphical user interfaces.

Incentivized

The best use case of SAP Master Data Governance is centrally consolidating your organizational data so you can make sense of it and make business decisions using that data. We live in a world where data can be all over the place, and making sense of it is a skill of its own.

Incentivized

• It is extremely fast, easy, and self-intuitive. Though it is a suite of services, it requires pretty less time to get control over it.
• As it is a managed service, one need not take care of a lot of underlying details. The identification of data schema, code generation, customization, and orchestration of the different job components allows the developers to focus on the core business problem without worrying about infrastructure issues.
• It is a pay-as-you-go service. So, there is no need to provide any capacity in advance. So, it makes scheduling much easier.

Incentivized

• dbt supports version control through GIT, this allows teams to collaborate and track the data transformation logic.
• dbt allows us to build data models which helps to break complex transformation logic into simple and smaller logic.
• dbt is completely based on SQL which allows data analyst and data engineers to build the transformation logic.
• dbt can be easily integrated with snowflake.

Incentivized

• Data conversation and Data uploading into SAP S/4 HANA system using SAP Master Data Governance during SAP implementation phase.
• Centrally Data governance to have visibility of data in different systems with integration to SAP Master Data Governance, including SAP S/4, SAP EWM and SAP Ariba.
• Data Management and Data correction using MDM through SAP Master Data Governance for Materials, which is subjected to approval before finally using in the S/4.

Incentivized

• In-Stream schema registries feature people can not use this more efficiently
• in Connections feature they can add more connectors as well
• The crucial problem with AWS Glue is that it only works with AWS.

Incentivized

• Field-level lineage (currently at table level)
• Documentation inheritance - if a field is documented the downstream field of the same name could inherit the doc info
• Adding python model support (in beta now)

Incentivized

• We've had to design our own unique or alternative solutions to satisfy our business objectives because plugins don't always perform well with the main product.
• It can be sluggish at times and lacks functionality offered by rival MDM packages, as well as limited resources and documentation for certain areas such as hierarchy.

Incentivized

MDG has proven to work and to bring results once implemented with the right approach, right engagement and sponsorship. Given that it is a very good tool to govern and control master data. Mainly if your company runs SAP systems which there will be a very straight forward integration, not needing any additional middleware or technologies.

Incentivized

While easy to set up and manage monitoring for large datasets, its complexity can be a barrier for new users. Integration with AWS Ecosystem, Managed Monitoring, Dashboards and monitoring tools for AWS Glue are generally easy to set up and maintain, Automated Data Pipelines. Automates data pipeline creation, making it efficient for certain data integration

Incentivized

dbt is very easy to use. Basically if you can write SQL, you will be able to use dbt to get what you need done. Of course more advanced users with more technical skills can do more things.

Incentivized

Upto 8 1. Powerful Features: It offers a wide range of powerful features. 2. Complex Data Management: It helps companies to comply with large and complex regulation and audit processes. Missing 2: 1. Less interactive user interface or less modern look. 2. Limited integration with other tools

Incentivized

Amazon responds in good time once the ticket has been generated but needs to generate tickets frequent because very few sample codes are available, and it's not cover all the scenarios.

Incentivized

SAP provides great support for Master Data Governance (MDG). They work with us when we are faced with issues in standard solutions. SAP has a great set of Fiori tools that can be used for a better user experience. The issue with SAP is their web dynpro UI screens and they need to provide better support for performance issues that customers face.

Incentivized

critical to get management on board earlier and prioritize the resources

Incentivized

AWS Glue is a fully managed ETL service that automates many ETL tasks, making it easier to set AWS Glue simplifies ETL through a visual interface and automated code generation.

I actually don't know what the alternative to dbt is. I'm sure one must exist other than more 'roll your own' options like Apache Airflow, say, bu tin terms of super easy managed/cloud data transforms, dbt really does seem to be THE tool to use. It's $50/month per dev, BUT there's a FREE version for 1 dev seat with no read-only access for anyone else, so you can always start with that and then buy yourself a seat later.

Incentivized

An environment provided by SAP Master Data Governance seamless and integration with other SAP products is very good. Also it ensures compatibility and streamlines procedures, this integration is beneficial for businesses who have already invested in SAP products.

Incentivized

• We are using GLUE for our ETL purpose. it’s ease with other our AWS services makes our ROI, 100% ROI.
• One missing piece was compatibility with other data source for which we found a work around and made our data source as S3 only, so our dependencies on other data source is also reducing

Incentivized

• Simplified our BI layer for faster load times
• Increased the quality of data reaching our end users
• Makes complex transformations manageable

Incentivized

• MDG is definitely an investment that takes a few years to recuperate. (If you are looking at pure, direct financial benefits.)
• However, if you look at all the productivity gains in the business due to not having to stop the business constantly because of data issues, the payback period would be substantially shorter.
• In particular, we were able to reduce master data maintenance staffing levels while increasing quality and decreasing workload for business requesters and approvers.
• Not having to worry about data integrity issues has shortened the time to realize benefits for several of our major process change related projects and it has also decreased the risk of these projects running into significant delays (often at the last minute) due to master data being misjudged during the process resdesign.

Incentivized