Astro by Astronomer vs. Azure Data Factory

Astronomer is well suited for workflow and dependency management for enterprise-level data lakes. It is not a product for data processing though. Different source systems can be integrated, it also provides powerful interfaces for alerting and monitoring. Easy to build DAGs, graphical UI, API support makes the product more user-friendly as well. Astronomer also does a great job on user training.

Incentivized

Azure Data Factory is a great data integration tool for developing a cloud data platform, especially within the Azure ecosystem. Azure Data Factory is very good for the Data Ingestion part. It can work for simple data transformation with its Data Flow, but it will also need cluster configuration, and there is some cost. Also, it is an excellent tool for orchestrating data pipelines. But for complex data transformations, you may need to use technologies like Databricks and PySpark.

Incentivized

• Workflow management
• Wide availability of plugins
• Dependency management on upstream

Incentivized

• Azure Data Factory supports a vast array of source and destination connectors, both from within the Microsoft ecosystem (like Azure Blob Storage, Azure SQL Database, Azure Cosmos DB) and external platforms (like Amazon S3, Google Cloud Storage, SAP, Salesforce, and many more).
• Azure Data Factory's Mapping Data Flows provides a code-free environment to design data transformations visually. Users can drag and drop elements to create complex ETL (Extract, Transform, Load) and ELT (Extract, Load, Transform) processes without needing to write any code.
• Azure Data Factory provides a unified monitoring dashboard that offers a holistic view of all pipeline activities. I think this makes it easier for users to track the status of various jobs, identify failures, and pinpoint bottlenecks.

Incentivized

• More language agnostic
• Flexible fork and join capabilities
• Near real time UI updates in case of deployment of enhanced DAGs

Incentivized

• For complex JSON when it comes to mapping nested attribute it's not easy to flatten out
• Data Factory V1 does not have a good implementation experience as compared to V2
• Work with on premise solutions sometimes is not too friendly because you will need to set a VPN

Incentivized

So far product has performed as expected. We were noticing some performance issues, but they were largely Synapse related. This has led to a shift from Synapse to Databricks. Overall this has delayed our analytic platform. Once databricks becomes fully operational, Azure Data Factory will be critical to our environment and future success.

Incentivized

We have not had need to engage with Microsoft much on Azure Data Factory, but they have been responsive and helpful when needed. This being said, we have not had a major emergency or outage requiring their intervention. The score of seven is a representation that they have done well for now, but have not proved out their support for a significant issue

Incentivized

Astronomer is a fast, secure, scalable workload management solution. It provides world-class user training along with easy to interact support.

Incentivized

The easy integration with other Microsoft software as well as high processing speed, very flexible cost, and high level of security of Microsoft Azure products and services stack up against other similar products.

Incentivized

• It helps to build scalable, available and low maintenance workloads
• Integrated Alerts and notifications helps to detect load issues in the early stages
• Ensures meeting data SLAs

Incentivized

• It is very useful and make things easier
• Debugging can improve
• Its better suited than other products with the same objective

Incentivized