Apache Airflow vs. Talend Data Integration

For a quick job scanning of status and deep-diving into job issues, details, and flows, AirFlow does a good job. No fuss, no muss. The low learning curve as the UI is very straightforward, and navigating it will be familiar after spending some time using it. Our requirements are pretty simple. Job scheduler, workflows, and monitoring. The jobs we run are >100, but still is a lot to review and troubleshoot when jobs don't run. So when managing large jobs, AirFlow dated UI can be a bit of a drawback.

Incentivized

This tool fits all kinds of organizations and helps to integrate data between many applications. We can use this tool as data integration is a key feature for all organizations. It is also available in the cloud, which makes the integration more seamless. The firm can opt for the required tools when there are no data integration needs.

Incentivized

• In charge of the ETL processes.
• As there is no incoming or outgoing data, we may handle the scheduling of tasks as code and avoid the requirement for monitoring.

Incentivized

• Talend Data Integration allows us to quickly build data integrations without a tremendous amount of custom coding (some Java and JavaScript knowledge is still required).
• I like the UI and it's very intuitive. Jobs are visual, allowing the team members to see the flow of the data, without having to read through the Java code that is generated.
• Dynamically table creation from new source.

• they should bring in some time based scheduling too not only event based
• they do not store the metadata due to which we are not able to analyze the workflows
• they only support python as of now for scripted pipeline writing

Incentivized

• Pricing for sure can be the area for improvement.
• Real time processing is slow as compared to other tools like Abinitio.
• While developing batches, it crashes a lot. It may be the issue with me, but I wanted to highlight it.

Incentivized

Easy to learn Easy to use Robust workflow orchestration framework Good in dependent job management

Incentivized

We use Talend Data Integration day in and day out. It is the best and easiest tool to jump on to and use. We can build a basic integration super-fast. We could build basic integrations as fast as within the hour. It is also easy to build transformations and use Java to perform some operations.

Incentivized

Good support, specially when it relates to PROD environment. The support team has access to the product development team. Things are internally escalated to development team if there is a bug encountered. This helps the customer to get quick fix or patch designed for problem exceptions. I have also seen support showing their willingness to help develop custom connector for a newly available cloud based big data solution

Incentivized

There are a number of reasons to choose Apache Airflow over other similar platforms- Integrations—ready-to-use operators allow you to integrate Airflow with cloud platforms (Google, AWS, Azure, etc) Apache Airflow helps with backups and other DevOps tasks, such as submitting a Spark job and storing the resulting data on a Hadoop cluster It has machine learning model training, such as triggering a Sage maker job.

Incentivized

In comparison with the other ETLs I used, Talend is more flexible than Data Services (where you cannot create complex commands). It is similar to Datastage speaking about commands and interfaces. It is more user-friendly than ODI, which has a metadata point of view on its own, while Talend is more classic. It has both on-prem and cloud approaches, while Matillion is only cloud-based.

Incentivized

• A lot of helpful features out-of-the-box, such as the DAG visualizations and task trees
• Allowed us to implement complex data pipelines easily and at a relatively low cost

Incentivized

• It’s only been a positive RoI with Talend given we’ve interfaced large datasets between critical on-Prem and cloud-native apps to efficiently run our business operations.
• 40K+ plots data, covering 1K+ crop varieties.
• 3K+ Customer & their credit data, 3K+ product inventory & pricing.

Incentivized