Apache Beam vs. AWS Data Pipeline vs. Control-M

AWS Data Pipeline is a web service that helps you reliably process and move data between different AWS compute and storage services, as well as on-premises data sources, at specified intervals. With AWS Data Pipeline, you can regularly access your data where it’s stored, transform and process it at scale, and efficiently transfer the results to AWS services such as Amazon S3, Amazon RDS, Amazon DynamoDB, and Amazon EMR.

Incentivized

Anytime you have a process that has to do multiple things, transfer data, interact with other systems, Control-M is critical. Not only does it provide the insight to what is going on, but it also lets you keep tight audit controls over access, reduces the need to spend large amounts of time tracking down issues, reduces the need to write custom "code" to do integrations with other systems and helps you better manage and track critical SLAs for workflows across the business.

Incentivized

• Helps you easily create complex data processing workloads
• Fault tolerant
• Highly available

Incentivized

• The good thing is that there are so many connectors available. Control-M provides lots of features, and we are using almost 60 to 70% of them. Control-M is providing us with so much capability to use during our daily problem-solving.
• Most of the job creation is very simple and quick and worked as per expectations.
• Testing and debugging are also very easy, and you can test multiple scenarios using temporary changes during job runs.
• Log and output presentations are also very good, short and detailed.
• To monitor specific job net, we can create viewpoint, which can be use on daily basis.

Incentivized

• Pipeline Stuck in Pending Status
• Pipeline Component Stuck in Waiting for Runner Status
• EMR Cluster Fails With Error

Incentivized

• I haven't come across too many spots where I'm not happy with the product. Most of the shortfalls were in my knowledge of the product as opposed to the actual product. Currently we're having a little bit of an issue with the deployment of the software to the servers, but it's more of an "us" problem than a product problem. I can't really give any good examples of shortfalls of the product that I've found so far.

Incentivized

It is one of the best solutions on the market, in terms of innovation, reliability and stability. Control-M provides security when used by the largest companies in Mexico such as banks, department stores and logistics. It has proven to be able to integrate with new technologies on the market and provide almost 100% availability, thanks to its automatic FailOver scheme.

Incentivized

User experience is meeting my expectations. We had a manual checklist, which Control-M Reports has now replaced, that helped us check the jobs without any issues. So, being fair with the work, the ratings should also be fair. More to come as the AI progresses; this will not only help motivate the Control-M Developers but also lead to the development of advanced technology.

Incentivized

Secondary Instances: Control-M supports the installation of a secondary instance of the entire Control-M environment, Control-M/EM, or Control-M/Server.Automatic & Manual Failover: In case of a failure on the primary host, Control-M can automatically failover to the secondary host if using Oracle or MSSQL databases. Manual failover is also an option, enabling a controlled switch during planned maintenance.Fallback: After resolving the issue on the primary host, you can easily fall back to it, or even designate the secondary host as the new primary. Database Replication: For high availability, Control-M leverages database replication from the primary site to a disaster recovery site. While replication is essential, its implementation and maintenance are the user's responsibility.

good page load times, efficient report completion, and minimal impact on integrated systems. Specifically, the well-designed GUI contributes to a positive user experience, and the platform's ability to automate various stages of the workflow, including Big Data processes, is highlighted as a key strength. Fast Page Loads: Control-M is reported to have a responsive user interface with fast page load times, allowing users to quickly navigate and manage their workflows

Support is generally excellent. Getting lower priority ones resolved can take a while, but it's rare for something to have to be dumped in the "unfixable" bin. If you end up speaking to Houston or Tel Aviv, then you know you've got a "live one".

Incentivized

Very knowledgeable instructors provide a hands-on, collaborative learning experience and can interact directly with instructors to develop our Control-M skills. This format allows for immediate feedback, in-depth discussions, and tailored guidance, leading to a deeper understanding of Control-M concepts and practical application. Face-to-face interaction fosters higher engagement and a more dynamic learning environment.

Incentivized

Simple and easy to use web based, well paced. Available any time. All online courses are simple and easy to access and use. Very practical everyday use scenarios and solutions. Incorporates software simulations, learning games, and built-in assessments to enhance comprehension and engagement. Online subscriptions are regularly updated with the latest product information, ensuring users have access to the most current knowledge.

Incentivized

As HA we have depend on the external DB, why don't we have HA feasibility with embedded DB. As with external DB, there are performance issues and fine tuning the DB. As if its embedded DB, Control-M it self take care of the functionality.

Incentivized

AWS data pipelines are easy to use over data factory for data engineers

Incentivized

Control-M: Known for its comprehensive workload automation capabilities, handling complex job scheduling, dependency management, and IT process automation. TWS: Traditionally strong in Batch processing and job scheduling, focusing on high-performance computing environments. TIDEL: Offers a combination of workload automation and IT process management, often used in mainframe environments.AutoSys: Provides job scheduling and workflow management with a reputation for scalability and performance.

Incentivized

While Control-M offers flexibility with usage-based and subscription-based pricing, some users might prefer more predictable, upfront costs, especially for large-scale deployments. A potential area for improvement could be offering more options for fixed-term contracts with predictable pricing based on factors like the number of agents or jobs, providing a clearer budget for long-term planning

awesome product.Control-M delivers advanced operational capabilities easily consumed by Dev, Ops, data teams, and lines of business.Control-M Workflow InsightsApplication and data workflow observability: Increased confidence that SLAs are being met for Control-M users and IT leadersComprehensive control and management capabilities: Enhanced dashboards and reporting with constant telemetry and intelligent analysis on executing workflowsSelf-service visibility: In-depth reporting to help teams work autonomously

Strengths: The vendor provided strong post-sales support, timely issue resolution, and effective onboarding. Their technical team was knowledgeable and responsive, ensuring smooth integration and minimal disruption. Training resources and documentation were comprehensive. Areas for Improvement: While overall service was excellent, occasional delays in advanced customization or escalations slightly impacted timelines. More proactive optimization suggestions could further enhance value.

Incentivized

• Easy to use
• Data engineers are able to create the data pipelines quickly and effectively
• Scalable and Fault tolerant

Incentivized

• Since centralizing all our workflows in Control-M, we've cut end to end processing time by nearly 30%
• Before Control-M we were babysitting scripts, manually rerunning failed jobs, and chasing ghost errors. With automated recovery, smart notifications, and fewer failures slipping through the cracks, we have saved 3 hours a day across teams
• Our workflows success rate sits at 99.95% and when things do fail, they are pinpointed immediately

Incentivized