Apache Kafka is an open-source stream processing platform developed by the Apache Software Foundation written in Scala and Java. The Kafka event streaming platform is used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications.
N/A
Control-M
Score 9.3 out of 10
N/A
Control-M from BMC is a platform for integrating, automating, and orchestrating application and data workflows in production across complex hybrid technology ecosystems. It provides deep operational capabilities, delivering speed, scale, security, and governance.
Apache Kafka is well-suited for most data-streaming use cases. Amazon Kinesis and Azure EventHubs, unless you have a specific use case where using those cloud PaAS for your data lakes, once set up well, Apache Kafka will take care of everything else in the background. Azure EventHubs, is good for cross-cloud use cases, and Amazon Kinesis - I have no real-world experience. But I believe it is the same.
Scenarios where it's well suited are where you're running jobs across platforms. We have processes on our mainframe that have to complete our daily batch processing and the controlling handles the scheduling of the daily batch processing. Then once that's done, it moves into executing stuff on servers like SQL loads and transferring files to load into SQL and then also running reporting on the SQL servers when the data's been loaded and is available. It makes for a really smooth transition of the overnight processing the bank has to go through. That's our main function that we use it for currently. It handles it very well.
Really easy to configure. I've used other message brokers such as RabbitMQ and compared to them, Kafka's configurations are very easy to understand and tweak.
Very scalable: easily configured to run on multiple nodes allowing for ease of parallelism (assuming your queues/topics don't have to be consumed in the exact same order the messages were delivered)
Not exactly a feature, but I trust Kafka will be around for at least another decade because active development has continued to be strong and there's a lot of financial backing from Confluent and LinkedIn, and probably many other companies who are using it (which, anecdotally, is many).
Workload change manager is one of our favorite features of this product. It enforces standards, which is a huge benefit. Users don't just make crazy changes that cause issues with other jobs. It does great promotions from one environment to another, transforming all the data to match the next environment.
Sometimes it becomes difficult to monitor our Kafka deployments. We've been able to overcome it largely using AWS MSK, a managed service for Apache Kafka, but a separate monitoring dashboard would have been great.
Simplify the process for local deployment of Kafka and provide a user interface to get visibility into the different topics and the messages being processed.
Learning curve around creation of broker and topics could be simplified
Certificate on all levels: Each agent uses different keystrokes for different functions, such as AI, web service calls, and MFT.
It's really hard to manage these keystrokes.
Trust stores, in case control-m, act as servers; it should be simple to implement certificates from a corporate CA.
Some functions must be performed in the fat client, while others can only be performed through the web interface. This should be streamlined as soon as possible.
It is one of the best solutions on the market, in terms of innovation, reliability and stability. Control-M provides security when used by the largest companies in Mexico such as banks, department stores and logistics. It has proven to be able to integrate with new technologies on the market and provide almost 100% availability, thanks to its automatic FailOver scheme.
Apache Kafka is highly recommended to develop loosely coupled, real-time processing applications. Also, Apache Kafka provides property based configuration. Producer, Consumer and broker contain their own separate property file
It has an excellent graphical user interface and robust functionalities, which any scheduling tool currently has in the current market. Strengthening security measures, such as role-based access control and encryption, is essential to protect sensitive data. Providing tools to optimize resource utilization and reduce costs would be valuable. Limited options for customizing the user interface can hinder productivity. Allowing users to tailor the interface to their needs would enhance the experience.
very good performance and robust.Control-M has received positive reviews for its performance, with some users calling it a reliable and versatile platform: TrustRadiusControl-M has an overall score of 8.1 out of 10 from 89 reviews, with high marks for multi-platform scheduling and central monitoring. Gartner Peer InsightsBMC Helix Control-M received an overall experience rating of 4.8, with 80% of ratings being 5 stars. IT Central StationIn June 2018, Control-M was ranked the #1 tool by average user ratings. EMA Radar for Workload AutomationControl-M received the highest overall score in EMA's 2019 EMA Radar Report for Workload Automation. Control-M is a high-end solution that can help with:Reducing workflow deployment timeReducing manual tasksReducing administration costsImproving SLAsAccelerating application deliveryAutomated agent and client deploymentReal-time analysis
Support for Apache Kafka (if willing to pay) is available from Confluent that includes the same time that created Kafka at Linkedin so they know this software in and out. Moreover, Apache Kafka is well known and best practices documents and deployment scenarios are easily available for download. For example, from eBay, Linkedin, Uber, and NYTimes.
He contactado varias veces con el soporte de BMC y ha sido bastante bueno, siempre han sabido darme una solución a lo que he pedido. Esta vez quiero hacer alguna nueva pregunta, pero no se si se me podrá contestar, ya que es algo que tal vez fuera de otro rango y no pertenezca a ellos.
awesome training.they have explained about best practices, the trainer is awesome.awesome training.they have explained about best practices, the trainer is awesome.awesome training.they have explained about best practices, the trainer is awesome.awesome training.they have explained about best practices, the trainer is awesome.awesome training.they have explained about best practices, the trainer is awesome.
As HA we have depend on the external DB, why don't we have HA feasibility with embedded DB. As with external DB, there are performance issues and fine tuning the DB. As if its embedded DB, Control-M it self take care of the functionality.
I used other messaging/queue solutions that are a lot more basic than Confluent Kafka, as well as another solution that is no longer in the market called Xively, which was bought and "buried" by Google. In comparison, these solutions offer way fewer functionalities and respond to other needs.
Redwood RunMyJobs seemed impressive as well since it was also cross-platform, flexible, and versatile. However, we went with Control-M because it was less expensive and we were already using it on our mainframe environment. The distributed version of Control-M was much more user-friendly due to the GUI instead of the command-line interface used on the mainframe.
awesome product.Control-M delivers advanced operational capabilities easily consumed by Dev, Ops, data teams, and lines of business.Control-M Workflow InsightsApplication and data workflow observability: Increased confidence that SLAs are being met for Control-M users and IT leadersComprehensive control and management capabilities: Enhanced dashboards and reporting with constant telemetry and intelligent analysis on executing workflowsSelf-service visibility: In-depth reporting to help teams work autonomously
Positive: Get a quick and reliable pub/sub model implemented - data across components flows easily.
Positive: it's scalable so we can develop small and scale for real-world scenarios
Negative: it's easy to get into a confusing situation if you are not experienced yet or something strange has happened (rare, but it does). Troubleshooting such situations can take time and effort.
Control-M has improved service delivery times, reliability and quality of batch processing.
It has simplified the management of the operation and the use of the alert system has made it possible to act in a coordinated and efficient manner to solve problems.
The implementation of policies has made it possible to make greater use of Control-M and thus reduce development costs that are generated unnecessarily when the potential of the system is not considered.