Apache Kafka is an open-source stream processing platform developed by the Apache Software Foundation written in Scala and Java. The Kafka event streaming platform is used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications.
N/A
Microsoft Power Automate
Score 8.2 out of 10
N/A
Microsoft Power Automate is an advanced automation platform offering a range of features, including AI-powered automation, robotic process automation (RPA), business process automation (BPA), digital process automation (DPA), and process/task mining. The platform aims to empower organizations to securely automate their operations at scale by leveraging low-code and AI technologies.
Apache Kafka is well-suited for most data-streaming use cases. Amazon Kinesis and Azure EventHubs, unless you have a specific use case where using those cloud PaAS for your data lakes, once set up well, Apache Kafka will take care of everything else in the background. Azure EventHubs, is good for cross-cloud use cases, and Amazon Kinesis - I have no real-world experience. But I believe it is the same.
If you are in a team that needs to develop automated solutions but have no coding experience and no access to a developer, Power Automate is one of the best places to start. It is incredibly flexible and likely has triggers and actions that can connect with your platforms, especially if you use the Microsoft stack. Additionally, using webhooks, you can connect to many different endpoints to get systems working together.
Really easy to configure. I've used other message brokers such as RabbitMQ and compared to them, Kafka's configurations are very easy to understand and tweak.
Very scalable: easily configured to run on multiple nodes allowing for ease of parallelism (assuming your queues/topics don't have to be consumed in the exact same order the messages were delivered)
Not exactly a feature, but I trust Kafka will be around for at least another decade because active development has continued to be strong and there's a lot of financial backing from Confluent and LinkedIn, and probably many other companies who are using it (which, anecdotally, is many).
Sometimes it becomes difficult to monitor our Kafka deployments. We've been able to overcome it largely using AWS MSK, a managed service for Apache Kafka, but a separate monitoring dashboard would have been great.
Simplify the process for local deployment of Kafka and provide a user interface to get visibility into the different topics and the messages being processed.
Learning curve around creation of broker and topics could be simplified
I REALLY wish I could just drop in a block of python or PowerShell code sometimes. The visual low-code things are great for moving around data but sometimes it would be faster if I could just tap out some code.
The DOM (Document Object Model) is not entirely apparent and takes a lot of trial and error, and sometimes it's even different between different connectors. I would love to be able to just see the path to access some specific variables from earlier processes.
The tool is very useful when used with its various native connectors, taking great advantage of the integration between the components and systems of the Office365 universe. However, its cost is still high, and automation using more advanced components containing AI resources becomes unfeasible for some companies. Due to the financial crisis that many companies are currently experiencing, investment in automation systems or tools is taking a back seat.
Apache Kafka is highly recommended to develop loosely coupled, real-time processing applications. Also, Apache Kafka provides property based configuration. Producer, Consumer and broker contain their own separate property file
Power Automate features a clean and intuitive user interface that allows users to create, manage, and monitor workflows easily. The UI is designed to be accessible to both technical and non-technical users, with drag-and-drop functionality for building workflows. Power Automate supports integration with a wide range of Microsoft and third-party applications. This flexibility in integration allows users to automate workflows across various systems, enhancing overall productivity and efficiency.
Support for Apache Kafka (if willing to pay) is available from Confluent that includes the same time that created Kafka at Linkedin so they know this software in and out. Moreover, Apache Kafka is well known and best practices documents and deployment scenarios are easily available for download. For example, from eBay, Linkedin, Uber, and NYTimes.
both Community support and Microsoft official support typically respond to (and resolve) reported issues in a VERY expedient manner, usually going above and beyond for education and bugfixing. I have been thoroughly impressed with the level of support I had been provided in the past.
after reviewing the main features of Power Automate, the Microsoft trainer focused on some of our real life use cases implementation, from simple to more advanced.
although it was productive, it is more difficult to stay focused and in a 7 hours a day online training (including screen share issues and the fact that the trainer just can't precisely show the exact location of your mistake)
I was part of the migration of classic SharePoint workflows and Nintex workflows. We didn't spend much time learning the new platform because of its user-friendly interface and intuitive drag-and-drop functionality. We have mostly O365 E1 and E3 licensed users, so they can participate in our Citizen Development program and utilize Power Automate to build day-to-day business processes.
I used other messaging/queue solutions that are a lot more basic than Confluent Kafka, as well as another solution that is no longer in the market called Xively, which was bought and "buried" by Google. In comparison, these solutions offer way fewer functionalities and respond to other needs.
I have many years of experience with Nintex applications and it stands up very well against Microsoft Power Automate. I hope Microsoft Power Automate can become the better workflow application. With the Integrated AI builder and continuous updates I expect they can become the better one in a few years.
Microsoft's professional services provide hands-on support throughout the implementation lifecycle of Power Automate.This includes initial setup, configuration, integration with existing systems, testing, and deployment. They ensure that workflows are correctly designed, optimized for performance, and aligned with security best practices.
Positive: Get a quick and reliable pub/sub model implemented - data across components flows easily.
Positive: it's scalable so we can develop small and scale for real-world scenarios
Negative: it's easy to get into a confusing situation if you are not experienced yet or something strange has happened (rare, but it does). Troubleshooting such situations can take time and effort.
