Apache Kafka is an open-source stream processing platform developed by the Apache Software Foundation written in Scala and Java. The Kafka event streaming platform is used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications.
N/A
Microsoft BI (MSBI)
Score 8.4 out of 10
N/A
Microsoft BI is a business intelligence product used for data analysis and generating reports on server-based data. It features unlimited data analysis capacity with its reporting engine, SQL Server Reporting Services alongside ETL, master data management, and data cleansing.
Apache Kafka is well-suited for most data-streaming use cases. Amazon Kinesis and Azure EventHubs, unless you have a specific use case where using those cloud PaAS for your data lakes, once set up well, Apache Kafka will take care of everything else in the background. Azure EventHubs, is good for cross-cloud use cases, and Amazon Kinesis - I have no real-world experience. But I believe it is the same.
Microsoft BI is well suited for Stream analytics, easy data integration, report creation and UI/UX designs (limited but what all available are great ones) Microsoft BI may be less appropriate for handling huge number of datasets and difficult queries. It may also be difficult for a company with heavy data.
Really easy to configure. I've used other message brokers such as RabbitMQ and compared to them, Kafka's configurations are very easy to understand and tweak.
Very scalable: easily configured to run on multiple nodes allowing for ease of parallelism (assuming your queues/topics don't have to be consumed in the exact same order the messages were delivered)
Not exactly a feature, but I trust Kafka will be around for at least another decade because active development has continued to be strong and there's a lot of financial backing from Confluent and LinkedIn, and probably many other companies who are using it (which, anecdotally, is many).
Sometimes it becomes difficult to monitor our Kafka deployments. We've been able to overcome it largely using AWS MSK, a managed service for Apache Kafka, but a separate monitoring dashboard would have been great.
Simplify the process for local deployment of Kafka and provide a user interface to get visibility into the different topics and the messages being processed.
Learning curve around creation of broker and topics could be simplified
The race to perfect gathering of Non-Traditional datasets is on-going; with Microsoft arguably not the leader of the pack in this category.
Licensing options for PowerBI visualizations may be a factor. I.e. if you need to implement B2C PowerBI visualizations, the cost is considerably high especially for startups.
Some clients are still resistant putting their data on the cloud, which restricts lots of functionality to Power BI.
Microsoft BI is fundamental to our suite of BI applications. That being said, Northcraft Analytics is focused on delighting our customers, so if the underlying factors of our decision change, we would choose to re-write our BI applications on a different stack. Luckily, mathematics are the fundamental IP of our technology... and is portable across all BI platforms for the foreseeable future.
Apache Kafka is highly recommended to develop loosely coupled, real-time processing applications. Also, Apache Kafka provides property based configuration. Producer, Consumer and broker contain their own separate property file
The Microsoft BI tools have great usability for both developers and end users alike. For developers familiar with Visual Studio, there is little learning curve. For those not, the single Visual Studio IDE means not having to learn separate tools for each component. For end-users, the web interface for SSRS is simple to navigate with intuitive controls. For ad-hoc analysis, Excel can connect directly to SSAS and provide a pivot table like experience which is familiar to many users. For database development, there is beginning to be some confusion, as there are now three tool choices (VS, SSMS, Azure Data Studio) for developers. I would like to see Azure Data Studio become the superset of SSMS and eventually supplant it.
SQL Server Reporting Services (SSRS) can drag at times. We created two report servers and placed them under an F5 load balancer. This configuration has worked well. We have seen sluggish performance at times due to the Windows Firewall.
Support for Apache Kafka (if willing to pay) is available from Confluent that includes the same time that created Kafka at Linkedin so they know this software in and out. Moreover, Apache Kafka is well known and best practices documents and deployment scenarios are easily available for download. For example, from eBay, Linkedin, Uber, and NYTimes.
While support from Microsoft isn't necessarily always best of breed, you're also not paying the price for premium support that you would on other platforms. The strength of the stack is in the ecosystem that surrounds it. In contrast to other products, there are hundreds, even thousands of bloggers that post daily as well as vibrant user communities that surround the tool. I've had much better luck finding help with SQL Server related issues than I have with any other product, but that help doesn't always come directly from Microsoft.
I have used on-line training from Microsoft and from Pragmatic Works. I would recommend Pragmatic Works as the best way to get up to speed quickly, and then use the Microsoft on-line training to deep dive into specific features that you need to get depth with.
We are a consulting firm and as such our best resources are always billing on client projects. Our internal implementation has weaknesses, but that's true for any company like ours. My rating is based on the product's ease of implementation.
I used other messaging/queue solutions that are a lot more basic than Confluent Kafka, as well as another solution that is no longer in the market called Xively, which was bought and "buried" by Google. In comparison, these solutions offer way fewer functionalities and respond to other needs.
We have used the built in ConnectWise Manager reports and custom reports. The reports provide static data. PowerBI shows us live data we can drill down into and easily adjust parameters. It's much more useful than a static PDF report.
Positive: Get a quick and reliable pub/sub model implemented - data across components flows easily.
Positive: it's scalable so we can develop small and scale for real-world scenarios
Negative: it's easy to get into a confusing situation if you are not experienced yet or something strange has happened (rare, but it does). Troubleshooting such situations can take time and effort.
As a SaaS provider we see being able to provide self-service BI to our client users as a competitive advantage. In fact the MSSQL enabled BI is a contributing factor to many winning RFPs we have done for prospective client organisations.
However MSSQL BI requires extensive knowledge and skills to design and develop data warehouses & data models as a foundation to support business analysts and users to interrogate data effectively and efficiently. Often times we find having strong in-house MSSQL expertise is a bless.
