HDInsight is an implementation of the Apache Hadoop technology stack on the Microsoft Azure cloud platform: It is based on the Hortonworks Hadoop distribution. Microsoft Azure HDInsight includes implementations of Apache Spark, HBase, Storm, Pig, Hive, Sqoop, Oozie, Ambari, etc. It also integrates with with business intelligence (BI) tools such as Power BI, Excel, SQL Server Analysis Services, and SQL Server Reporting Services.
N/A
IBM watsonx.data
Score 8.8 out of 10
N/A
Watsonx.data is presented as an open, hybrid and governed data store that makes it possible for enterprises to scale analytics and AI with a fit-for-purpose data store, built on an open lakehouse architecture, supported by querying, governance and open data formats to access and share data.
Well suited: A tiny-mid sized company with no immediate plans of growing the volume of their data processing, that can afford long response times from support. Also it helps if you are not prone to put your hands on Linux and Spark configuration. In fact, it can make things go really faster if you also work with the bundle-in Jupyter. And, if you need to perform some diagnostics and / or administrative tasks, that's full of tools to find an understand the Root Cause. Ideal for non experts. Less appropriate: Big Data company, intense on demand cluster creation, mission critical, costs reduction, latest versions of libraries required, sophisticate customizations required.
Real-time transaction processing (both reads and writes) is where DataStax Enterprise shines. It's very fast with linear scalability should more resources be needed. Additional nodes are added very easily. DataStax Enterprise on its own (without Solr or Spark enabled) isn't well suited for long complicated reports. The data model doesn't support joining multiple tables together which is common in BI reporting.
Datastax Cassandra provides high availability and good performance for a database. It is built on top of open source Apache Cassandra so you can always somewhat understand the internal functioning and why.
Datastax Cassandra is fairly simple to start using, you can install/setup your cluster and be productive in 1 day.
Datastax Cassandra provides a lot of good detailed documentation, and when starting, the detailed free videos on the Datastax site and documentation are very helpful.
Datastax Enterprise Edition of Cassandra provides more tools, good support, and quick response SLA for enterprise business support.
The only problem I have come across is when loading large volumes of data I sometimes get an error message, I assume this means something is corrupt from within. I would love a way for this to be resolved without having to start over.
Integration complexity with Security Tools while watsonx.Data is well-suited for native tools, but integration with third-party security tools requires custom connectors or manual ETL pipelines. which leads to an increase in setup time.
As an open source technology Cassandra can be readily used with or without any commercial support. DataStax provides value-added services and features, and in the end it is up to individual situations to strike a balance between the desirability of such support/service versus the associated cost.
Azure HDInsight is usable on the top of Azure Data Lake and gives us the benefit of analyzing large scale data workload in Hadoop. Usability and support from Microsoft are outstanding.
DataStax has a good community built around it and has amazing scalability options. Though the initial setup is a bit costly, in the long run, it makes up for it. It also has powerful monitoring tools and a clean UI.
Inexpert, isolated teams... not good for support an excessively complex platform. Lots of weeks or months for a complex problem troubleshoot. Many time lost stuck on MindTree, before the case was finally escalated with Microsoft!
We have had a few situations where we caused an outage or something has gone wrong and we are able to get a support person to offer live help within minutes. The escalation process is excellent - the best I've seen - and the support team is incredibly strong. Outside of emergencies, the team is very helpful with general questions and working through data model exercises and the subscription I believe still comes with some hours to help get the data model reviewed.
At this time I have not used any other similar products... I am open to it but Azure HDInsight and its components really work well for our organization.
Pinecone and IBM watsonx.data (Milvus in our case) both work great as a full-managed cloud-based vector database. We selected IBM watsonx.data because it integrates well with watson.ai and is a little more beginner friendly than Pinecone, but I think both are great anyway.
