37 Ratings
<a href='https://www.trustradius.com/static/about-trustradius-scoring' target='_blank' rel='nofollow noopener noreferrer'>trScore algorithm: Learn more.</a>
Score 8.4 out of 100
24 Ratings
<a href='https://www.trustradius.com/static/about-trustradius-scoring' target='_blank' rel='nofollow noopener noreferrer'>trScore algorithm: Learn more.</a>
Score 8.6 out of 100

Highlights

Databricks and Azure HDInsight are solutions for processing big data workloads and tend to be deployed at larger enterprises. Databricks handles data ingestion, data pipeline engineering, and ML/data science with its collaborative workbook for writing in R, Python, etc. HDInsight is a managed cloud service that allows users to run open-source frameworks like Apache HadoopSpark, and Kafka, easier.

Features

Databricks and HDInsight are generally well-liked solutions for big data processing, but standout reasons include the following:

Databricks is praised for its core competencies; its data science notebook is better than alternatives (e.g. Jupyter Notebook) for enabling flexible and fast analysis on massive amounts of data while swapping between work in SQL, R, Scala, Python. Its open-source community documentation, available to all, is well regarded.

HDInsight benefits from features of Azure: it is highly available with a satisfactory SLA, and the service itself is regarded as a cost-effective way for processing and retrieving data stored on Hadoop, or Azure Data Lakes.

Limitations

A few limitations exist that might cause one to look elsewhere for big data processing needs.

Azure HDInsight is cost-effective but some say its cost can balloon if it is used for long-term frequently queried data warehousing vs on-prem solutions that may be superior for this use case. Additionally, some users report glitches and performance issues when loading or processing very large volumes of data.

Databricks is costly, as is its certification cost. Additionally, Databricks can be hard to use for non-technical users, who say its in-app help is unclear and hard to use. And a few say Databricks lacks good visualizations for displaying work.

Pricing

Databricks is available open-source and free via its community edition, or through its Enterprise Cloud editions, on Azure or AWS. Pricing can be complex.

Azure Databricks “Databricks Units” are priced on workload type (Data Engineering, Data Engineering Light, or Data Analytics) and service tier: Standard vs. Premium. Premium adds authentication, access features, and audit log. The Data Analytics workload is $.40 per DBU hour ($.55 premium tier) and includes data prep and data science notebook. The Data Engineering tier includes data pipeline and workload processing, for $.15 per DBU hour ($.30 Premium tier). Data Engineering Light is $.07 per DBU hour ($.22 Premium tier) and only allows users to run jobs.

Databricks AWS is also priced based on service tier (Standard, Premium, Enterprise) and workload type. Higher service tiers add Optimized Autoscaling, role-based access, federated IAM, HIPAA compliant storage, access lists for audit, and customer-managed keys. The Jobs Compute workload allows users to run data engineering pipelines and manage & clean data lakes (priced $.07, $.10, .$13 per service tier). The All-Purpose Compute service ($.40, $.55, $.65) is fully featured.

Azure HDInsight Clusters are billed on a per-minute basis; clusters run a group of nodes that vary depending on the component. Processing Hadoop, Spark, Interactive Query, Kafka, Storm, and HBase does not incur a component cost (Kafka requires managed disks, however) while HDInsight Machine Learning Services incurs a cost of $0.016 per core-hour, and adding the Enterprise Security Package incurs a cost of $0.01 per core-hour. On Azure Virtual Machines general-purpose nodes for HDInsight costs from $0.06 per hour (1 CPU, 2GB RAM) to $0.631 per hour (8 CPU, 64 GB RAM). Memory-optimized nodes are available at an incrementally higher rate ($0.184/hour to $5.415/hour for an instance with 64 vCPUs and 432 GB RAM), as well as compute-optimized nodes ($0.295/hour to $1.179/hour for an instance with 16 CPUs and 32 GB RAM). Dev/test discounts are available for users on a Visual Studio subscription plan.

Likelihood to Recommend

Azure HDInsight

If you want to save costs and just pay for what you use, I highly recommend it. It will help you also to work with data for your reports and analytics. on the other hand I think it could be the subscription you have but high volume of data make it slow but not so much. anyway I think it's really good because it's from Microsoft which always is friendly to use it as all the suit they have.
Anonymous | TrustRadius Reviewer

Databricks Unified Analytics Platform

Databricks has helped my teams write PySpark and Spark SQL jobs and test them out before formally integrating them in Spark jobs. Through Databricks we can create parquet and JSON output files. Datamodelers and scientists who are not very good with coding can get good insight into the data using the notebooks that can be developed by the engineers.
Anonymous | TrustRadius Reviewer

Feature Rating Comparison

Platform Connectivity

Azure HDInsight
Databricks Unified Analytics Platform
8.3
Connect to Multiple Data Sources
Azure HDInsight
Databricks Unified Analytics Platform
9.0
Extend Existing Data Sources
Azure HDInsight
Databricks Unified Analytics Platform
9.0
Automatic Data Format Detection
Azure HDInsight
Databricks Unified Analytics Platform
7.0

Data Exploration

Azure HDInsight
Databricks Unified Analytics Platform
6.0
Visualization
Azure HDInsight
Databricks Unified Analytics Platform
6.0
Interactive Data Analysis
Azure HDInsight
Databricks Unified Analytics Platform
6.0

Data Preparation

Azure HDInsight
Databricks Unified Analytics Platform
8.0
Interactive Data Cleaning and Enrichment
Azure HDInsight
Databricks Unified Analytics Platform
8.0
Data Transformations
Azure HDInsight
Databricks Unified Analytics Platform
9.0
Data Encryption
Azure HDInsight
Databricks Unified Analytics Platform
7.0
Built-in Processors
Azure HDInsight
Databricks Unified Analytics Platform
8.0

Platform Data Modeling

Azure HDInsight
Databricks Unified Analytics Platform
8.3
Multiple Model Development Languages and Tools
Azure HDInsight
Databricks Unified Analytics Platform
9.0
Automated Machine Learning
Azure HDInsight
Databricks Unified Analytics Platform
8.0
Single platform for multiple model development
Azure HDInsight
Databricks Unified Analytics Platform
9.0
Self-Service Model Delivery
Azure HDInsight
Databricks Unified Analytics Platform
7.0

Model Deployment

Azure HDInsight
Databricks Unified Analytics Platform
7.5
Flexible Model Publishing Options
Azure HDInsight
Databricks Unified Analytics Platform
7.0
Security, Governance, and Cost Controls
Azure HDInsight
Databricks Unified Analytics Platform
8.0

Pros

Azure HDInsight

  • Data is presented without interfering others (IT or other dept).
  • Data is managed properly and is available for retrievable any time.
  • Legacy use of CD/DVD and Pendrive are not required.
Anonymous | TrustRadius Reviewer

Databricks Unified Analytics Platform

  • Extremely Flexible in Data Scenarios
  • Fantastic Performance
  • DB is always updating the system so we can have latest features.
Anonymous | TrustRadius Reviewer

Cons

Azure HDInsight

  • Spark version is old and crappy.
  • Lack of integration with other Azure platforms.
  • There is more room for improvement in workload based scaling.
  • Not easy to use. Log report hardly shows anything and the interface is not user friendly.
Partha Protim Pegu | TrustRadius Reviewer

Databricks Unified Analytics Platform

  • The navigation through which one would create a workspace is a bit confusing at first. It takes a couple minutes to figure out how to create a folder and upload files since it is not the same as traditional file systems such as box.com
  • Also, when you create a table, if you forgot to copy the link where the table is stored, it is hard to relocate it. Most of the time I would have to delete the table and re-created.
Ann Le | TrustRadius Reviewer

Usability

Azure HDInsight

Azure HDInsight 8.2
Based on 5 answers
Azure HDInsight is usable on the top of Azure Data Lake and gives us the benefit of analyzing large scale data workload in Hadoop. Usability and support from Microsoft are outstanding.
Krishn Garg | TrustRadius Reviewer

Databricks Unified Analytics Platform

Databricks Unified Analytics Platform 9.0
Based on 1 answer
This has been very useful in my organization for shared notebooks, integrated data pipeline automation and data sources integrations. Integration with AWS is seamless. Non tech users can easily learn how to use Databricks. You can have your company LDAP connect to it for login based access controls to some extent
Anonymous | TrustRadius Reviewer

Support Rating

Azure HDInsight

Azure HDInsight 8.2
Based on 5 answers
Well suited in advertising and marketing to serve the business purpose and best fit in the transformation of high volume data.
Krishn Garg | TrustRadius Reviewer

Databricks Unified Analytics Platform

No score
No answers yet
No answers on this topic

Alternatives Considered

Azure HDInsight

At this time I have not used any other similar products... I am open to it but Azure HDInsight and its components really work well for our organization.
Kristin Page | TrustRadius Reviewer

Databricks Unified Analytics Platform

Easier to set up and get started. Less of a learning curve.
Anonymous | TrustRadius Reviewer

Return on Investment

Azure HDInsight

  • ROI is of course there, as no legacy software for data presentation.
  • No manual intervention for data retrieval.
  • Data is available anywhere as requested.
Anonymous | TrustRadius Reviewer

Databricks Unified Analytics Platform

  • Rapid growth of analytics within our company.
  • Cost model aligns with usage allowing us to make a reasonable initial investment and scale the cost as we realize the value.
  • Platform is easy to learn and Databricks provides excellent support and training.
  • Platform does not require a large DevOPs investment
Anonymous | TrustRadius Reviewer

Pricing Details

Azure HDInsight

General

Free Trial
Free/Freemium Version
Premium Consulting/Integration Services
Entry-level set up fee?
No

Databricks Unified Analytics Platform

General

Free Trial
Free/Freemium Version
Premium Consulting/Integration Services
Entry-level set up fee?
No

Rating Summary

Likelihood to Recommend

Azure HDInsight
8.2
Databricks Unified Analytics Platform
8.9

Usability

Azure HDInsight
8.2
Databricks Unified Analytics Platform
9.0

Support Rating

Azure HDInsight
8.2
Databricks Unified Analytics Platform

Add comparison