Azure Synapse Analytics (Azure SQL Data Warehouse) vs. Databricks Lakehouse Platform (Unified Analytics Platform)

It's well suited for large, fastly growing, and frequently changing data warehouses (e.g., in startups). It's also suited for companies that want a single, relatively easy-to-use, centralized cloud service for all their data needs. Larger, more structured organizations could still benefit from this service by using Synapse Dedicated SQL Pools, knowing that costs will be much higher than other solutions. I think this product is not suited for smaller, simpler workloads (where an Azure SQL Database and a Data Factory could be enough) or very large scenarios, where it may be better to build custom infrastructure.

If you need a managed big data megastore, which has native integration with highly optimized Apache Spark Engine and native integration with MLflow, go for Databricks Lakehouse Platform. The Databricks Lakehouse Platform is a breeze to use and analytics capabilities are supported out of the box. You will find it a bit difficult to manage code in notebooks but you will get used to it soon.

• Create data pipelines to connect with multiple data workspace(s) and external data
• Ability to connect with Azure Data Lake (sequentially) for data warehousing
• Being able to manage connections and create integration runtimes (for onPrem data capture)

• Process raw data in One Lake (S3) env to relational tables and views
• Share notebooks with our business analysts so that they can use the queries and generate value out of the data
• Try out PySpark and Spark SQL queries on raw data before using them in our Spark jobs
• Modern day ETL operations made easy using Databricks. Provide access mechanism for different set of customers

• It takes some time to setup a proper SQL Datawarehouse architecture. Without proper SSIS/automation scripts, this can be a very daunting task.
• It takes a lot of foresight when designing a Data Warehouse. If not properly designed, it can be very troublesome to use and/or modify later on.
• It takes a lot of effort to maintain. Businesses are continually changing. With that, a full time staff member or more will be required to maintain the SQL Data Warehouse.

• Better Localized Testing
• When they were primarily OSS Spark; it was easier to test/manage releases versus the newer DB Runtime. Wish there was more configuration in Runtime less pick a version.
• Graphing Support went non-existent; when it was one of their compelling general engine.

The data warehouse portion is very much like old style on-prem SQL server, so most SQL skills one has mastered carry over easily. Azure Data Factory has an easy drag and drop system which allows quick building of pipelines with minimal coding. The Spark portion is the only really complex portion, but if there's an in-house python expert, then the Spark portion is also quiet useable.

Because it is an amazing platform for designing experiments and delivering a deep dive analysis that requires execution of highly complex queries, as well as it allows to share the information and insights across the company with their shared workspaces, while keeping it secured.

in terms of graph generation and interaction it could improve their UI and UX

Microsoft does its best to support Synapse. More and more articles are being added to the documentation, providing more useful information on best utilizing its features. The examples provided work well for basic knowledge, but more complex examples should be added to further assist in discovering the vast abilities that the system has.

One of the best customer and technology support that I have ever experienced in my career. You pay for what you get and you get the Rolls Royce. It reminds me of the customer support of SAS in the 2000s when the tools were reaching some limits and their engineer wanted to know more about what we were doing, long before "data science" was even a name. Databricks truly embraces the partnership with their customer and help them on any given challenge.

When client is already having or using Azure then it’s wise to go with Synapse rather than using Snowflake. We got a lot of help from Microsoft consultants and Microsoft partners while implementing our EDW via Synapse and support is easily available via Microsoft resources and blogs. I don’t see that with Snowflake

Databricks has a much better edge than Synapse in hundred different ways. Databricks has Photon engine, faster available release in cloud and databricks does not run on Open source spark version so better optimization, better performance and better agility and all kind of performance boost can be achieved in Databricks rather Open source synapse spark

Basically, the billing is predictable, and this all about it.

The problem with this tool and all other ones that are at the top of the industry, it's so expensive that soon as another one will be on the market and deliver the same or different value, it will be catastrophic for them. So you get the fact that they are cashing every dime right now like SAS or Hadoop once did. Now, look at them

Again, another level of professional services, this is not their biggest strength but this is the cherry on top. I couldn't think about any other professional services like this one. Now I'm talking about meaningful services that really help out our project and delivery.

• We have had an improvement in our overall processing time
• Cost was much lower than most of its competitors
• Our reporting needs have grown and housing the data here has been great

• Machine learning is a very new concept and not many universities offer to teach it. My school and a few others have been utilizing Databricks as one of the tools to teach and learn machine learning. By doing this, my university is creating a strong future workforce for the job market.