Azure Synapse Analytics is described as the former Azure SQL Data Warehouse, evolved, and as a limitless analytics service that brings together enterprise data warehousing and Big Data analytics. It gives users the freedom to query data using either serverless or provisioned resources, at scale. Azure Synapse brings these two worlds together with a unified experience to ingest, prepare, manage, and serve data for immediate BI and machine learning needs.
$4,700
per month 5000 Synapse Commit Units (SCUs)
Amazon Redshift
Score 7.9 out of 10
N/A
Amazon Redshift is a hosted data warehouse solution, from Amazon Web Services.
It's well suited for large, fastly growing, and frequently changing data warehouses (e.g., in startups). It's also suited for companies that want a single, relatively easy-to-use, centralized cloud service for all their data needs. Larger, more structured organizations could still benefit from this service by using Synapse Dedicated SQL Pools, knowing that costs will be much higher than other solutions. I think this product is not suited for smaller, simpler workloads (where an Azure SQL Database and a Data Factory could be enough) or very large scenarios, where it may be better to build custom infrastructure.
If the number of connections is expected to be low, but the amounts of data are large or projected to grow it is a good solutions especially if there is previous exposure to PostgreSQL. Speaking of Postgres, Redshift is based on several versions old releases of PostgreSQL so the developers would not be able to take advantage of some of the newer SQL language features. The queries need some fine-tuning still, indexing is not provided, but playing with sorting keys becomes necessary. Lastly, there is no notion of the Primary Key in Redshift so the business must be prepared to explain why duplication occurred (must be vigilant for)
[Amazon] Redshift has Distribution Keys. If you correctly define them on your tables, it improves Query performance. For instance, we can define Mapping/Meta-data tables with Distribution-All Key, so that it gets replicated across all the nodes, for fast joins and fast query results.
[Amazon] Redshift has Sort Keys. If you correctly define them on your tables along with above Distribution Keys, it further improves your Query performance. It also has Composite Sort Keys and Interleaved Sort Keys, to support various use cases
[Amazon] Redshift is forked out of PostgreSQL DB, and then AWS added "MPP" (Massively Parallel Processing) and "Column Oriented" concepts to it, to make it a powerful data store.
[Amazon] Redshift has "Analyze" operation that could be performed on tables, which will update the stats of the table in leader node. This is sort of a ledger about which data is stored in which node and which partition with in a node. Up to date stats improves Query performance.
It takes some time to setup a proper SQL Datawarehouse architecture. Without proper SSIS/automation scripts, this can be a very daunting task.
It takes a lot of foresight when designing a Data Warehouse. If not properly designed, it can be very troublesome to use and/or modify later on.
It takes a lot of effort to maintain. Businesses are continually changing. With that, a full time staff member or more will be required to maintain the SQL Data Warehouse.
We've experienced some problems with hanging queries on Redshift Spectrum/external tables. We've had to roll back to and old version of Redshift while we wait for AWS to provide a patch.
Redshift's dialect is most similar to that of PostgreSQL 8. It lacks many modern features and data types.
Constraints are not enforced. We must rely on other means to verify the integrity of transformed tables.
The data warehouse portion is very much like old style on-prem SQL server, so most SQL skills one has mastered carry over easily. Azure Data Factory has an easy drag and drop system which allows quick building of pipelines with minimal coding. The Spark portion is the only really complex portion, but if there's an in-house python expert, then the Spark portion is also quiet useable.
Just very happy with the product, it fits our needs perfectly. Amazon pioneered the cloud and we have had a positive experience using RedShift. Really cool to be able to see your data housed and to be able to query and perform administrative tasks with ease.
Microsoft does its best to support Synapse. More and more articles are being added to the documentation, providing more useful information on best utilizing its features. The examples provided work well for basic knowledge, but more complex examples should be added to further assist in discovering the vast abilities that the system has.
The support was great and helped us in a timely fashion. We did use a lot of online forums as well, but the official documentation was an ongoing one, and it did take more time for us to look through it. We would have probably chosen a competitor product had it not been for the great support
When client is already having or using Azure then it’s wise to go with Synapse rather than using Snowflake. We got a lot of help from Microsoft consultants and Microsoft partners while implementing our EDW via Synapse and support is easily available via Microsoft resources and blogs. I don’t see that with Snowflake
Than Vertica: Redshift is cheaper and AWS integrated (which was a plus because the whole company was on AWS). Than BigQuery: Redshift has a standard SQL interface, though recently I heard good things about BigQuery and would try it out again. Than Hive: Hive is great if you are in the PB+ range, but latencies tend to be much slower than Redshift and it is not suited for ad-hoc applications.
Redshift is relatively cheaper tool but since the pricing is dynamic, there is always a risk of exceeding the cost. Since most of our team is using it as self serve and there is no continuous tracking by a dedicated team, it really needs time & effort on analyst's side to know how much it is going to cost.
Our company is moving to the AWS infrastructure, and in this context moving the warehouse environments to Redshift sounds logical regardless of the cost.
Development organizations have to operate in the Dev/Ops mode where they build and support their apps at the same time.
Hard to estimate the overall ROI of moving to Redshift from my position. However, running Redshift seems to be inexpensive compared to all the licensing and hardware costs we had on our RDBMS platform before Redshift.
