Amazon EMR is a cloud-native big data platform for processing vast amounts of data quickly, at scale. Using open source tools such as Apache Spark, Apache Hive, Apache HBase, Apache Flink, Apache Hudi (Incubating), and Presto, coupled with the scalability of Amazon EC2 and scalable storage of Amazon S3, EMR gives analytical teams the engines and elasticity to run Petabyte-scale analysis.
N/A
Google BigQuery
Score 8.6 out of 10
N/A
Google's BigQuery is part of the Google Cloud Platform, a database-as-a-service (DBaaS) supporting the querying and rapid analysis of enterprise data.
$6.25
per TiB (after the 1st 1 TiB per month, which is free)
Amazon EMR (Elastic Map Reduce) compares well against GCP and Azure - but you need to be careful of the costs involved in spinning up such a cluster. It is easy to configure however and it is my preferred platform to deploy our solutions because of its ease of use.
We are running it to perform preparation which takes a few hours on EC2 to be running on a spark-based EMR cluster to total the preparation inside minutes rather than a few hours. Ease of utilization and capacity to select from either Hadoop or spark. Processing time diminishes from 5-8 hours to 25-30 minutes compared with the Ec2 occurrence and more in a few cases.
Google BigQuery really shines in scenarios requiring real-time analytics on large data streams and predictive analytics with its machine learning integration. Teams have been using it extensively all over. However, it may not be the best fit for organizations dealing with small datasets because of the higher costs. And also, it might not be the best fit for highly complex data transformations, where simpler or more specialized solutions could be more appropriate.
Amazon Elastic MapReduce works well for managing analyses that use multiple tools, such as Hadoop and Spark. If it were not for the fact that we use multiple tools, there would be less need for MapReduce.
MapReduce is always on. I've never had a problem getting data analyses to run on the system. It's simple to set up data mining projects.
Amazon Elastic MapReduce has no problems dealing with very large data sets. It processes them just fine. With that said, the outputs don't come instantaneously. It takes time.
Its serverless architecture and underlying Dremel technology are incredibly fast even on complex datasets. I can get answers to my questions almost instantly, without waiting hours for traditional data warehouses to churn through the data.
Previously, our data was scattered across various databases and spreadsheets and getting a holistic view was pretty difficult. Google BigQuery acts as a central repository and consolidates everything in one place to join data sets and find hidden patterns.
Running reports on our old systems used to take forever. Google BigQuery's crazy fast query speed lets us get insights from massive datasets in seconds.
Sometimes bootstrapping certain tools comes with debugging costs. The tools provided by some of the enterprise editions are great compared to EMR.
Like some of the enterprise editions EMR does not provide on premises options.
No UI client for saving the workbooks or code snippets. Everything has to go through submitting process. Not really convenient for tracking the job as well.
It is challenging to predict costs due to BigQuery's pay-per-query pricing model. User-friendly cost estimation tools, along with improved budget alerting features, could help users better manage and predict expenses.
The BigQuery interface is less intuitive. A more user-friendly interface, enhanced documentation, and built-in tutorial systems could make BigQuery more accessible to a broader audience.
We have to use this product as its a 3rd party supplier choice to utilise this product for their data side backend so will not be likely we will move away from this product in the future unless the 3rd party supplier decides to change data vendors.
I give Amazon EMR this rating because while it is great at simplifying running big data frameworks, providing the Amazon EMR highlights, product details, and pricing information, and analyzing vast amounts of data, it can be run slow, freeze and glitch sometimes. So overall Amazon EMR is pretty good to use other than some basic issues.
web UI is easy and convenient. Many RDBMS clients such as aqua data studio, Dbeaver data grid, and others connect. Range of well-documented APIs available. The range of features keeps expanding, increasing similar features to traditional RDBMS such as Oracle and DB2
There's a vast group of trained and certified (by AWS) professionals ready to work for anyone that needs to implement, configure or fix EMR. There's also a great amount of documentation that is accessible to anyone who's trying to learn this. And there's also always the help of AWS itself. They have people ready to help you analyze your needs and then make a recommendation.
BigQuery can be difficult to support because it is so solid as a product. Many of the issues you will see are related to your own data sets, however you may see issues importing data and managing jobs. If this occurs, it can be a challenge to get to speak to the correct person who can help you.
Snowflake is a lot easier to get started with than the other options. Snowflake's data lake building capabilities are far more powerful. Although Amazon EMR isn't our first pick, we've had an excellent experience with EC2 and S3. Because of our current API interfaces, it made more sense for us to continue with Hadoop rather than explore other options.
I have used Snowflake and DataGrip for data retrieval as well as Google BigQuery and can say that all these tools compete for head to head. It is very difficult to say which is better than the other but some features provided by Google BigQuery give it an edge over the others. For example, the reliability of Google is unmatchable by others. One thing that I really like is the ability to integrate Data Studio so easily with Google BigQuery.
Google Support has kindly provide individual support and consultants to assist with the integration work. In the circumstance where the consultants are not present to support with the work, Google Support Helpline will always be available to answer to the queries without having to wait for more than 3 days.
Pricing has been very reasonable for us. The first 10 GB of storage is free each month and costs start at 2 cents per GB per month after that. For example, if you store 1 terabyte (TB) for a month, then the cost would be $20. Streaming data inserts start at 1 cent per 200 megabytes (MBs). The first 1 TB of queries is free, with additional analysis at $5 per TB thereafter. Meta data operations are free.
Big Query helps reduce the bar for data analytics, ML and AI. BQ takes care of mundane tasks and streamlines for easy data processing, consumption. The most impressive thing is the ML and AI integration as SQL functions, so the need for moving data around is minimized.
The visuals of ML models is very helpful to fine tune training, model building and prediction, etc.
