Apache Spark vs. AWS Glue

Overview
ProductRatingMost Used ByProduct SummaryStarting Price
Apache Spark
Score 8.9 out of 10
N/A
N/AN/A
AWS Glue
Score 7.8 out of 10
N/A
AWS Glue is a managed extract, transform, and load (ETL) service designed to make it easy for customers to prepare and load data for analytics. With it, users can create and run an ETL job in the AWS Management Console. Users point AWS Glue to data stored on AWS, and AWS Glue discovers data and stores the associated metadata (e.g. table definition and schema) in the AWS Glue Data Catalog. Once cataloged, data is immediately searchable, queryable, and available for ETL.
$0.44
billed per second, 1 minute minimum
Pricing
Apache SparkAWS Glue
Editions & Modules
No answers on this topic
per DPU-Hour
$0.44
billed per second, 1 minute minimum
Offerings
Pricing Offerings
Apache SparkAWS Glue
Free Trial
NoNo
Free/Freemium Version
NoNo
Premium Consulting/Integration Services
NoNo
Entry-level Setup FeeNo setup feeNo setup fee
Additional Details——
More Pricing Information
Community Pulse
Apache SparkAWS Glue
Considered Both Products
Apache Spark
Chose Apache Spark
Spark is simply awesome to work on with any data sets and also has an in-memory database which makes it very flexible.
AWS Glue
Chose AWS Glue
AWS Glue is a fully managed ETL service that automates many ETL tasks, making it easier to set AWS Glue simplifies ETL through a visual interface and automated code generation.
Top Pros
Top Cons
Best Alternatives
Apache SparkAWS Glue
Small Businesses

No answers on this topic

IBM SPSS Modeler
IBM SPSS Modeler
Score 7.4 out of 10
Medium-sized Companies
Cloudera Manager
Cloudera Manager
Score 9.9 out of 10
IBM InfoSphere Information Server
IBM InfoSphere Information Server
Score 8.0 out of 10
Enterprises
IBM Analytics Engine
IBM Analytics Engine
Score 7.8 out of 10
IBM InfoSphere Information Server
IBM InfoSphere Information Server
Score 8.0 out of 10
All AlternativesView all alternativesView all alternatives
User Ratings
Apache SparkAWS Glue
Likelihood to Recommend
9.3
(24 ratings)
7.1
(8 ratings)
Likelihood to Renew
10.0
(1 ratings)
-
(0 ratings)
Usability
8.7
(4 ratings)
7.0
(1 ratings)
Support Rating
8.7
(4 ratings)
7.0
(1 ratings)
User Testimonials
Apache SparkAWS Glue
Likelihood to Recommend
Apache
Well suited: To most of the local run of datasets and non-prod systems - scalability is not a problem at all. Including data from multiple types of data sources is an added advantage. MLlib is a decently nice built-in library that can be used for most of the ML tasks. Less appropriate: We had to work on a RecSys where the music dataset that we used was around 300+Gb in size. We faced memory-based issues. Few times we also got memory errors. Also the MLlib library does not have support for advanced analytics and deep-learning frameworks support. Understanding the internals of the working of Apache Spark for beginners is highly not possible.
Read full review
Amazon AWS
One of AWS Glue's most notable features that aid in the creation and transformation of data is its data catalog. Support, scheduling, and the automation of the data schema recognition make it superior to its competitors aside from that. It also integrates perfectly with other AWS tools. The main restriction may be integrated with systems outside of the AWS environment. It functions flawlessly with the current AWS services but not with other goods. Another potential restriction that comes to mind is that glue operates on a spark, which means the engineer needs to be conversant in the language.
Read full review
Pros
Apache
  • Rich APIs for data transformation making for very each to transform and prepare data in a distributed environment without worrying about memory issues
  • Faster in execution times compare to Hadoop and PIG Latin
  • Easy SQL interface to the same data set for people who are comfortable to explore data in a declarative manner
  • Interoperability between SQL and Scala / Python style of munging data
Read full review
Amazon AWS
  • It is extremely fast, easy, and self-intuitive. Though it is a suite of services, it requires pretty less time to get control over it.
  • As it is a managed service, one need not take care of a lot of underlying details. The identification of data schema, code generation, customization, and orchestration of the different job components allows the developers to focus on the core business problem without worrying about infrastructure issues.
  • It is a pay-as-you-go service. So, there is no need to provide any capacity in advance. So, it makes scheduling much easier.
Read full review
Cons
Apache
  • Memory management. Very weak on that.
  • PySpark not as robust as scala with spark.
  • spark master HA is needed. Not as HA as it should be.
  • Locality should not be a necessity, but does help improvement. But would prefer no locality
Read full review
Amazon AWS
  • In-Stream schema registries feature people can not use this more efficiently
  • in Connections feature they can add more connectors as well
  • The crucial problem with AWS Glue is that it only works with AWS.
Read full review
Likelihood to Renew
Apache
Capacity of computing data in cluster and fast speed.
Read full review
Amazon AWS
No answers on this topic
Usability
Apache
If the team looking to use Apache Spark is not used to debug and tweak settings for jobs to ensure maximum optimizations, it can be frustrating. However, the documentation and the support of the community on the internet can help resolve most issues. Moreover, it is highly configurable and it integrates with different tools (eg: it can be used by dbt core), which increase the scenarios where it can be used
Read full review
Amazon AWS
We give 7 rating because of usefulness in AWS world without worrying about infrastructure and services interaction, it’s pretty out of the box gives us the flexibility to interact with them and use them. we take the data source in s3 from external system and then transform it using other AWS services and putting it back for other external services to consume in S3 form.
Read full review
Support Rating
Apache
1. It integrates very well with scala or python. 2. It's very easy to understand SQL interoperability. 3. Apache is way faster than the other competitive technologies. 4. The support from the Apache community is very huge for Spark. 5. Execution times are faster as compared to others. 6. There are a large number of forums available for Apache Spark. 7. The code availability for Apache Spark is simpler and easy to gain access to. 8. Many organizations use Apache Spark, so many solutions are available for existing applications.
Read full review
Amazon AWS
Amazon responds in good time once the ticket has been generated but needs to generate tickets frequent because very few sample codes are available, and it's not cover all the scenarios.
Read full review
Alternatives Considered
Apache
Spark in comparison to similar technologies ends up being a one stop shop. You can achieve so much with this one framework instead of having to stitch and weave multiple technologies from the Hadoop stack, all while getting incredibility performance, minimal boilerplate, and getting the ability to write your application in the language of your choosing.
Read full review
Amazon AWS
AWS Glue is a fully managed ETL service that automates many ETL tasks, making it easier to set AWS Glue simplifies ETL through a visual interface and automated code generation.
Read full review
Return on Investment
Apache
  • Business leaders are able to take data driven decisions
  • Business users are able access to data in near real time now . Before using spark, they had to wait for at least 24 hours for data to be available
  • Business is able come up with new product ideas
Read full review
Amazon AWS
  • It had a positive impact on the way we build our data lake.
  • It is the single source of truth for data structure (schemas/tables/views).
Read full review
ScreenShots