Skip to main content
TrustRadius
Apache Spark

Apache Spark

Overview

Learn from top reviewers

Apache Spark: Lightning-Fast Distributed Computing with a Learning Curve

Rating: 10 out of 10
August 18, 2023
IncentivizedVetted ReviewVerified User
If you are working on large and big scale data with analytics - don't go further without the use of Apache Spark! One of the projects that I was involved in using Apache Spark...
Continue reading
Ananth Gouri
Professional - Engineering
Assistant Professor
The National Institute of Engineering, Mysuru
Apache Spark4 years of experience

Lightning Fast In-Memory Cluster Computing Framework

Rating: 10 out of 10
August 30, 2022
Vetted ReviewVerified User
Earlier we were using RDBMS like Oracle for retail and eCommerce data. We faced challenges such as cost, performance, and a huge amount of transactions coming in. After a lot ...
Continue reading
Riyaz Khan
Team Lead - Information Technology
Staff Engineer
Nagarro
Apache Spark5 years of experience

Apache Spark is the next generation of big data computing.

Rating: 9 out of 10
April 18, 2022
Vetted ReviewVerified User
We need to calculate risk-weighted assets (RWA) daily and monthly for different positions the bank holds on a T+1 basis. The volume of calculations is large: more than million...
Continue reading
Steven Li
Consultant - Information Technology
Senior Software Developer (Consultant)
Morgan Stanley
Apache Spark1 year of experience

Apache Spark - your go to technology for distributed data processing

Rating: 9 out of 10
May 3, 2021
IncentivizedVetted ReviewVerified User
We are using Apache Spark in Digital - Data teams to build data products and help business teams to take data-driven decisions.We use Apache Spark to source that from differen...
Continue reading
Surendranatha Reddy Chappidi
Engineer - Information Technology
Senior Data Engineer
A.P. Moller - Maersk
Apache Spark4 years of experience

Epic Computation Engine Framework

Rating: 9 out of 10
November 8, 2020
Vetted ReviewVerified User
Apache Spark is being used by our organization for writing ETL applications. It enables us to ingest thousands of records of data to database tables.
Continue reading
Chetan Munegowda
Engineer - Information Technology
Software Engineer
SemanticBits
Apache Spark1 year of experience
Log in with LinkedIn to see content from your network
Return to navigation

Product Demos

Spark Project | Spark Tutorial | Online Spark Training | Intellipaat

YouTube

Spark SQL Tutorial | Spark SQL Using Scala | Apache Spark Tutorial For Beginners | Simplilearn

YouTube

Apache Spark Full Course | Apache Spark Tutorial For Beginners | Learn Spark In 7 Hours |Simplilearn

YouTube

Apache Spark Architecture | Spark Cluster Architecture Explained | Spark Training | Edureka

YouTube

Introduction to Databricks [New demo linked in description]

YouTube

Apache Spark Tutorial | Spark Tutorial for Beginners | Spark Big Data | Intellipaat

YouTube
Return to navigation

Product Details

What is Apache Spark?

Apache Spark Technical Details

Operating SystemsUnspecified
Mobile ApplicationNo
Return to navigation

Comparisons

View all alternatives
Return to navigation

Reviews From Top Reviewers

View all reviews
(1-5 of 23)

Use Apache Spark to Speed Up Cluster Computing

Rating: 7 out of 10
Incentivized
January 23, 2018
Verified User
Employee in Engineering
Hospital & Health Care Company, 501-1000 employees
Vetted Review
Verified User
Apache Spark
1 year of experience
Use Cases and Deployment Scope
In our company, we used Spark for a healthcare analytical project, where we need to do large-scale data processing in a Hadoop environment. The project is about building an enterprise data lake where we bring data from multiple products and consolidate. Further, in the downstream, we will develop some business reports.
Pros
  • We used to make our batch processing faster. Spark is faster in batch processing than MapReduce with it in memory computing
  • Spark will run along with other tools in the Hadoop ecosystem including Hive and Pig
  • Spark supports both batch and real-time processing
  • Apache Spark has Machine Learning Algorithms support
Cons
  • Consumes more memory
  • Difficult to address issues around memory utilization
  • Expensive - In-memory processing is expensive when we look for a cost-efficient processing of big data
Likelihood to Recommend
Well suited:
1. Data can be integrated from several sources including click stream, logs, transactional systems
2. Real-time ingestion through Kafka, Kinesis, and other streaming platforms

Apache Spark Should Spark Your Interest

Rating: 9 out of 10
Incentivized
December 13, 2017
Verified User
Engineer in Engineering
Computer Software Company, 51-200 employees
Vetted Review
Verified User
Apache Spark
2 years of experience
Use Cases and Deployment Scope
At my current company, we are using Spark in a variety of ways ranging from batch processing to data analysis to machine learning techniques. It has become our main driver for any distributed processing applications. It has gained quick adoption across the organization for its ease of use, integration into the Hadoop stack, and for its support in a variety of languages.
Pros
  • Ease of use, the Spark API allows for minimal boilerplate and can be written in a variety of languages including Python, Scala, and Java.
  • Performance, for most applications we have found that jobs are more performant running via Spark than other distributed processing technologies like Map-Reduce, Hive, and Pig.
  • Flexibility, the frameworks comes with support for streaming, batch processing, sql queries, machine learning, etc. It can be used in a variety of applications without needing to integrate a lot of other distributed processing technologies.
Cons
  • Resource heavy, jobs, in general, can be very memory intensive and you will want the nodes in your cluster to reflect that.
  • Debugging, it has gotten better with every release but sometimes it can be difficult to debug an error due to ambiguous or misleading exceptions and stack traces.
Likelihood to Recommend
If you are running a distributed environment and are running applications that make use of batch processing, analytics, streaming, machine learning, or graphing then I cannot recommend Spark enough. It is easy to get going, simple to learn (relative to similar technologies), and can be used in a variety of use cases. All while giving you great performance.

My Apache Spark Review

Rating: 9 out of 10
Incentivized
June 07, 2018
KC
Kartik Chavan
Data Analyst
The University of Texas at Arlington (Electrical/Electronic Manufacturing, 1001-5000 employees)
Vetted Review
Verified User
Apache Spark
1 year of experience
View profile
Use Cases and Deployment Scope
My company uses Apache Spark in various ways including machine learning, analytics and batch processing. [We] Grab the data from other sources and put it into a Hadoop environment. [We] Build data lakes. SparkSQL is also used for analysis of data and to develop reports. We have deployed the clusters in Cloudera. Because of Apache Spark, it has become very easy to apply data science in a big data field.
Pros
  • Easy ELT Process
  • Easy clustering on cloud
  • Amazing speed
  • Batch & real time processing
Cons
  • Debugging is difficult as it is new for most people
  • There are fewer learning resources
Likelihood to Recommend
When the data is very big, and you cannot afford a lot of computational timing such as in a real-time environment, it is advisable to use Apache Spark. There are alternatives to Apache Spark, but it is the most common and robust tool to work with. It is great at batch processing.

Apache Spark, the be all End All.

Rating: 9 out of 10
Incentivized
March 27, 2018
AA
Anson Abraham
Data Czar
Envisagenics, Inc. (Marketing and Advertising, 51-200 employees)
Vetted Review
Verified User
Apache Spark
5 years of experience
View profile
Use Cases and Deployment Scope
Spark was/is being used in myriad of ways. With Kafka, using Spark Streams to grab data from kafka queue into our hdfs environment. SparkSQL used for analysis of data for those not familiar with spark. Using Spark for data analysis as well and for main workflow process. Using spark over mapreduce. Using Spark for some machine learning algo's with the data.
Pros
  • Machine Learning.
  • Data Analysis
  • WorkFlow process (faster than MapReduce).
  • SQL connector to multiple data sources
Cons
  • Memory management. Very weak on that.
  • PySpark not as robust as scala with spark.
  • spark master HA is needed. Not as HA as it should be.
  • Locality should not be a necessity, but does help improvement. But would prefer no locality
Likelihood to Recommend
Spark is great as a workflow process and extract transform layer process tool. Is really good for machine learning especially for large datasets that can be processed in split file paralallelization.
Spark streaming is scalable for close to real-time data workflow process.
what it's not good for, is smaller subset of data processing.

Want to save dollars, resources and time processing big data, switch to Apache Spark

Rating: 9 out of 10
Incentivized
March 27, 2019
Verified User
Engineer in Engineering
Management Consulting Company, 51-200 employees
Vetted Review
Verified User
Apache Spark
1 year of experience
Use Cases and Deployment Scope
We sold a data science product to one of the leading US-based e-commerce firms. Suddenly, their data started growing at a very fast rate. The product, at this stage, was based on R programming. With such huge data, the product started taking a lot of time. We then started thinking of an alternative to R, to process multiplying big data such as this client has. We eventually came across Apache Spark. With the permission of the client, we started switching the codes from R to Apache Spark. It took a very long time to learn and code in Spark, but it was worth the effort. The R codes, which were taking days of time to run, came down to a few hours.
Pros
  • Very good tool to process big datasets.
  • Inbuilt fault tolerance.
  • Supports multiple languages.
  • Supports advanced analytics.
  • A large number of libraries available -- GraphX, Spark SQL, Spark Streaming, etc.
Cons
  • Very slow with smaller amounts of data.
  • Expensive, as it stores data in memory.
Likelihood to Recommend
If your data is very huge, I recommend converting the underlying technology into Apache Spark. This will save you a lot of time and effort in the near future due to your growing data. The Apache Spark scalability feature also means it handles all the future data related processing.
Return to navigation