Skip to main content
TrustRadius
Apache Spark

Apache Spark

Overview

Recent Reviews

TrustRadius Insights

Apache Spark is an incredibly versatile tool that has been widely adopted across various departments for processing very large datasets …
Continue reading
TrustRadius Insights
TrustRadius Insights

Apache Spark in Telco

10 out of 10
July 22, 2021
Incentivized
Apache Spark is being widely used within the company. In Advanced Analytics department data engineers and data scientists work closely in …
Continue reading

Apache Spark Review

7 out of 10
March 16, 2019
Incentivized
We used Apache Spark within our department as a Solution Architecture team. It helped make big data processing more efficient since the …
Continue reading
Read all reviews

Reviewer Pros & Cons

View all pros & cons
Return to navigation

Product Demos

Spark Project | Spark Tutorial | Online Spark Training | Intellipaat

YouTube

Spark SQL Tutorial | Spark SQL Using Scala | Apache Spark Tutorial For Beginners | Simplilearn

YouTube

Apache Spark Full Course | Apache Spark Tutorial For Beginners | Learn Spark In 7 Hours |Simplilearn

YouTube

Apache Spark Architecture | Spark Cluster Architecture Explained | Spark Training | Edureka

YouTube

Introduction to Databricks [New demo linked in description]

YouTube

Apache Spark Tutorial | Spark Tutorial for Beginners | Spark Big Data | Intellipaat

YouTube
Return to navigation

Product Details

What is Apache Spark?

Apache Spark Technical Details

Operating SystemsUnspecified
Mobile ApplicationNo
Return to navigation

Comparisons

View all alternatives
Return to navigation

Reviews and Ratings

(159)

Community Insights

TrustRadius Insights are summaries of user sentiment data from TrustRadius reviews and, when necessary, 3rd-party data sources. Have feedback on this content? Let us know!

Apache Spark is an incredibly versatile tool that has been widely adopted across various departments for processing very large datasets and generating summary statistics. Users have found it particularly useful for creating simple graphics when working with big data, making it a valuable asset for analytics departments. It is also used extensively in the banking industry to calculate risk-weighted assets on a daily and monthly basis for different positions. The integration of Apache Spark with Scala and Apache Spark clusters enables users to load and process large volumes of data, implementing complex formulas and algorithms. Additionally, Apache Spark is often utilized alongside Kafka and Spark Streams to extract data from Kafka queues into HDFS environments, allowing for streamlined data analysis and processing.

One of the key strengths of Apache Spark lies in its ability to handle large volumes of retail and eCommerce data, providing cost and performance benefits over traditional RDBMS solutions. This makes it a preferred choice for companies in these industries. Furthermore, Apache Spark plays a crucial role in supporting data-driven decision-making by digital data teams. Its capabilities allow these teams to build data products, source data from different systems, process and transform it, and store it in data lakes.

Apache Spark is highly regarded for its ability to perform data cleansing and transformation before inserting it into the final target layer in data warehouses. This makes it a vital tool for ensuring the accuracy and reliability of data. Its faster data processing capabilities compared to Hadoop MapReduce have made Apache Spark a go-to choice for tasks such as machine learning, analytics, batch processing, data ingestion, and report development. Moreover, educational institutions rely on Apache Spark to optimize scheduling by assigning classrooms based on student course enrollment and professor schedules.

Overall, Apache Spark proves itself as an indispensable product that meets the needs of various industries by offering efficient distributed data processing, advanced analytics capabilities, and seamless integration with other technologies. Its versatility allows it to support a wide range of use cases, making it an essential tool for anyone working with big data.

Great Computing Engine: Apache Spark is praised by many users for its capabilities in handling complex transformative logic and sophisticated data processing tasks. Several reviewers have mentioned that it is a great computing engine, indicating its effectiveness in solving intricate problems.

Valuable Insights and Analysis: Many reviewers find Apache Spark to be useful for understanding data and performing data analytical work. They appreciate the valuable insights and analysis capabilities provided by the software, suggesting that it helps them gain deeper understanding of their data.

Extensive Set of Libraries and APIs: The extensive set of libraries and APIs offered by Apache Spark has been highly appreciated by users. It provides a wide range of tools and functionalities to solve various day-to-day problems, making it a versatile choice for different data processing needs.

Challenging to Understand and Use: Some users have found Apache Spark to be challenging to understand and use for modeling big data. They struggle with the complexity of the software, leading to a high learning curve.

Lack of User-Friendliness: The software is considered not user-friendly, with a confusing user interface and graphics that are not of high quality. This has resulted in frustration among some users who find it difficult to navigate and work with.

Time-Consuming Processing: Apache Spark can be time-consuming when processing large data sets across multiple nodes. This has been reported by several users who have experienced delays in their data processing tasks, affecting overall efficiency.

When using Spark for big data tasks, users commonly recommend familiarizing yourself with the documentation and gaining experience. They emphasize investing time in reading and understanding the documentation to overcome any initial challenges. As users gain experience, they find working with Spark becomes easier and more efficient.

Users also suggest utilizing Spark specifically for true big data problems, where its capabilities and performance shine. They highlight that Spark is well-suited for tackling large-scale data processing tasks.

Additionally, users find value in leveraging the R and Python APIs in Spark. These APIs allow them to work with Spark using familiar programming languages such as R and Python, making it easier to analyze and process data.

Overall, users advise diving into the documentation, utilizing Spark for big data challenges, and leveraging the R and Python APIs to enhance their experience with Spark.

Attribute Ratings

Reviews

(1-24 of 24)
Companies can't remove reviews or game the system. Here's why
August 18, 2023

Apache Spark: Lightning-Fast Distributed Computing with a Learning Curve

Ananth Gouri | TrustRadius Reviewer
Ananth Gouri
Assistant Professor
The National Institute of Engineering, Mysuru (Education Management, 501-1000 employees)
Score 10 out of 10
Vetted Review
Verified User
Incentivized
Likelihood to Recommend
Well suited: To most of the local run of datasets and non-prod systems - scalability is not a problem at all. Including data from multiple types of data sources is an added advantage. MLlib is a decently nice built-in library that can be used for most of the ML tasks.

Less appropriate: We had to work on a RecSys where the music dataset that we used was around 300+Gb in size. We faced memory-based issues. Few times we also got memory errors. Also the MLlib library does not have support for advanced analytics and deep-learning frameworks support. Understanding the internals of the working of Apache Spark for beginners is highly not possible.
August 30, 2022

Lightning Fast In-Memory Cluster Computing Framework

Riyaz Khan | TrustRadius Reviewer
Riyaz Khan
Staff Engineer
Nagarro (Information Technology & Services, 10,001+ employees)
Score 10 out of 10
Vetted Review
Verified User
Likelihood to Recommend
Well suited for batch processing and provides performance improvement through optimization techniques. Data Streaming is getting better with Apache Spark Structured Streaming. Out of memory issues and Data Skewness problems when data is not properly organized. Integration with BI tools such as Tableau could be better.
July 22, 2021

Apache Spark in Telco

Verified User
Engineer in Information Technology
Telecommunications Company, 10,001+ employees
Score 10 out of 10
Vetted Review
Verified User
Incentivized
Likelihood to Recommend
Apache Spark is very good for prosessing large amount of data but not that good if you need many joins or low latency. With combination of delta engine performance improved alot. Especially having ACID support, time travel features and consistent view for simultaneous read and writes it’s now ready for next level.
July 04, 2021

Spark is useful, but requires lots of very valuable questions to justify the effort, and be prepared for failure in answering posed questions

Thomas Young | TrustRadius Reviewer
Thomas Young
Owner, previous CEO
Econometric Studios (Financial Services, 11-50 employees)
Score 9 out of 10
Vetted Review
Verified User
Incentivized
Likelihood to Recommend
The software appears to run more efficiently than other big data tools, such as Hadoop. Given that, Apache Spark is well-suited for querying and trying to make sense of very, very large data sets. The software offers many advanced machine learning and econometrics tools, although these tools are used only partially because very large data sets require too much time when the data sets get too large. The software is not well-suited for projects that are not big data in size. The graphics and analytical output are subpar compared to other tools.

May 20, 2021

good solution for long and narrow data

Verified User
Analyst in Professional Services
Financial Services Company, 10,001+ employees
Score 9 out of 10
Vetted Review
Verified User
Incentivized
Likelihood to Recommend
I would recommend Apache Spark to the colleague if that person is working with long but narrow dataset. This would be a great tool to help the person fully utilize the CPU cores and speed up the work process. However, I would not recommend this tool if the dataset is wide not not very large.
May 03, 2021

Apache Spark - your go to technology for distributed data processing

Surendranatha Reddy Chappidi | TrustRadius Reviewer
Surendranatha Reddy Chappidi
Senior Data Engineer
A.P. Moller - Maersk (Logistics & Supply Chain, 10,001+ employees)
Score 9 out of 10
Vetted Review
Verified User
Incentivized
Likelihood to Recommend
Specific scenarios where Apache Spark is well suited:
1. real-time processing of streaming data
2. processing unstructured data, semi-structured data, and structured data from multiple sources
3. avoid vendor lock-in and cloud platform lock-in while developing products
September 19, 2020

A powerhouse processing engine.

Verified User
Engineer in Information Technology
Information Technology & Services Company, 11-50 employees
Score 9 out of 10
Vetted Review
Verified User
Incentivized
Likelihood to Recommend
Well suited for: large datasets, fault tolerance, parallel processing, ETL, batch processing, streaming, analytics, graphing, or machine learning. Mostly any kind of large-scale processing, since it will save you a lot of time (days of processing). Less appropriate for: smaller datasets, you are better off using pandas or other libraries.
January 12, 2020

Apache Spark -- The best big data solution

YM
Yogesh Mhasde
Technical Manager
Rishabh Software Private Limited (Information Technology & Services, 501-1000 employees)
Score 8 out of 10
Vetted Review
Verified User
Likelihood to Recommend
1. Suitable where the requirement for advanced analytics is prominent.
2. When you want big data to be processed at a very fast pace.
3. For large datasets, Spark is a viable solution.
4. When you need fault tolerance to be at a precision, go for Spark.

Spark is not suitable:
1. If you want your data to be processed in real-time, then Spark is not a good solution.
2. When you need automatic optimization, then Spark fails at that point.
December 13, 2019

Great open source tool for data processing

Verified User
Executive in Information Technology
Consumer Goods Company, 10,001+ employees
Score 9 out of 10
Vetted Review
Verified User
Incentivized
Likelihood to Recommend
Spark is a one-size-fits-all data processing platform. You can run batch and in-motion streams, you can use for ETL, machine learning or even graphs. You do not have multiple tools, so it makes your TCO and management tasks way easier. As every new platform, has room to grow: storage and security are the main opportunities we found.
March 27, 2019

Want to save dollars, resources and time processing big data, switch to Apache Spark

Verified User
Engineer in Engineering
Management Consulting Company, 51-200 employees
Score 9 out of 10
Vetted Review
Verified User
Incentivized
Likelihood to Recommend
If your data is very huge, I recommend converting the underlying technology into Apache Spark. This will save you a lot of time and effort in the near future due to your growing data. The Apache Spark scalability feature also means it handles all the future data related processing.
March 16, 2019

Apache Spark Review

Verified User
Analyst in Information Technology
Computer Networking Company, 1001-5000 employees
Score 7 out of 10
Vetted Review
Verified User
Incentivized
Likelihood to Recommend
It is beneficial to use Apache Spark if:
  • You are working with big data, preprocessing data before machine learning
  • Building simple microservices and creating PoC. It makes it easier to create REST and simple web APIs.
  • If you need great customer service, Apache Spark would be a great choice since they provide it 24/7.
December 14, 2018

Apache Spark - defacto for big data processing/analytics

SS
Shiv Shivakumar
Acquisitions Leader
Abbott (Computer Software, 1001-5000 employees)
Score 9 out of 10
Vetted Review
Verified User
Incentivized
Likelihood to Recommend
Apache Spark is very well suited for big data analytics in conjunction with the hadoop file system and also does a good job of providing fast access to data in SQL workloads since it has an in memory data processing engine that can very quickly process data. In addition, it can also be used for streaming data processing.
August 28, 2018

Very useful application for Big Data processing and excellent for large volume production workflows

Carla Borges | TrustRadius Reviewer
Carla Borges
Consultor Tecnico - Java Developer and Php Developer.
Consultec-TI (Computer Software, 51-200 employees)
Score 10 out of 10
Vetted Review
Verified User
Incentivized
Likelihood to Recommend
It is suitable for processing large amounts of data, as it is very easy to use and its syntax is simple and understandable. I also find it useful to use in a variety of applications without the need to integrate many other processing technologies, and it is very fast and has many machine learning algorithms that can be used for data problems. I find it less appropriate for data that is not so large, as it uses too many resources.
July 21, 2018

Apache Spark: One stop shop for distributed data processing, machine learning and graph processing

Nitin Pasumarthy | TrustRadius Reviewer
Nitin Pasumarthy
Software Engineer
LinkedIn (Internet, 5001-10,000 employees)
Score 10 out of 10
Vetted Review
Verified User
Incentivized
Likelihood to Recommend
Apache Spark has rich APIs for regular data transformations or for ML workloads or for graph workloads, whereas other systems may not such a wide range of support. Choose it when you need to perform data transformations for big data as offline jobs, whereas use MongoDB-like distributed database systems for more realtime queries.

June 07, 2018

My Apache Spark Review

Kartik Chavan | TrustRadius Reviewer
Kartik Chavan
Data Analyst
The University of Texas at Arlington (Electrical/Electronic Manufacturing, 1001-5000 employees)
Score 9 out of 10
Vetted Review
Verified User
Incentivized
Likelihood to Recommend
When the data is very big, and you cannot afford a lot of computational timing such as in a real-time environment, it is advisable to use Apache Spark. There are alternatives to Apache Spark, but it is the most common and robust tool to work with. It is great at batch processing.
March 27, 2018

Apache Spark, the be all End All.

Anson Abraham | TrustRadius Reviewer
Anson Abraham
Data Czar
Envisagenics, Inc. (Marketing and Advertising, 51-200 employees)
Score 9 out of 10
Vetted Review
Verified User
Incentivized
Likelihood to Recommend
Spark is great as a workflow process and extract transform layer process tool. Is really good for machine learning especially for large datasets that can be processed in split file paralallelization.
Spark streaming is scalable for close to real-time data workflow process.
what it's not good for, is smaller subset of data processing.
December 13, 2017

Apache Spark Should Spark Your Interest

Verified User
Engineer in Engineering
Computer Software Company, 51-200 employees
Score 9 out of 10
Vetted Review
Verified User
Incentivized
Likelihood to Recommend
If you are running a distributed environment and are running applications that make use of batch processing, analytics, streaming, machine learning, or graphing then I cannot recommend Spark enough. It is easy to get going, simple to learn (relative to similar technologies), and can be used in a variety of use cases. All while giving you great performance.
October 26, 2017

Apache Spark - Simple Syntax, Huge Data Handling, Best Optimization, Parallel processing

Kamesh Emani | TrustRadius Reviewer
Kamesh Emani
Software Developer Intern
Louisiana Tech University (Higher Education, 1001-5000 employees)
Score 10 out of 10
Vetted Review
Verified User
Incentivized
Likelihood to Recommend
For large data
For best optimization
For parallel processing
For machine learning on huge data because presently available machine learning software like RapidMiner, are are limited to data size whereas Spark is not
August 02, 2017

Apache Spark if great for high volume production workflows

Verified User
Director in Engineering
Computer Software Company, 10,001+ employees
Score 10 out of 10
Vetted Review
Verified User
Incentivized
Likelihood to Recommend
Well suited for batch and near-real time data processing tasks as well as production deployments of machine learning, especially at large scale. Not well suited for general analytics workflows for small and medium sized data sets; SQL based data warehouses like Redshift, Vertica, and etc. are better for those use cases.
September 12, 2016

A useful replacement for MapReduce for Big Data processing

Jordan Moore | TrustRadius Reviewer
Jordan Moore
Staff Consultant
Avalon Consulting, LLC (Information Technology and Services, 51-200 employees)
Score 8 out of 10
Vetted Review
Verified User
Incentivized
Likelihood to Recommend
On the plus side, Spark is a good tool to learn to apply to various data processing problems.

As described in the Cons - Spark may not be needed unless there is truly a large amount of data to operate on. Other libraries may be better suited for the same task.
Return to navigation