TrustRadius Insights
Apache Spark is an incredibly versatile tool that has been widely adopted across various departments for processing very large datasets …
Overview
Recent Reviews
Reviewer Pros & Cons
Assistant Professor
The National Institute of Engineering, Mysuru (Education Management, 501-1000 employees)
Product Demos
Spark Project | Spark Tutorial | Online Spark Training | Intellipaat
YouTube
Spark SQL Tutorial | Spark SQL Using Scala | Apache Spark Tutorial For Beginners | Simplilearn
YouTube
Apache Spark Full Course | Apache Spark Tutorial For Beginners | Learn Spark In 7 Hours |Simplilearn
YouTube
Apache Spark Architecture | Spark Cluster Architecture Explained | Spark Training | Edureka
YouTube
Introduction to Databricks [New demo linked in description]
YouTube
Apache Spark Tutorial | Spark Tutorial for Beginners | Spark Big Data | Intellipaat
YouTube
Product Details
- About
- Tech Details
What is Apache Spark?
Apache Spark Technical Details
| Operating Systems | Unspecified |
|---|---|
| Mobile Application | No |
Comparisons
Compare with
Reviews and Ratings
(159)
Community Insights
TrustRadius Insights are summaries of user sentiment data from TrustRadius reviews and, when necessary, 3rd-party data sources. Have feedback on this content? Let us know!
- Business Problems Solved
- Pros
- Cons
- Recommendations
Apache Spark is an incredibly versatile tool that has been widely adopted across various departments for processing very large datasets and generating summary statistics. Users have found it particularly useful for creating simple graphics when working with big data, making it a valuable asset for analytics departments. It is also used extensively in the banking industry to calculate risk-weighted assets on a daily and monthly basis for different positions. The integration of Apache Spark with Scala and Apache Spark clusters enables users to load and process large volumes of data, implementing complex formulas and algorithms. Additionally, Apache Spark is often utilized alongside Kafka and Spark Streams to extract data from Kafka queues into HDFS environments, allowing for streamlined data analysis and processing.
One of the key strengths of Apache Spark lies in its ability to handle large volumes of retail and eCommerce data, providing cost and performance benefits over traditional RDBMS solutions. This makes it a preferred choice for companies in these industries. Furthermore, Apache Spark plays a crucial role in supporting data-driven decision-making by digital data teams. Its capabilities allow these teams to build data products, source data from different systems, process and transform it, and store it in data lakes.
Apache Spark is highly regarded for its ability to perform data cleansing and transformation before inserting it into the final target layer in data warehouses. This makes it a vital tool for ensuring the accuracy and reliability of data. Its faster data processing capabilities compared to Hadoop MapReduce have made Apache Spark a go-to choice for tasks such as machine learning, analytics, batch processing, data ingestion, and report development. Moreover, educational institutions rely on Apache Spark to optimize scheduling by assigning classrooms based on student course enrollment and professor schedules.
Overall, Apache Spark proves itself as an indispensable product that meets the needs of various industries by offering efficient distributed data processing, advanced analytics capabilities, and seamless integration with other technologies. Its versatility allows it to support a wide range of use cases, making it an essential tool for anyone working with big data.
Great Computing Engine: Apache Spark is praised by many users for its capabilities in handling complex transformative logic and sophisticated data processing tasks. Several reviewers have mentioned that it is a great computing engine, indicating its effectiveness in solving intricate problems.
Valuable Insights and Analysis: Many reviewers find Apache Spark to be useful for understanding data and performing data analytical work. They appreciate the valuable insights and analysis capabilities provided by the software, suggesting that it helps them gain deeper understanding of their data.
Extensive Set of Libraries and APIs: The extensive set of libraries and APIs offered by Apache Spark has been highly appreciated by users. It provides a wide range of tools and functionalities to solve various day-to-day problems, making it a versatile choice for different data processing needs.
Challenging to Understand and Use: Some users have found Apache Spark to be challenging to understand and use for modeling big data. They struggle with the complexity of the software, leading to a high learning curve.
Lack of User-Friendliness: The software is considered not user-friendly, with a confusing user interface and graphics that are not of high quality. This has resulted in frustration among some users who find it difficult to navigate and work with.
Time-Consuming Processing: Apache Spark can be time-consuming when processing large data sets across multiple nodes. This has been reported by several users who have experienced delays in their data processing tasks, affecting overall efficiency.
When using Spark for big data tasks, users commonly recommend familiarizing yourself with the documentation and gaining experience. They emphasize investing time in reading and understanding the documentation to overcome any initial challenges. As users gain experience, they find working with Spark becomes easier and more efficient.
Users also suggest utilizing Spark specifically for true big data problems, where its capabilities and performance shine. They highlight that Spark is well-suited for tackling large-scale data processing tasks.
Additionally, users find value in leveraging the R and Python APIs in Spark. These APIs allow them to work with Spark using familiar programming languages such as R and Python, making it easier to analyze and process data.
Overall, users advise diving into the documentation, utilizing Spark for big data challenges, and leveraging the R and Python APIs to enhance their experience with Spark.
Attribute Ratings
Reviews
(1-24 of 24)Companies can't remove reviews or game the system. Here's why
- We used Apache Spark for one of the research projects. The ROI though cannot be measured here - but the research paper got accepted to a good conference. What else would a project require??!!
August 30, 2022
Lightning Fast In-Memory Cluster Computing Framework
Score 10 out of 10
Vetted Review
Verified User
- No investment as it is open source
- Cheap commodity hardwares can save lot of money
Score 9 out of 10
Vetted Review
Verified User
- With the daily risk reports being calculated via Apache Spark, the bank is able to comply with the FHC rule in the US and other regions and control capitals much better with counterparties.
July 22, 2021
Apache Spark in Telco
- Time to market
- Churn reduction
- Customer satisfaction
- In one sense, Apache Spark has been a positive ROI because it helps us figure out details of the vast amounts of data. Sometimes the software leads to answers to questions that are surprising. Small data software tools probably would have failed in discovering some of the insights Spark makes possible.
- Spark has been a negative ROI in the sense that it takes lots and lots of time to produce simple answers to simple questions, and often the answers are what was expected. Because of the confirmatory rather than insightful nature of the software, it seems like a lot of effort for the results garnered.
- Apache Spark represents a positive ROI on the instances when it gives a well-producing machine learning model, a model that produces predictions that actually get used.
May 20, 2021
good solution for long and narrow data
- reduce time
- need tuning
- hard to debug
- Business leaders are able to take data driven decisions
- Business users are able access to data in near real time now . Before using spark, they had to wait for at least 24 hours for data to be available
- Business is able come up with new product ideas
November 08, 2020
Epic Computation Engine Framework
Score 9 out of 10
Vetted Review
Verified User
- Saves lot of time
- Very powerful
- Automates lots of manual work
- Higher depth of knowledge is required to understand and perform analysis
September 19, 2020
A powerhouse processing engine.
- Saved time and resources for the company because of it's agility
- High performance data processing.
January 12, 2020
Apache Spark -- The best big data solution
Score 8 out of 10
Vetted Review
Verified User
- The ROI was increased by considerable percentage after using Apache Spark.
- Apache Spark provided the agility towards supporting multiple applications.
December 13, 2019
Great open source tool for data processing
- Simplified our landscape.
- Drove great performance for data processing.
- We saved a lot of time and resources, thereby saving a lot of dollars for our company as well as the client.
March 16, 2019
Apache Spark Review
- Positive impact on analyzing big data.
- Fast customer service saved our time.
- Easy to use which means less time spent on training the team.
December 14, 2018
Apache Spark - defacto for big data processing/analytics
- overall positive impact to the business for analysis of big data using hadoop file system
- very well received by data scientists in the business despite its shortcoming on analytical dashboarding
August 28, 2018
Very useful application for Big Data processing and excellent for large volume production workflows
- It has had a very positive impact, as it helps reduce the data processing time and thus helps us achieve our goals much faster.
- Being easy to use, it allows us to adapt to the tool much faster than with others, which in turn allows us to access various data sources such as Hadoop, Apache Mesos, Kubernetes, independently or in the cloud. This makes it very useful.
- It was very easy for me to use Apache Spark and learn it since I come from a background of Java and SQL, and it shares those basic principles and uses a very similar logic.
July 21, 2018
Apache Spark: One stop shop for distributed data processing, machine learning and graph processing
- Switching from PIG Latin to Apache Spark sped up the overall development time and also the resource utilization has gone up.
- Our offline jobs also run faster than traditional map-reduce like systems.
- Integrating with Jupyter like notebook environments, the development experience becomes more pleasant and we can iterate much faster.
June 07, 2018
My Apache Spark Review
- Apache Spark has faster performance compared to MapReduce.
- Combination of Python & Spark is the best. Shorter code, faster and efficient performance.
- Can replace RDBMS
March 27, 2018
Apache Spark, the be all End All.
- Workflow process using spark went from 1 day to 2 hours
- Spark Streaming allowed for quick determiniation of data validity
- spark on yarn was good for manangement. But Spark with Kubernetes was easier to use.
January 23, 2018
Use Apache Spark to Speed Up Cluster Computing
- We were able to make batch job faster by 20 times as compared to MapReduce
- With the language support like Scala, Java, and Python, easily manageable
December 13, 2017
Apache Spark Should Spark Your Interest
- Faster turn around on feature development, we have seen a noticeable improvement in our agile development since using Spark.
- Easy adoption, having multiple departments use the same underlying technology even if the use cases are very different allows for more commonality amongst applications which definitely makes the operations team happy.
- Performance, we have been able to make some applications run over 20x faster since switching to Spark. This has saved us time, headaches, and operating costs.
October 26, 2017
Apache Spark - Simple Syntax, Huge Data Handling, Best Optimization, Parallel processing
- Optimization at its best (Super Fast).
- Handles huge data with simple syntax whereas other programming language takes hell a lot of coding.
- Best for parallel computing applications.
August 02, 2017
Apache Spark if great for high volume production workflows
- Positive: we don't worry about scale.
- Positive: large support community.
- Negative: Takes time to set up, overkill for many simpler workflows.
June 26, 2017
Sparkling Spark
- Time saved in developing applications is less.
- ROI on time, resources, money.
- Can replace the traditional database systems.
September 12, 2016
A useful replacement for MapReduce for Big Data processing
- By learning Spark, we can become certified and/or provide proper recommendations or implementations on Spark solutions.
- With a background in Hadoop distributed processes, it has been easy to understand and diagnose how Spark handles the transfer of data within a cluster. Especially when using YARN as the resource manager and HDFS as the data source.
- Staying up to date with the latest changes to Spark has become a repetitive task. While most Hadoop distributions only support Spark 1.6 at the moment, Spark 2.0 has introduced some useful features, but those require a re-write of existing applications.