Apache Spark vs IBM InfoSphere DataStage | TrustRadius

What users are saying about

Score 8.7 out of 10

Based on 127 reviews and ratings

Score 8.2 out of 10

Based on 16 reviews and ratings

Feature Rating Comparison

Apache Spark

IBM InfoSphere DataStage

Connect to traditional data sources

Connecto to Big Data and NoSQL

Apache Spark

IBM InfoSphere DataStage

Simple transformations

Complex transformations

Apache Spark

IBM InfoSphere DataStage

Data model creation

Metadata management

Business rules and workflow

Collaboration

Testing and debugging

feature 1

Apache Spark

IBM InfoSphere DataStage

Integration with data quality tools

Integration with MDM tools

Pros

Rich APIs for data transformation making for very each to transform and prepare data in a distributed environment without worrying about memory issues
Faster in execution times compare to Hadoop and PIG Latin
Easy SQL interface to the same data set for people who are comfortable to explore data in a declarative manner
Interoperability between SQL and Scala / Python style of munging data

Read full review

Nitin Pasumarthy

Software Engineer

LinkedInInternet, 5001-10,000 employees

View all 20 answers on this topic

Very reliable in handling data extraction, data transformation and loading
Flexibility in connecting to different type of databases, relational or non-relational
Great features such as parallel processing, hash handling, etc.
You can also take advantage of its FTP functions, and scheduling features if you need to.

Read full review

Verified User

Manager in Information Technology

Financial Services Company, 51-200 employees

View all 5 answers on this topic

Cons

Memory management. Very weak on that.
PySpark not as robust as scala with spark.
spark master HA is needed. Not as HA as it should be.
Locality should not be a necessity, but does help improvement. But would prefer no locality

Read full review

Data Czar

Envisagenics, Inc.Marketing and Advertising, 51-200 employees

View all 20 answers on this topic

You must understand and know the algorithms, since the wrong use of them generates more time in processing.
Metadata. You need to develop with connectors, and taking all the Metadata from the menu, all the data that you complete manually, you can't track it.

Read full review

Verified User

Team Lead in Customer Service

Logistics and Supply Chain Company, 201-500 employees

View all 5 answers on this topic

Usability

Apache Spark 8.7

Based on 3 answers

Apache integrates with multiple big data frameworks. It does not exert too much load on the disks. Moreover, it is easy to program and use. It reduces the headache of using different applications separately through its high-level APIs. Big data processing has never been as easy as it is with Apache Spark.

Read full review

Partha Protim Pegu

Domain Consultant

InfosysInformation Technology & Services, 10,001+ employees

View all 3 answers on this topic

IBM InfoSphere DataStage 9.0

Based on 2 answers

Our development teams in the company can easily achieve and develop any ETL scripts that are needed to massage and move the data as needed. The company has also maintain this tool for a very long time making it automatic when it comes to ETL needs. We are currently trying to make DataStage our main and probably unique ETL tool

Read full review

Herber Gonzalez

Lead Developer

Office DepotRetail, 10,001+ employees

View all 2 answers on this topic

Performance

No score

No answers yet

No answers on this topic

IBM InfoSphere DataStage 9.0

Based on 1 answer

It could load thousands of records in seconds. But in the Parallel version, you need to understand how to particionate the data. If you use the algorithms erroneously, or the functionalities that it gives for the parsing of data, the performance can fall drastically, even with few records.It is necessary to have people with experience to be able to determine which algorithm to use and understand why.

Read full review

Verified User

Team Lead in Customer Service

Logistics and Supply Chain Company, 201-500 employees

View all 1 answers on this topic

Support Rating

Apache Spark 8.3

Based on 6 answers

1. It integrates very well with scala or python.2. It's very easy to understand SQL interoperability.3. Apache is way faster than the other competitive technologies.4. The support from the Apache community is very huge for Spark.5. Execution times are faster as compared to others.6. There are a large number of forums available for Apache Spark.7. The code availability for Apache Spark is simpler and easy to gain access to.8. Many organizations use Apache Spark, so many solutions are available for existing applications.

Read full review

Technical Manager

Rishabh Software Private LimitedInformation Technology & Services, 501-1000 employees

View all 6 answers on this topic

IBM InfoSphere DataStage 5.6

Based on 4 answers

I believe that IBM generally has one of the worst and most complex assistance systems (physical and online) that exists.

Read full review

Filippo Orlando

Data Analyst | Data Developer - Advanced Analytics

Unieuro S.p.A.Retail, 1001-5000 employees

View all 4 answers on this topic

Alternatives Considered

Spark in comparison to similar technologies ends up being a one stop shop. You can achieve so much with this one framework instead of having to stitch and weave multiple technologies from the Hadoop stack, all while getting incredibility performance, minimal boilerplate, and getting the ability to write your application in the language of your choosing.

Read full review

Verified User

Engineer in Engineering

Computer Software Company, 51-200 employees

View all 20 answers on this topic

DataStage offers better integration capabilities without the need to write code manually. It also has a native ETL engine whereas MSIS requires a SQL Server. It has better integration capabilities with data quality, data profiling and data governance tools. The main drawback of DataStage vs. MSIS is pricing.

Read full review

Gonzalo Angeleri

Regional Product & Solution Architect Manager

GRUPO DATCOInformation Technology and Services, 501-1000 employees

View all 5 answers on this topic

Return on Investment

It has had a very positive impact, as it helps reduce the data processing time and thus helps us achieve our goals much faster.
Being easy to use, it allows us to adapt to the tool much faster than with others, which in turn allows us to access various data sources such as Hadoop, Apache Mesos, Kubernetes, independently or in the cloud. This makes it very useful.
It was very easy for me to use Apache Spark and learn it since I come from a background of Java and SQL, and it shares those basic principles and uses a very similar logic.

Read full review

Consultor Tecnico - Java Developer and Php Developer.

Consultec-TIComputer Software, 51-200 employees

View all 20 answers on this topic

Provides us an excellent ETL application
Made our data handling easy
Provide the business with high quality reports

Read full review

Verified User

Manager in Information Technology

Financial Services Company, 51-200 employees

View all 5 answers on this topic

Pricing Details

General

Free Trial

—

Free/Freemium Version

—

Premium Consulting/Integration Services

—

Entry-level set up fee?

No

General

Free Trial

—

Free/Freemium Version

—

Premium Consulting/Integration Services

—

Entry-level set up fee?

No

Rating Summary

Apache Spark

IBM InfoSphere DataStage

Apache Spark

IBM InfoSphere DataStage

Apache Spark

IBM InfoSphere DataStage

Apache Spark

IBM InfoSphere DataStage

Add comparison