Apache Spark vs. Oracle Exadata

Well suited: To most of the local run of datasets and non-prod systems - scalability is not a problem at all. Including data from multiple types of data sources is an added advantage. MLlib is a decently nice built-in library that can be used for most of the ML tasks. Less appropriate: We had to work on a RecSys where the music dataset that we used was around 300+Gb in size. We faced memory-based issues. Few times we also got memory errors. Also the MLlib library does not have support for advanced analytics and deep-learning frameworks support. Understanding the internals of the working of Apache Spark for beginners is highly not possible.

Incentivized

Oracle Exadata is well-suited for environments where massive performance for Oracle databases is required. Storage indexes reduce the unnecessary I/O. Smart Flash
Cache accelerates random reads/writes.

Our OLTP application demands very high concurrency. Multi-node Exadata provides high availability and zero downtime during DB patching. It comes with lots of built-in automations, so it reduces many routine tasks for sysadmins, like network, storage, and VM configuration, and it also reduces many Oracle DBA tasks, like Oracle software installation, patching, and upgrades.

Incentivized

• Rich APIs for data transformation making for very each to transform and prepare data in a distributed environment without worrying about memory issues
• Faster in execution times compare to Hadoop and PIG Latin
• Easy SQL interface to the same data set for people who are comfortable to explore data in a declarative manner
• Interoperability between SQL and Scala / Python style of munging data

Incentivized

• Oracle Database : Deliver industry-leading security, high availability and scalability with Oracle Database, which has been significantly enhanced to take advantage of the Oracle Exadata Storage Servers.
• Exadata Smart Scan : Improve query performance by offloading intensive query processing and data mining scoring to scalable intelligent storage servers.
• Smart Flash Cache : Transparently cache 'hot' read and write data to fast solid-state storage, improving query response times and throughput. Exadata systems use the latest PCI flash technology rather than flash disks. PCI flash delivers ultra-high performance by placing flash directly on the high speed PCI bus rather than behind slow disk controllers.
• Hybrid Columnar Compression : Reduce the size of data warehousing tables by 10x, and archive tables by 50x, to improve performance and lower storage costs for primary, standby, and backup databases. Query high, query low, archive high and archive low.
• Infiniband Network : Connect multiple Oracle Exadata Database Machines using the InfiniBand fabric to form a larger single system image configuration. Each InfiniBand link provides 40 Gigabits of bandwidth–many times higher than traditional storage or server networks.
• Petabyte Scalability : Easily scale data warehouse to support enterprise data growth.

Incentivized

• Memory management. Very weak on that.
• PySpark not as robust as scala with spark.
• spark master HA is needed. Not as HA as it should be.
• Locality should not be a necessity, but does help improvement. But would prefer no locality

Incentivized

• The process of patching and upgrade of Exadata server components could be improved with a goal to minimize the overall effort, make it fully automated and transparent.
• Improved guidelines and possibly more sophisticated tools for sizing of new Exadata servers for migration from old legacy hardware.

Incentivized

Capacity of computing data in cluster and fast speed.

If the team looking to use Apache Spark is not used to debug and tweak settings for jobs to ensure maximum optimizations, it can be frustrating. However, the documentation and the support of the community on the internet can help resolve most issues. Moreover, it is highly configurable and it integrates with different tools (eg: it can be used by dbt core), which increase the scenarios where it can be used

Incentivized

I am comparing Exadata with the Oracle RAC database experience. In addition to Oracle RAC features, Exadata provides automatic performance optimization through Smart Scan and storage indexes. Deep integration with the Oracle ecosystem and tight coupling with Oracle Enterprise Manager
for monitoring and management. Some downsides of Exadata are: a steep learning curve, concepts like cell offloading, IORM, and flash cache behavior aren’t intuitive initially. Operating
Exadata requires specialized DBA skills.

Incentivized

1. It integrates very well with scala or python. 2. It's very easy to understand SQL interoperability. 3. Apache is way faster than the other competitive technologies. 4. The support from the Apache community is very huge for Spark. 5. Execution times are faster as compared to others. 6. There are a large number of forums available for Apache Spark. 7. The code availability for Apache Spark is simpler and easy to gain access to. 8. Many organizations use Apache Spark, so many solutions are available for existing applications.

Spark in comparison to similar technologies ends up being a one stop shop. You can achieve so much with this one framework instead of having to stitch and weave multiple technologies from the Hadoop stack, all while getting incredibility performance, minimal boilerplate, and getting the ability to write your application in the language of your choosing.

Incentivized

Oracle Exadata Database Machine had the best performance overall hands down. It clearly beat the competition and we were seeing 1000X improvement on SAP HANA. Oracle Exadata Database Machine beat that without us refactoring our code. To achieve that in HANA, we had to refactor the code somewhat. Now this was for our limited POC of 5 use cases. Given the large number of stored procedures we had in Sybase, we need to capture more production metrics but we are seeing incredible performance.

Incentivized

• Business leaders are able to take data driven decisions
• Business users are able access to data in near real time now . Before using spark, they had to wait for at least 24 hours for data to be available
• Business is able come up with new product ideas

Incentivized

• Single support from a single vendor with both machine and database from Oracle, which is costing us less.
• With Exadata, we need less technical manpower and less technical support. A business transaction with the integrated and centralized database helps us focus on other business needs.
• We don't need to buy additional licenses and Hardware for the next 3 to 5 years.

Incentivized