Apache Spark vs. Oracle Database

Well suited: To most of the local run of datasets and non-prod systems - scalability is not a problem at all. Including data from multiple types of data sources is an added advantage. MLlib is a decently nice built-in library that can be used for most of the ML tasks. Less appropriate: We had to work on a RecSys where the music dataset that we used was around 300+Gb in size. We faced memory-based issues. Few times we also got memory errors. Also the MLlib library does not have support for advanced analytics and deep-learning frameworks support. Understanding the internals of the working of Apache Spark for beginners is highly not possible.

Incentivized

We migrated from NoSQL to an Oracle database. One of the reasons was robust backup and recovery options available in the Oracle database, which provide zero data loss. A transactional database like Oracle is a better fit for our use case than NoSQL. On a large scale, deployment was evaluated as a cheaper option than the NoSQL engine. This conclusion came even after considering Oracle license is expensive.

Incentivized

• Rich APIs for data transformation making for very each to transform and prepare data in a distributed environment without worrying about memory issues
• Faster in execution times compare to Hadoop and PIG Latin
• Easy SQL interface to the same data set for people who are comfortable to explore data in a declarative manner
• Interoperability between SQL and Scala / Python style of munging data

Incentivized

• Supports most of the Operating Systems like Unix, Linux and Windows Server.
• It works well in high load environment under intense parallel transactions setup.
• Highly reliable DBMS, especially RAC is very much reliable.
• Well managed and predictable release of security patches.
• We have highly scaled it from on-prem to a cloud cluster environment for our product.
• One of the best-performing DBMSs on Linux machines under test delivers high throughput (QPS).

Incentivized

• Memory management. Very weak on that.
• PySpark not as robust as scala with spark.
• spark master HA is needed. Not as HA as it should be.
• Locality should not be a necessity, but does help improvement. But would prefer no locality

Incentivized

• The memory demand and management makes it impossible to run it in a container.
• It is hard to perform local unit testing with Oracle even using the personal edition (aggressive all the available memory grab for itself).
• Lack of built in database migrations (e.g. as Flyway).
• The need to install the Oracle client in addition to its drivers.
• The cost of running it, especially in the Cloud.
• Comes with very spartan community grade client/management tools whereas the commercial offerings tend to demand a premium price.

Incentivized

Capacity of computing data in cluster and fast speed.

There is a lot of sunk cost in a product like Oracle 12c. It is doing a great job, it would not provide us much benefit to switch to another product even if it did the same thing due to the work involved in making such a switch. It would not be cost effective.

Incentivized

If the team looking to use Apache Spark is not used to debug and tweak settings for jobs to ensure maximum optimizations, it can be frustrating. However, the documentation and the support of the community on the internet can help resolve most issues. Moreover, it is highly configurable and it integrates with different tools (eg: it can be used by dbt core), which increase the scenarios where it can be used

Incentivized

Many of the powerful options can be auto-configured but there are still many things to take into account at the moment of installing and configuring an Oracle Database, compared with SQL Server or other databases. At the same time, that extra complexity allows for detailed configuration and guarantees performance, scalability, availability and security.

Incentivized

1. It integrates very well with scala or python. 2. It's very easy to understand SQL interoperability. 3. Apache is way faster than the other competitive technologies. 4. The support from the Apache community is very huge for Spark. 5. Execution times are faster as compared to others. 6. There are a large number of forums available for Apache Spark. 7. The code availability for Apache Spark is simpler and easy to gain access to. 8. Many organizations use Apache Spark, so many solutions are available for existing applications.

1. I have very good experience with Oracle Database support team. Oracle support team has pool of talented Oracle Analyst resources in different regions. To name a few regions - EMEA, Asia, USA(EST, MST, PST), Australia. Their support staffs are very supportive, well trained, and customer focused. Whenever I open Oracle Sev1 SR(service request), I always get prompt update on my case timely. 2. Oracle has zoom call and chat session option linked to Oracle SR. Whenever you are in Oracle portal - you can chat with the Oracle Analyst who is working on your case. You can request for Oracle zoom call thru which you can share the your problem server screen in no time. This is very nice as it saves lot of time and energy in case you have to follow up with oracle support for your case. 3.Oracle has excellent knowledge base in which all the customer databases critical problems and their solutions are well documented. It is very easy to follow without consulting to support team at first.

Incentivized

Overall the implementation went very well and after that everything came out as expected - in terms of performance and scalability. People should always install and upgrade a stable version for production with the latest patch set updates, test properly as much as possible, and should have a backup plan if anything unexpected happens

Spark in comparison to similar technologies ends up being a one stop shop. You can achieve so much with this one framework instead of having to stitch and weave multiple technologies from the Hadoop stack, all while getting incredibility performance, minimal boilerplate, and getting the ability to write your application in the language of your choosing.

Incentivized

Because of a rich user base and support for any critical issue, this is one of the best options to choose. In case the project has a TCO issue, it can compromise and choose Postgres as the best alternative. SQL server is also good and easy to code and maintain but performance is not as good as the Oracle

Incentivized

• Business leaders are able to take data driven decisions
• Business users are able access to data in near real time now . Before using spark, they had to wait for at least 24 hours for data to be available
• Business is able come up with new product ideas

Incentivized

• Multiple applications can use the same database and still get high performance
• Licensing cost is still a concern compared to the other options available in the market that are very very inexpensive
• Almost a maintenance free database
• Oracle Grid makes life easy in terms of monitoring and managing the databases

Incentivized