Apache Spark vs. MongoDB

Apache Spark has rich APIs for regular data transformations or for ML workloads or for graph workloads, whereas other systems may not such a wide range of support. Choose it when you need to perform data transformations for big data as offline jobs, whereas use MongoDB-like distributed database systems for more realtime queries.

Incentivized

MongoDB [is] great at storing JSON data grouped into "collections". In this format, you can store any JSON documents and conveniently categorize them by collections. The JSON document contained in MongoDB is called binary JSON or BSON and, like any other document in this format, is unstructured. Therefore, unlike traditional DBMS, any kind of data can be stored in collections, and this flexibility is combined with the horizontal scalability of the database. It should be noted that MongoDB does not have links between documents and “collections” (this is partially compensated by the Database Reference - links in the DBMS, but this does not completely solve the problem). As a result, a situation arises in which there is a certain set of data that is not related to other information in the database, and there is no way to combine data from different documents. In SQL systems, this would be an elementary task.

• It performs a conventional disk-based process when the data sets are too large to fit into memory, which is very useful because, regardless of the size of the data, it is always possible to store them.
• It has great speed and ability to join multiple types of databases and run different types of analysis applications. This functionality is super useful as it reduces work times
• Apache Spark uses the data storage model of Hadoop and can be integrated with other big data frameworks such as HBase, MongoDB, and Cassandra. This is very useful because it is compatible with multiple frameworks that the company has, and thus allows us to unify all the processes.

Incentivized

• Easy to learn. When I picked up MongoDB for the first time, I had little background in database management or modeling. If you have a background in javascript (and JSON)... then you can figure out how to use MongoDB pretty fast.
• Fast performance.
• It's relatively easy to set up in certain environments because there are lots of ready-made solutions out there.
• There's a lot of support in the existing ecosystem for it —, especially in the node.js realm.
• Query syntax is pretty simple to grasp and utilize.
• Aggregate functions are powerful.
• Scaling options.
• Documentation is quite good and versioned for each release.

Incentivized

• Memory management. Very weak on that.
• PySpark not as robust as scala with spark.
• spark master HA is needed. Not as HA as it should be.
• Locality should not be a necessity, but does help improvement. But would prefer no locality

Incentivized

• I love the idea of Map-Reduce native support in MongoDB. Admittedly I have not used it as much as I would like -- it always seems to trip me up.
• Recent additions to the aggregation queries have helped reduce (no pun intended) my need to better wield the weapon that is Map-Reduce.

Incentivized

Capacity of computing data in cluster and fast speed.

MongoDB is one of the most famous non-relational databases in the world, there are famous active projects that use this database. I think that the same company that develops the database gives you the online induction totally free is something that really is very positive. Accounts with a first-class support to be able to relate the correct implementation of the database, in addition to teaching you the best practices to optimize your projects, I believe that with this decision it is more than obvious which is the best decision at the time of seeing with which database to work.

Incentivized

If the team looking to use Apache Spark is not used to debug and tweak settings for jobs to ensure maximum optimizations, it can be frustrating. However, the documentation and the support of the community on the internet can help resolve most issues. Moreover, it is highly configurable and it integrates with different tools (eg: it can be used by dbt core), which increase the scenarios where it can be used

Incentivized

It is one of the reasons why we prefer it to store documents in a JSON-style format, to access the desired document very quickly regardless of its size, to be readable by human eyes, and to be easily scalable and manageable.

Incentivized

1. It integrates very well with scala or python. 2. It's very easy to understand SQL interoperability. 3. Apache is way faster than the other competitive technologies. 4. The support from the Apache community is very huge for Spark. 5. Execution times are faster as compared to others. 6. There are a large number of forums available for Apache Spark. 7. The code availability for Apache Spark is simpler and easy to gain access to. 8. Many organizations use Apache Spark, so many solutions are available for existing applications.

I have reached multiple times to the MongoDB community for the help and they have provided each and easy solution for every problem. Over the internet and on stack overflow many people responds over the challenges. Now this tool is very much used in every company and projects so internally many people are there to give a support.

Incentivized

While the setup and configuration of MongoDB is pretty straight forward, having a vendor that performs automatic backups and scales the cluster automatically is very convenient. If you do not have a system administrator or DBA familiar with MongoDB on hand, it's a very good idea to use a 3rd party vendor that specializes in MongoDB hosting. The value is very well worth it over hosting it yourself since the cost is often reasonable among providers.

Incentivized

We used Surprise Kit for one of the other research works. It is more fine-tuned to Recommendation systems and their algorithms. Apache Spark has MLlib for majority of ML problems. Where as software like Surprse Kit - it suitable for a specific task of Recommendations only

Incentivized

The environment I work in is somewhat unique in that we use both MySQL and MongoDB. However, each is used for specific purposes that the other is not well suited for. MongoDB is not a relational database like MySQL, so it serves as the perfect place to dump key bits of data for quick retrieval later. This is something we can't easily do with MySQL. On this smaller database, MongoDB also lets us retrieve data more quickly with its fast and efficient querying.

Incentivized

• Faster turn around on feature development, we have seen a noticeable improvement in our agile development since using Spark.
• Easy adoption, having multiple departments use the same underlying technology even if the use cases are very different allows for more commonality amongst applications which definitely makes the operations team happy.
• Performance, we have been able to make some applications run over 20x faster since switching to Spark. This has saved us time, headaches, and operating costs.

Incentivized

• We can make more open and flexible systems due to its easy adaptation to new evolutions in web applications.
• In the latest versions it offers support for different transactions and we could carry out real tests related to the concurrency of the application.
• MongoDB allows you to have distributed clusters, which improves the speed of the queries by reducing the latency that exists between the database cluster and the service that executes the query.

Incentivized