Apache Spark vs. Google Compute Engine

Well suited: To most of the local run of datasets and non-prod systems - scalability is not a problem at all. Including data from multiple types of data sources is an added advantage. MLlib is a decently nice built-in library that can be used for most of the ML tasks. Less appropriate: We had to work on a RecSys where the music dataset that we used was around 300+Gb in size. We faced memory-based issues. Few times we also got memory errors. Also the MLlib library does not have support for advanced analytics and deep-learning frameworks support. Understanding the internals of the working of Apache Spark for beginners is highly not possible.

Incentivized

Independently Invited by TrustRadius

A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info

You can use Google Cloud Compute Engine as an option to configure your Gitlab, GitHub, and Azure DevOps self-hosted runners. This allows full control and management of your runners rather than using the default runners, which you cannot manage. Additionally, they can be used as a workspace, which you can provide to the employees, where they can test their workloads or use them as a local host and then deploy to the actual production-grade instance.

Incentivized

Invited by TrustRadius (TR) on Vendor's Behalf

A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info

• Rich APIs for data transformation making for very each to transform and prepare data in a distributed environment without worrying about memory issues
• Faster in execution times compare to Hadoop and PIG Latin
• Easy SQL interface to the same data set for people who are comfortable to explore data in a declarative manner
• Interoperability between SQL and Scala / Python style of munging data

Incentivized

Independently Invited by TrustRadius

A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info

• Scaling - whether it's traffic spikes or just steady growth, Google Compute Engine's auto-scaling makes sure we've got the compute power we need without any manual juggling acts
• Load balancing - Keeping things smooth with that load balancing across multiple VMs, so our users don't have to deal with slow load times or downtime even when things get crazy busy
• Customizability - Mix and match configs for CPU, RAM, storage and whatnot to suit our specific app needs

Incentivized

Invited by TrustRadius (TR) on Vendor's Behalf

A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info

• Memory management. Very weak on that.
• PySpark not as robust as scala with spark.
• spark master HA is needed. Not as HA as it should be.
• Locality should not be a necessity, but does help improvement. But would prefer no locality

Incentivized

Independently Invited by TrustRadius

A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info

• Built-in monitoring via Stackdriver is quite expensive for what it provides.
• Initially provided quotas (ie. max compute units one can use) are very low and it took several requests to get an appropriate amount.
• Support on GCE is limited to their knowledge base and forums. For more hands-on support provided by Google, you must pay for their Premium services.

Incentivized

Independently Invited by TrustRadius

A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info

Capacity of computing data in cluster and fast speed.

Its pretty good, easy and good performance. Also, interface is very good for starters compared to competitors. Infra as Code (IaC) using Terraform even added easiness for creation, management and deletion of compute Virtual Machines (VM). Overall, very good and very easy cloud based compute platform which simplified infrastructure, very much recommend.

Incentivized

Invited by TrustRadius (TR) on Vendor's Behalf

A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. TR verified that a representative sample of customers was invited. More Info

If the team looking to use Apache Spark is not used to debug and tweak settings for jobs to ensure maximum optimizations, it can be frustrating. However, the documentation and the support of the community on the internet can help resolve most issues. Moreover, it is highly configurable and it integrates with different tools (eg: it can be used by dbt core), which increase the scenarios where it can be used

Incentivized

Independently Invited by TrustRadius

A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info

Having interacted with several cloud services, GCE stands out to me as more usable than most. The naming and locating of features is a little more intuitive than most I've interacted with, and hinting is also quite helpful. Getting staff up to speed has proven to be overall less painful than others.

Incentivized

Independently Invited by TrustRadius

A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info

Google Compute Engine works well for cloud project with lesser geographical audience. It sometimes gives error while everything is set up perfectly. We also keep on check any updates available because that's one reason of site getting down. Google Compute Engine is ultimately a top solution to build an app and publish it online within a few minutes

Incentivized

Invited by TrustRadius (TR) on Vendor's Behalf

A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info

It works great all the time except for occasional issues, but overall, I am very happy with the performance. It delivers on the promise it makes and as per the SLAs provided. Networking is great with a premium network, and AZs are also widespread across geographies. Overall, it is a great infra item to have, which you can scale as you want.

Incentivized

Invited by TrustRadius (TR) on Vendor's Behalf

A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info

1. It integrates very well with scala or python. 2. It's very easy to understand SQL interoperability. 3. Apache is way faster than the other competitive technologies. 4. The support from the Apache community is very huge for Spark. 5. Execution times are faster as compared to others. 6. There are a large number of forums available for Apache Spark. 7. The code availability for Apache Spark is simpler and easy to gain access to. 8. Many organizations use Apache Spark, so many solutions are available for existing applications.

• The documentation needs to be better for intermediate users - There are first steps that one can easily follow, but after that, the documentation is often spotty or not in a form where one can follow the steps and accomplish the task. Also, the documentation and the product often go out of sync, where the commands from the documentation do not work with the current version of the product.
• Google support was great and their presence on site was very helpful in dealing with various issues.

Incentivized

Independently Invited by TrustRadius

A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info

Spark in comparison to similar technologies ends up being a one stop shop. You can achieve so much with this one framework instead of having to stitch and weave multiple technologies from the Hadoop stack, all while getting incredibility performance, minimal boilerplate, and getting the ability to write your application in the language of your choosing.

Incentivized

Independently Invited by TrustRadius

A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info

Google Compute Engine provides a one stop solution for all the complex features and the UI is better than Amazon's EC2 and Azure Machine Learning for ease of usability. It's always good to have an eco-system of products from Google as it's one of the most used search engine and IoT services provider, which helps with ease of integration and updates in the future.

Incentivized

Invited by TrustRadius (TR) on Vendor's Behalf

A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info

It works really well with other Google Cloud services, making it easy to build scalable solutions across different teams and locations.

Incentivized

Invited by TrustRadius (TR) on Vendor's Behalf

A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info

• Business leaders are able to take data driven decisions
• Business users are able access to data in near real time now . Before using spark, they had to wait for at least 24 hours for data to be available
• Business is able come up with new product ideas

Incentivized

Independently Invited by TrustRadius

A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info

• With Google Compute we don't have the overhead of managing our own data centers reducing costs and reducing the staff needed to manage systems.
• As I said earlier, Google's costs are ~1/2 of AWS, so we are able to see a ROI much faster.

Incentivized

Independently Invited by TrustRadius

A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info