Amazon EMR is a cloud-native big data platform for processing vast amounts of data quickly, at scale. Using open source tools such as Apache Spark, Apache Hive, Apache HBase, Apache Flink, Apache Hudi (Incubating), and Presto, coupled with the scalability of Amazon EC2 and scalable storage of Amazon S3, EMR gives analytical teams the engines and elasticity to run Petabyte-scale analysis.
N/A
AWS Lambda
Score 8.3 out of 10
N/A
AWS Lambda is a serverless computing platform that lets users run code without provisioning or managing servers. With Lambda, users can run code for virtually any type of app or backend service—all with zero administration. It takes of requirements to run and scale code with high availability.
$NaN
Per 1 ms
Google App Engine
Score 8.2 out of 10
N/A
Google App Engine is Google Cloud's platform-as-a-service offering. It features pay-per-use pricing and support for a broad array of programming languages.
$0.05
Per Hour Per Instance
Pricing
Amazon EMR (Elastic MapReduce)
AWS Lambda
Google App Engine
Editions & Modules
No answers on this topic
128 MB
$0.0000000021
Per 1 ms
1024 MB
$0.0000000167
Per 1 ms
10240 MB
$0.0000001667
Per 1 ms
Starting Price
$0.05
Per Hour Per Instance
Max Price
$0.30
Per Hour Per Instance
Offerings
Pricing Offerings
Amazon EMR
AWS Lambda
Google App Engine
Free Trial
No
No
No
Free/Freemium Version
No
No
Yes
Premium Consulting/Integration Services
No
No
No
Entry-level Setup Fee
No setup fee
No setup fee
No setup fee
Additional Details
—
—
—
More Pricing Information
Community Pulse
Amazon EMR (Elastic MapReduce)
AWS Lambda
Google App Engine
Considered Multiple Products
Amazon EMR
No answer on this topic
AWS Lambda
Verified User
Engineer
Chose AWS Lambda
AWS Lambda is good for short running functions, and ideally in response to events within AWS. Google App Engine is a more robust environment which can have complex code running for long periods of time, and across more than one instance of hardware. Google App Engine allows for …
When we use Lambda, we do not need to worry about the infrastructure and costs. AWS can handle it all on its own. For an optimum use case, one can always use AWS Lambda along with API Gateway and Route 53 for the best use case. Cloudwatch can help you identify any issues and …
For our organization, we selected Google App Engine which provides a reliable and efficient way to create and deploy apps moreover it supports a lot of languages and provides automatic debugging of code which enables us to deploy code to production as soon as development is …
If you have a small team which is also responsible for development of the product then surely go for it. And if you have a larger team with dedicated person to take care of deployments. Go for cheaper options such as compute engine or AWS (be sure to do your research on pricing …
You can create and scale Kubernetes clusters quickly, but you have to keep an eye on that cluster. In-App Engine, you don't have to worry about infrastructure, but in some scenarios, Kubernetes fits better.
Azure App Service is in par with Google App Engine although you may want to use Azure App Service if you are integrating with other Microsoft IT components, for example SQL Server. Google App Engine is great when in long run, you will be using Google cloud components, for …
The two giants are Google and Amazon. Both are very similar however Google App Engine allows you to deploy your web applications through platforms like Python where as if you're using AWS, you have full control on the operating system services. Google is good because you pay as …
I think that Microsoft and Amazon are simply investing more in their offerings, and there are a bunch of cool PaaS solutions out there as well. Google App Engine is solid, and is probably the right choice for some projects. But ultimately one should evaluate each platform …
We are running it to perform preparation which takes a few hours on EC2 to be running on a spark-based EMR cluster to total the preparation inside minutes rather than a few hours. Ease of utilization and capacity to select from either Hadoop or spark. Processing time diminishes from 5-8 hours to 25-30 minutes compared with the Ec2 occurrence and more in a few cases.
Lambda excels at event-driven, short-lived tasks, such as processing files or building simple APIs. However, it's less ideal for long-running, computationally intensive, or applications that rely on carrying the state between jobs. Cold starts and constant load can easily balloon the costs.
App Engine is such a good resource for our team both internally and externally. You have complete control over your app, how it runs, when it runs, and more while Google handles the back-end, scaling, orchestration, and so on. If you are serving a tool, system, or web page, it's perfect. If you are serving something back-end, like an automation or ETL workflow, you should be a little considerate or careful with how you are structuring that job. For instance, the Standard environment in Google App Engine will present you with a resource limit for your server calls. If your operations are known to take longer than, say, 10 minutes or so, you may be better off moving to the Flexible environment (which may be a little more expensive but certainly a little more powerful and a little less limited) or even moving that workflow to something like Google Compute Engine or another managed service.
EMR does well in managing the cost as it uses the task node cores to process the data and these instances are cheaper when the data is stored on s3. It is really cost efficient. No need to maintain any libraries to connect to AWS resources.
EMR is highly available, secure and easy to launch. No much hassle in launching the cluster (Simple and easy).
EMR manages the big data frameworks which the developer need not worry (no need to maintain the memory and framework settings) about the framework settings. It's all setup on launch time. The bootstrapping feature is great.
It would have been better if packages like HBase and Flume were available with Amazon EMR. This would make the product even more helpful in some cases.
Products like Cloudera provide the options to move the whole deployment into a dedicated server and use it at our discretion. This would have been a good option if available with EMR.
If EMR gave the option to be used with any choice of cloud provider, it would have helped instead of having to move the data from another cloud service to S3.
Developing test cases for Lambda functions can be difficult. For functions that require some sort of input it can be tough to develop the proper payload and event for a test.
For the uninitiated, deploying functions with Infrastructure as Code tools can be a challenging undertaking.
Logging the output of a function feels disjointed from running the function in the console. A tighter integration with operational logging would be appreciated, perhaps being able to view function logs from the Lambda console instead of having to navigate over to CloudWatch.
Sometimes its difficult to determine the correct permissions needed for Lambda execution from other AWS services.
There is a slight learning curve to getting used to code on Google App Engine.
Google Cloud Datastore is Google's NoSQL database in the cloud that your applications can use. NoSQL databases, by design, cannot give handle complex queries on the data. This means that sometimes you need to think carefully about your data structures - so that you can get the results you need in your code.
Setting up billing is a little annoying. It does not seem to save billing information to your account so you can re-use the same information across different Cloud projects. Each project requires you to re-enter all your billing information (if required)
App Engine is a solid choice for deployments to Google Cloud Platform that do not want to move entirely to a Kubernetes-based container architecture using a different Google product. For rapid prototyping of new applications and fairly straightforward web application deployments, we'll continue to leverage the capabilities that App Engine affords us.
Documentation is quite good and the product is regularly updated, so new features regularly come out. The setup is straightforward enough, especially once you have already established the overall platform infrastructure and the aws-cli APIs are easy enough to use. It would be nice to have some out-of-the-box integrations for checking logs and the Spark UI, rather than relying on know-how and digging through multiple levels to find the informations
I give it a seven is usability because it's AWS. Their UI's are always clunkier than the competition and their documentation is rather cumbersome. There's SO MUCH to dig through and it's a gamble if you actually end up finding the corresponding info if it will actually help. Like I said before, going to google with a specific problem is likely a better route because AWS is quite ubiquitous and chances are you're not the first to encounter the problem. That being said, using SAM (Serverless application model) and it's SAM Local environment makes running local instances of your Lambdas in dev environments painless and quite fun. Using Nodejs + Lambda + SAM Local + VS Code debugger = AWESOME.
I had to revisit the UI after a year of just setting up and forgetting. The UI got some improvements but the amount of navigation we have to go through to setup a new app has increased but also got easier to setup. Gemini now is integrated and make getting answers faster
I give the overall support for Amazon EMR this rating because while the support technicians are very knowledgeable and always able to help, it sometimes takes a very long time to get in contact with one of the support technicians. So overall the support is pretty good for Amazon EMR.
Amazon consistently provides comprehensive and easy-to-parse documentation of all AWS features and services. Most development team members find what they need with a quick internet search of the AWS documentation available online. If you need advanced support, though, you might need to engage an AWS engineer, and that could be an unexpected (or unwelcome) expense.
Good amount of documentation available for Google App Engine and in general there is large developer community around Google App Engine and other products it interacts with. Lastly, Google support is great in general. No issues so far with them.
Snowflake is a lot easier to get started with than the other options. Snowflake's data lake building capabilities are far more powerful. Although Amazon EMR isn't our first pick, we've had an excellent experience with EC2 and S3. Because of our current API interfaces, it made more sense for us to continue with Hadoop rather than explore other options.
AWS Lambda is good for short running functions, and ideally in response to events within AWS. Google App Engine is a more robust environment which can have complex code running for long periods of time, and across more than one instance of hardware. Google App Engine allows for both front-end and back-end infrastructure, while AWS Lambda is only for small back-end functions
We were on another much smaller cloud provider and decided to make the switch for several reasons - stability, breadth of services, and security. In reviewing options, GCP provided the best mixtures of meeting our needs while also balancing the overall cost of the service as compared to the other major players in Azure and AWS.
It was obviously cheaper and convenient to use as most of our data processing and pipelines are on AWS. It was fast and readily available with a click and that saved a ton of time rather than having to figure out the down time of the cluster if its on premises.
It saved time on processing chunks of big data which had to be processed in short period with minimal costs. EMR solved this as the cluster setup time and processing was simple, easy, cheap and fast.
It had a negative impact as it was very difficult in submitting the test jobs as it lags a UI to submit spark code snippets.
Positive - Only paying for when code is run, unlike virtual machines where you pay always regardless of processing power usage.
Positive - Scalability and accommodating larger amounts of demand is much cheaper. Instead of scaling up virtual machines and increasing the prices you pay for that, you are just increasing the number of times your lambda function is run.
Negative - Debugging/troubleshooting, and developing for lambda functions take a bit more time to get used to, and migrating code from virtual machines and normal processes to Lambda functions can take a bit of time.
Effective integration to other java based frameworks.
Time to market is very quick. Build, test, deploy and use.
The GAE Whitelist for java is an important resource to know what works and what does not. So use it. It would also be nice for Google to expand on items that are allowed on GAE platform.
