Amazon EMR is a cloud-native big data platform for processing vast amounts of data quickly, at scale. Using open source tools such as Apache Spark, Apache Hive, Apache HBase, Apache Flink, Apache Hudi (Incubating), and Presto, coupled with the scalability of Amazon EC2 and scalable storage of Amazon S3, EMR gives analytical teams the engines and elasticity to run Petabyte-scale analysis.
N/A
AWS Elastic Beanstalk
Score 8.0 out of 10
N/A
AWS Elastic Beanstalk is the platform-as-a-service offering provided by Amazon and designed to leverage AWS services such as Amazon Elastic Cloud Compute (Amazon EC2), Amazon Simple Storage Service (Amazon S3).
$35
per month
AWS Lambda
Score 8.3 out of 10
N/A
AWS Lambda is a serverless computing platform that lets users run code without provisioning or managing servers. With Lambda, users can run code for virtually any type of app or backend service—all with zero administration. It takes of requirements to run and scale code with high availability.
$NaN
Per 1 ms
Pricing
Amazon EMR (Elastic MapReduce)
AWS Elastic Beanstalk
AWS Lambda
Editions & Modules
No answers on this topic
No Charge
$0
Users pay for AWS resources (e.g. EC2, S3 buckets, etc.) used to store and run the application.
There are many services like AWS Elastic beanstalk, but there are none with the maturity in the platform or the cost-effectiveness of AWS Elastic Beanstalk. Also, AWS Elastic Beanstalk is the oldest among them, so there are more people with AWS experience than the other …
The AWS platform provides a great deal of configurability that is abstracted and provided very well through AWS Elastic Beanstalk. This is the main reason for choosing Elastic Beanstalk over competing services. Another reason for selecting AWS Beanstalk was vendor …
Honestly, I haven't tried any other alternative products. As already mentioned, I am already heavily invested in AWS, so EBS was a natural choice for me. In other reviews, I have found, AWS is better than its competitors. There are more flavors, and options in AWS, better …
AWS Lambda is much easier to use than the near alternatives. It is so straightforward and lightweight it is my primary service for handling small transactions or triggers. The other services require more setup time and are more complex to use. AWS Lambda takes your code snippet …
AWS is great product and a close match our expectations. It is close to Azure in function but more feature rich with API and support documents. From my experience, it is cheaper compared with our competitors and provides better interface. Overall our dev engineers prefer AWS …
When we use Lambda, we do not need to worry about the infrastructure and costs. AWS can handle it all on its own. For an optimum use case, one can always use AWS Lambda along with API Gateway and Route 53 for the best use case. Cloudwatch can help you identify any issues and …
We are running it to perform preparation which takes a few hours on EC2 to be running on a spark-based EMR cluster to total the preparation inside minutes rather than a few hours. Ease of utilization and capacity to select from either Hadoop or spark. Processing time diminishes from 5-8 hours to 25-30 minutes compared with the Ec2 occurrence and more in a few cases.
I have been using AWS Elastic Beanstalk for more than 5 years, and it has made our life so easy and hassle-free. Here are some scenarios where it excels -
I have been using different AWS services like EC2, S3, Cloudfront, Serverless, etc. And Elastic Beanstalk makes our lives easier by tieing each service together and making the deployment a smooth process.
N number of integrations with different CI/CD pipelines make this most engineer's favourite service.
Scalability & Security comes with the service, which makes it the absolute perfect product for your business.
Personally, I haven't found any situations where it's not appropriate for the use cases it can be used. The pricing is also very cost-effective.
Lambda excels at event-driven, short-lived tasks, such as processing files or building simple APIs. However, it's less ideal for long-running, computationally intensive, or applications that rely on carrying the state between jobs. Cold starts and constant load can easily balloon the costs.
EMR does well in managing the cost as it uses the task node cores to process the data and these instances are cheaper when the data is stored on s3. It is really cost efficient. No need to maintain any libraries to connect to AWS resources.
EMR is highly available, secure and easy to launch. No much hassle in launching the cluster (Simple and easy).
EMR manages the big data frameworks which the developer need not worry (no need to maintain the memory and framework settings) about the framework settings. It's all setup on launch time. The bootstrapping feature is great.
Getting a project set up using the console or CLI is easy compared to other [computing] platforms.
AWS Elastic Beanstalk supports a variety of programming languages so teams can experiment with different frameworks but still use the same compute platform for rapid prototyping.
Common application architectures can be referenced as patterns during project [setup].
Multiple environments can be deployed for an application giving more flexibility for experimentation.
It would have been better if packages like HBase and Flume were available with Amazon EMR. This would make the product even more helpful in some cases.
Products like Cloudera provide the options to move the whole deployment into a dedicated server and use it at our discretion. This would have been a good option if available with EMR.
If EMR gave the option to be used with any choice of cloud provider, it would have helped instead of having to move the data from another cloud service to S3.
Limited to the frameworks and configurations that AWS supports. There is no native way to use Elastic Beanstalk to deploy a Go application behind Nginx, for example.
It's not always clear what's changed on an underlying system when AWS updates an EB stack; the new version is announced, but AWS does not say what specifically changed in the underlying configuration. This can have unintended consequences and result in additional work in order to figure out what changes were made.
Developing test cases for Lambda functions can be difficult. For functions that require some sort of input it can be tough to develop the proper payload and event for a test.
For the uninitiated, deploying functions with Infrastructure as Code tools can be a challenging undertaking.
Logging the output of a function feels disjointed from running the function in the console. A tighter integration with operational logging would be appreciated, perhaps being able to view function logs from the Lambda console instead of having to navigate over to CloudWatch.
Sometimes its difficult to determine the correct permissions needed for Lambda execution from other AWS services.
As our technology grows, it makes more sense to individually provision each server rather than have it done via beanstalk. There are several reasons to do so, which I cannot explain without further diving into the architecture itself, but I can tell you this. With automation, you also loose the flexibility to morph the system for your specific needs. So if you expect that in future you need more customization to your deployment process, then there is a good chance that you might try to do things individually rather than use an automation like beanstalk.
Documentation is quite good and the product is regularly updated, so new features regularly come out. The setup is straightforward enough, especially once you have already established the overall platform infrastructure and the aws-cli APIs are easy enough to use. It would be nice to have some out-of-the-box integrations for checking logs and the Spark UI, rather than relying on know-how and digging through multiple levels to find the informations
The overall usability is good enough, as far as the scaling, interactive UI and logging system is concerned, could do a lot better when it comes to the efficiency, in case of complicated node logics and complicated node architectures. It can have better software compatibility and can try to support collaboration with more softwares
I give it a seven is usability because it's AWS. Their UI's are always clunkier than the competition and their documentation is rather cumbersome. There's SO MUCH to dig through and it's a gamble if you actually end up finding the corresponding info if it will actually help. Like I said before, going to google with a specific problem is likely a better route because AWS is quite ubiquitous and chances are you're not the first to encounter the problem. That being said, using SAM (Serverless application model) and it's SAM Local environment makes running local instances of your Lambdas in dev environments painless and quite fun. Using Nodejs + Lambda + SAM Local + VS Code debugger = AWESOME.
I give the overall support for Amazon EMR this rating because while the support technicians are very knowledgeable and always able to help, it sometimes takes a very long time to get in contact with one of the support technicians. So overall the support is pretty good for Amazon EMR.
As I described earlier it has been really cost effective and really easy for fellow developers who don't want to waste weeks and weeks into learning and manually deploying stuff which basically takes month to create and go live with the Minimal viable product (MVP). With AWS Beanstalk within a week a developer can go live with the Minimal viable product easily.
Amazon consistently provides comprehensive and easy-to-parse documentation of all AWS features and services. Most development team members find what they need with a quick internet search of the AWS documentation available online. If you need advanced support, though, you might need to engage an AWS engineer, and that could be an unexpected (or unwelcome) expense.
- Do as many experiments as you can before you commit on using beanstalk or other AWS features. - Keep future state in mind. Think through what comes next, and if that is technically possible to do so. - Always factor in cost in terms of scaling. - We learned a valuable lesson when we wanted to go multi-region, because then we realized many things needs to change in code. So if you plan on using this a lot, factor multiple regions.
Snowflake is a lot easier to get started with than the other options. Snowflake's data lake building capabilities are far more powerful. Although Amazon EMR isn't our first pick, we've had an excellent experience with EC2 and S3. Because of our current API interfaces, it made more sense for us to continue with Hadoop rather than explore other options.
We also use Heroku and it is a great platform for smaller projects and light Node.js services, but we have found that in terms of cost, the Elastic Beanstalk option is more affordable for the projects that we undertake. The fact that it sits inside of the greater AWS Cloud offering also compels us to use it, since integration is simpler. We have also evaluated Microsoft Azure and gave up trying to get an extremely basic implementation up and running after a few days of struggling with its mediocre user interface and constant issues with documentation being outdated. The authentication model is also badly broken and trying to manage resources is a pain. One cannot compare Azure with anything that Amazon has created in the cloud space since Azure really isn't a mature platform and we are always left wanting when we have to interface with it.
AWS Lambda is good for short running functions, and ideally in response to events within AWS. Google App Engine is a more robust environment which can have complex code running for long periods of time, and across more than one instance of hardware. Google App Engine allows for both front-end and back-end infrastructure, while AWS Lambda is only for small back-end functions
It was obviously cheaper and convenient to use as most of our data processing and pipelines are on AWS. It was fast and readily available with a click and that saved a ton of time rather than having to figure out the down time of the cluster if its on premises.
It saved time on processing chunks of big data which had to be processed in short period with minimal costs. EMR solved this as the cluster setup time and processing was simple, easy, cheap and fast.
It had a negative impact as it was very difficult in submitting the test jobs as it lags a UI to submit spark code snippets.
Positive - Only paying for when code is run, unlike virtual machines where you pay always regardless of processing power usage.
Positive - Scalability and accommodating larger amounts of demand is much cheaper. Instead of scaling up virtual machines and increasing the prices you pay for that, you are just increasing the number of times your lambda function is run.
Negative - Debugging/troubleshooting, and developing for lambda functions take a bit more time to get used to, and migrating code from virtual machines and normal processes to Lambda functions can take a bit of time.
