Hadoop is an open source software from Apache, supporting distributed processing and data storage. Hadoop is popular for its scalability, reliability, and functionality available across commoditized hardware.
N/A
AWS Lambda
Score 8.8 out of 10
N/A
AWS Lambda is a serverless computing platform that lets users run code without provisioning or managing servers. With Lambda, users can run code for virtually any type of app or backend service—all with zero administration. It takes of requirements to run and scale code with high availability.
Altogether, I want to say that Apache Hadoop is well-suited to a larger and unstructured data flow like an aggregation of web traffic or even advertising. I think Apache Hadoop is great when you literally have petabytes of data that need to be stored and processed on an ongoing basis. Also, I would recommend that the software should be supplemented with a faster and interactive database for a better querying service. Lastly, it's very cost-effective so it is good to give it a shot before coming to any conclusion.
Scenarios where AWS Lambda is well suited: 1. When we need to run a periodic task few times in a day or every hour, we may deploy it on AWS Lambda so it would not increase load on our server which is handling client requests and at the same time we don't have to pay for AWS Lambda when it is not running. So, overall we only pay for few function invocations. 2. When some compute intensive processing is to be done but the number of requests per unit of time fluctuates. For example, we had deployed an AWS Lambda for processing images into different sizes and storing them on AWS S3 once user uploads them. Now, this is something that may happen few times every hour on a particular day or may not happen even once on other days. To handle this kind of tasks AWS Lambda is a better choice as we don't have to pay for the idle time of the server and also we don't have to worry about scaling when the load is high. Scenarios where AWS Lambda is not appropriate to use: 1. When we expect a large request volume continuously on the server. 2. When we don't want latency even in case of concurrent requests.
Lambda provides multiple methods for triggering functions, this includes AWS resources and services and external triggers like APIs and CLI calls.
The compute provided my Lambda is largely hands off for operations teams. Once the function is deployed, the management overhead is minimal since there are no servers to maintain.
Lambda's pricing can be very cost effective given that users are only charged for the time the function runs and associated costs like network or storage if those are used. A function that executes quickly and is not called often can cost next to nothing.
Putting a significant portion of your codebase into AWS Lambda and taking advantage of the high level of integration with other AWS services comes with the risk of vendor lock-in.
While the AWS Lambda environment is "not your problem," it's also not at your disposal to extend or modify, nor does it preserve state between function executions.
AWS Lambda functions are subject to strict time limitations, and will be aborted if they exceed five minutes of execution time. This can be a problem for some longer-running tasks that are otherwise well-suited to serverless delivery.
Hadoop is organization-independent and can be used for various purposes ranging from archiving to reporting and can make use of economic, commodity hardware. There is also a lot of saving in terms of licensing costs - since most of the Hadoop ecosystem is available as open-source and is free
Great! Hadoop has an easy to use interface that mimics most other data warehouses. You can access your data via SQL and have it display in a terminal before exporting it to your business intelligence platform of choice. Of course, for smaller data sets, you can also export it to Microsoft Excel.
I give it a seven is usability because it's AWS. Their UI's are always clunkier than the competition and their documentation is rather cumbersome. There's SO MUCH to dig through and it's a gamble if you actually end up finding the corresponding info if it will actually help. Like I said before, going to google with a specific problem is likely a better route because AWS is quite ubiquitous and chances are you're not the first to encounter the problem. That being said, using SAM (Serverless application model) and it's SAM Local environment makes running local instances of your Lambdas in dev environments painless and quite fun. Using Nodejs + Lambda + SAM Local + VS Code debugger = AWESOME.
We went with a third party for support, i.e., consultant. Had we gone with Azure or Cloudera, we would have obtained support directly from the vendor. my rating is more on the third party we selected and doesn't reflect the overall support available for Hadoop. I think we could have done better in our selection process, however, we were trying to use an already approved vendor within our organization. There is plenty of self-help available for Hadoop online.
I have not needed support for AWS Lambda, since it is already using Python, which has resources all over the internet. AWS blog posts have information about how to install some libraries, which is necessary for some more complex operations, but this is available online and didn't require specific customer support for.
Not used any other product than Hadoop and I don't think our company will switch to any other product, as Hadoop is providing excellent results. Our company is growing rapidly, Hadoop helps to keep up our performance and meet customer expectations. We also use HDFS which provides very high bandwidth to support MapReduce workloads.
Azure Functions is another product that provides lambda functionality, but the documentation for some of Azure's products is quite hard to read. Additionally, AWS Lambda was one of the first cloud computing products on a large cloud service that implemented lambda functions, so they have had the most time to develop the product, increase the quality of service, and extend functionality to more languages. Amazon, by far, has the best service for Lambda that I know.
There are many advantages of Hadoop as first it has made the management and processing of extremely colossal data very easy and has simplified the lives of so many people including me.
Hadoop is quite interesting due to its new and improved features plus innovative functions.
I was able to perform a lot of processing on data delivered from my website and little or no cost. This was a big plus to me.
Programming AWS Lambda is quite easy once you understand the time limits to the functions.
AWS Lambda has really good integration with the AWS S3 storage system. This a very good method of delivering data to be processed and a good place to pick it up after processing.
