Amazon CloudWatch vs. Apache Mesos vs. Kubernetes

For out business we find that AWS Cloudwatch is good at providing real-time metrics for monitoring and analysing the performance and usage of our platform by customers. It is possible to create custom metrics from log events, such people adding items to a basket, checking out or abandoning their orders.

Incentivized

There's really no reason to ever use Mesos. We switched over to Kubernetes and it's been a breath of fresh air - better CD support, easy CLI for browsing logs, no mysterious dangling redeploys. If you're looking for a tool to manage a fleet of Docker containers on VMs, Kubernetes beats Mesos by a wide margin.

Incentivized

K8s should be avoided - If your application works well without being converted into microservices-based architecture & fits correctly in a VM, needs less scaling, have a fixed traffic pattern then it is better to keep away from Kubernetes. Otherwise, the operational challenges & technical expertise will add a lot to the OPEX. Also, if you're the one who thinks that containers consume fewer resources as compared to VMs then this is not true. As soon as you convert your application to a microservice-based architecture, a lot of components will add up, shooting your resource consumption even higher than VMs so, please beware. Kubernetes is a good choice - When the application needs quick scaling, is already in microservice-based architecture, has no fixed traffic pattern, most of the employees already have desired skills.

Incentivized

• It provides lot many out of the box dashboard to observe the health and usage of your cloud deployments. Few examples are CPU usage, Disk read/write, Network in/out etc.
• It is possible to stream CloudWatch log data to Amazon Elasticsearch to process them almost real time.
• If you have setup your code pipeline and wants to see the status, CloudWatch really helps. It can trigger lambda function when certain cloudWatch event happens and lambda can store the data to S3 or Athena which Quicksight can represent.

Incentivized

• Mesos may have many frameworks. If you have Mesos installed on your servers, you may use it for many kinds of tasks. Today we're running only web applications but the idea is to install a different framework for big data soon.
• There is a good community growing around it.

Incentivized

• Complex cluster management can be done with simple commands with strong authentication and authorization schemes
• Exhaustive documentation and open community smoothens the learning process
• As a user a few concepts like pod, deployment and service are sufficient to go a long way

Incentivized

• Memory metrics on EC2 are not available on CloudWatch. Depending on workloads if we need visibility on memory metrics we use Solarwinds Orion with the agent installed. For scalable workloads, this involves customization of images being used.
• Visualization out of the box. But this can easily be addressed with other solutions such as Grafana.
• By design, this is only used for AWS workloads so depending on your environment cannot be used as an all in one solution for your monitoring.

Incentivized

• Unreliable deployments that would fail for no good reason. Sometimes our Docker container would be "restarting" forever because Mesos thought it didn't have enough resources to start the container.
• Impossibly slow UI. Built in React under the hood with a lot of bloatware backed in, so loading the Mesos UI on a slow internet connection was painful.
• No real logging solution - it would stream "console.log()" output to the UI, but searching for logs wasn't really possible without downloading a huge file.
• No built-in support for redeploying containers from a CI. We had to create a service whose whole job was to expose an HTTP endpoint that restarted a container, and then made Circle CI ping the endpoint whenever we wanted to redeploy.

Incentivized

• Local development, Kubernetes does tend to be a bit complicated and unnecessary in environments where all development is done locally.
• The need for add-ons, Helm is almost required when running Kubernetes. This brings a whole new tool to manage and learn before a developer can really start to use Kubernetes effectively.
• Finicy configmap schemes. Kubernetes configmaps often have environment breaking hangups. The fail safes surrounding configmaps are sadly lacking.

Incentivized

The Kubernetes is going to be highly likely renewed as the technologies that will be placed on top of it are long term as of planning. There shouldn't be any last minute changes in the adoption and I do not anticipate sudden change of the core underlying technology. It is just that the slow process of technology adoption that makes it hard to switch to something else.

It's excellent at collecting logs. It's easy to set up. The viewing & querying part could be much better, though. The query syntax takes some time to get used to, & the examples are not helpful. Also, while being great, Log Insights requires manual picking of log streams to query across every time.

Incentivized

It is an eminently usable platform. However, its popularity is overshadowed by its complexity. To properly leverage the capabilities and possibilities of Kubernetes as a platform, you need to have excellent understanding of your use case, even better understanding of whether you even need Kubernetes, and if yes - be ready to invest in good engineering support for the platform itself

Incentivized

Support is effective, and we were able to get any problems that we couldn't get solved through community discussion forums solved for us by the AWS support team. For example, we were assisted in one instance where we were not sure about the best metrics to use in order to optimize an auto-scaling group on EC2. The support team was able to look at our metrics and give a useful recommendation on which metrics to use.

Incentivized

No real support channel, the Mesos GitHub issues list was the only one we found and it wasn't particularly helpful.

Incentivized

Grafana is definitely a lot better and flexible in comparison with Amazon CloudWatch for visualisation, as it offers much more options and is versatile. VictoriaMetrics and Prometheus are time-series databases which can do almost everything cloudwatch can do in a better and cheaper way. Integrating Grafana with them will make it more capable Elasticsearch for log retention and querying will surpass cloudwatch log monitoring in both performance and speed

Incentivized

Kubernetes is really great and their community is growing really fast (Google influence). We evaluated it in the beginning and it would fit for our web applications workload. We decided to proceed with Mesos because it has more potential. You may use a different framework for different kinds of tasks on Mesos. There is a Kubernetes framework for Mesos, by the way.

Incentivized

Most of the required features for any orchestration tool or framework, which is provided by Kubernetes. After understanding all modules and features of the K8S, it is the best fit for us as compared with others out there.

Incentivized

• Positive for alarms and alert notifications once configured/customized.
• Has upfront learning curve, and cost can increase as does the alarm activity and monitoring details you may require.
• Cost-effective for any size organization keeping with AWS and utilizing its native tools is a savings in long-term ROI.

Incentivized

• It's optimizing our resources.
• It's improving our process. This argument is not just for Mesos, but we needed a tool like this to start changing and it works like a charm.
• It's open source.

Incentivized

• Because of microservices, Kubernetes makes it easy to find the cost of each application easily.
• Like every new technology, initially, it took more resources to educate ourselves but over a period of time, I believe it's going to be worth it.

Incentivized