Apache Mesos vs. CoreOS rkt / Container Linux (project ended)

There's really no reason to ever use Mesos. We switched over to Kubernetes and it's been a breath of fresh air - better CD support, easy CLI for browsing logs, no mysterious dangling redeploys. If you're looking for a tool to manage a fleet of Docker containers on VMs, Kubernetes beats Mesos by a wide margin.

Incentivized

CoreOS rkt is well suited for any development environment where operating systems and hardware are not homogeneous. CoreOS rkt allows us to write code on one machine with the confidence that it will work on any other. This has been immensely helpful as our developers are often switching to the latest and greatest machines and operating systems. CoreOS rkt is less suited for environments that are not Software as a Service. There is often no need to bring the entire developer environment and associated dependencies when delivering a one time product. In these environments CoreOS rkt just adds unneeded overhead.

Incentivized

• Mesos may have many frameworks. If you have Mesos installed on your servers, you may use it for many kinds of tasks. Today we're running only web applications but the idea is to install a different framework for big data soon.
• There is a good community growing around it.

Incentivized

• Running a secure container without messing up with low-level details
• Very clear and straightforward approach to building a container
• A way to go for new projects thinking of containers
• Comprehensive and well-written documentation compatible with UNIX keep it simple way of thinking

Incentivized

• Unreliable deployments that would fail for no good reason. Sometimes our Docker container would be "restarting" forever because Mesos thought it didn't have enough resources to start the container.
• Impossibly slow UI. Built in React under the hood with a lot of bloatware backed in, so loading the Mesos UI on a slow internet connection was painful.
• No real logging solution - it would stream "console.log()" output to the UI, but searching for logs wasn't really possible without downloading a huge file.
• No built-in support for redeploying containers from a CI. We had to create a service whose whole job was to expose an HTTP endpoint that restarted a container, and then made Circle CI ping the endpoint whenever we wanted to redeploy.

Incentivized

• Market share, it's often very difficult to find new talent who use CoreOS rkt.
• Lack of wow features, CoreOS rkt doesn't necessarily offer any immediate advantages over other container solutions.

Incentivized

No real support channel, the Mesos GitHub issues list was the only one we found and it wasn't particularly helpful.

Incentivized

Kubernetes is really great and their community is growing really fast (Google influence). We evaluated it in the beginning and it would fit for our web applications workload. We decided to proceed with Mesos because it has more potential. You may use a different framework for different kinds of tasks on Mesos. There is a Kubernetes framework for Mesos, by the way.

Incentivized

Docker, lxc, Ubuntu Snappy, partisan chroot+unshare Reformulating the problem and realizing a container is not necessary when a testing environment with clearly defined behavior.

Incentivized

• It's optimizing our resources.
• It's improving our process. This argument is not just for Mesos, but we needed a tool like this to start changing and it works like a charm.
• It's open source.

Incentivized

• Developers spend less time configuring and more time coding.
• Less time training developers as CoreOS rkt lets them use whatever hardware and operating system they want.
• Reduced our IT costs, solutions are containerized using CoreOS rkt meaning they can write one solution with many developers in mind.

Incentivized