> Swarm is very easy to manage and maintain, using Puppet or Ansible is not less complicated

The idea that dockerized software somehow is less dependent on configuration management seems to be a popular and completely misguided one. The two trends are completely separate, but I would argue from experience that unless you have absolutely nailed the configuration and integration of all your moving parts, don't even look at containers yet.

Containers tend to lead to more moving parts, not less. And unless you know how to configure them, and perhaps even more importantly how to test them, that will only make matters worse.

If you design your infrastructure and choose your tooling well, then containerized (not "dockerized") software is far less dependent upon configuration management; indeed, using Chef/Puppet/etc can be completely unnecessary for the containerized workload. To be clear, however, there is absolutely still a need for the now-traditional configuration management layer at the level of the hosts running your containerized workloads. What's kind of exciting about this is that the giant spaghetti madness that our configuration management repo has become—and I'm pretty sure it's not just us ;-)— at our org is going to be reduced in complexity and LOC by probably an order of magnitude as we transition to Kubernetes-based infrastructure.

> indeed, using Chef/Puppet/etc can be completely unnecessary for the containerized workload

This is more than naive. As long as your software needs any kind of configuration, there is a need for configuration management. There will be access tokens, certificates, backend configuration, partner integration of various kinds, and monitoring and backup configuration and you will want guarantees that these are consistent for your various testing and staging environments. You will want to track and bisect changes. You can either roll your own framework for this or use Ansible/Puppet.

Whether you distribute your software pieces with tar balls, linux packages or docker images or completely orthogonal to how you tie these pieces to a working whole. And the need for configuration management absolutely increases when moving towards containerized solutions, not by the change in software packaging format but by the organizational changes most go through where more people are empowered to deploy more software which can only increase integration across your environment.

I see organizations that have ignored this because they believe this magic container dust will alleviate the need of keeping a tight grip over what they run, and find themselves with this logic spread over their whole toolchain instead. That's when they need help cleaning up the mess.

I never said anything about magic container dust, nor did I say anything about having less of a grip over our operations. I was attempting to make a point about how your workloads themselves (applications/jobs) can be free of a direct need for Chef/Puppet/etc, which can dramatically simplify your configuration management layer. I never intended to claim that somehow magically our pods need no configuration bits at all, and honestly I’m not sure where you got that idea.

The statement was that containerized workloads are less dependent on configuration management. That could easily be interpreted as if configuration management gets less important when you containerize, which is an idea that seems to spread easily on its own, while I have found the complete opposite to be true. That's why number one guideline is to get a grip on your infrastructure and configuration before you move to containers. Otherwise you will end up with a mess worse than before.

No, it's not 'less dependent'. The way how it helps is that you can standardize configuration management across environments relatively easily.

Containerization does not result in less moving parts in a given environment; but it does result in less moving parts across the whole development flow (from a developer laptop to a production cluster).

Thanks to CoreOS containerLinux, you don’t really need Puppet or Ansible. You specify your Ignition config, and just drop it as a tiny partition on the servers, and drop the OS onto a separate, adjecent partition, and then have local storage even separately. Commonly you use PXE to avoid having these locally.

Then everything you do on top of that is handled by the container scheduler, and your containers.

> Commonly you use PXE to avoid having these locally.

I’ve not seen PXE used anywhere that the DC wasn’t O&O (or essentially close to it). As that’s the exception to the rule these days, isn’t your premise a bit cavalier?

I’ve used PXE a lot in my past [0] to great benefit (well, more specifically iPXE through chainloading), so I’m not detracting from it, just saying it’s applicability is limited for most folks.

[0] I wrote “Genesis” for Tumblr which handled initial machine provisioning from first power-on after racking to being ready for deploys.

Using PXE (or, rather, iPXE) is the recommended deploy mode for container linux, that's why I mentioned it.

> Using PXE (or, rather, iPXE) is the recommended deploy mode for container linux

Really? Is that a new thing? I don’t remember that last time I was reading the docs, but maybe I missed it (that it’s the “preferred” method rather than being just “an option”).

Seems rather odd to limit your audience like that in the age of “cloud everything” as I think it’s generally more rare that folks fully control their layer 2, but what do I know. :)

Well, you can always still use the disk install, and instead mount the ignition config as elastic block storage read-only. That's what most people do in the cloud.

But in either case, you want the config to be centrally stored, so you can modify it without having to ssh onto every machine.

Network booting combined with good out of band management controls allows fully automated provisioning of nodes in a baremetal setup, so I don’t see how this is a negative? The alternative is a live cd/usb or attaching a disk with something already preinstalled.

If you’re on prem with VMware or openstack there’s better options which you can use, but that’s not exactly baremetal. in those environments CoreOS recommends using different provisioning options more suited for those providers.

> people ask why would someone use an orchestrator on a small cluster (dozens of hosts)

I’d love to know who these mythical folks are?

1) dozens of hosts (heck, hosts >1) is exactly why you need orchestration

2) while there are huge deployments across the globe, I wouldn’t consider “dozens of hosts” small by no means. That’s actually probably above average.

3) k8s is actually easier to maintain than you allude. I see these comments about Swarm over k8s generally from folks who never even tried it (or did so years ago), is that the case here?

Disagree. Do not find using K8s any more complicated and believe it being more complicated is a myth that just gets repeated over and over to be believed it is true and not based on actual experience.

It just sounds accurate to say it is like trying to be like Google. Well if it is but less complicated then there is no down side.

The future of Docker, Inc. is being an Oracle acquisition.

I can't believe I'm saying this, but if some giant corporation is going to buy Docker, Inc. then I really, really hope it's Microsoft. An Oracle acquisition would be an absolute nightmare.