What line am I replacing in the current CoreOS provisioning template?

[skloeckner]

In the readme, it links to a nonexistant template(likely because a lot of the community templates are now migrated into the main foreman project).

Line 32 is something entirely different than what is referenced in the current provisioning template.

[skloeckner]

I believe I found the line.

There may be some edits to the README if this works...

[skloeckner]

Few notes...Kubernetes latest version 1.6 uses etcd3 and this sets up etcd2. I don't think this will work going forward.

I am going to test a 1.5 version and see if that works.

[johscheuer]

Hi, thanks for reporting the issue. I didn't use this template for a while. Basically these templates are based on the "offical" Kubernetes CoreOS Cloud-Configs:

https://kubernetes.io/docs/getting-started-guides/coreos/cloud-configs/master.yaml
https://kubernetes.io/docs/getting-started-guides/coreos/cloud-configs/node.yaml

If you have a working setup feel free to create a PR containing the changes. The change to etcd3 shouldn'd be a big deal (possibly just change the service name from etcd2 to etcd3 and maybe some minor options updates).

I'll have a look this week but as I said feel free to create a PR with your changes (even if you have a currently not working setup, maybe I can give you some support). Would be nice to see that this still works with 1.6 :) (probably some parameters got renamed)

Thanks!

[skloeckner]

Yeah so I got 1.5.3 working.

I had to change the way I approached the instructions, they do not work with the version of Foreman I am using since the cloudconfig template is now named coreos_cloudconfig so it's simple stuff like that which threw me off.

Thanks for those links. I was thinking the same thing about the etcd3 stuff. I am still just learning about Kubernetes but it's a pain to setup, especially on Coreos, it seems, so this template really helped me out.

I am back to work today but I may test this out in my lab at home. I'll report back hopefully with a PR once I can verify that flannel and all the other bits are working as expected(Especially as I'm learning this stuff as I go).

[skloeckner]

Also wanted to add, this should definitely be upstreamed into the foreman project! It's kind of a no brainer, really, since Kubernetes is known for being difficult. Also, what better way to manage it than on an OS that is designed for Docker usage?

[johscheuer]

I'm looking forward to see the changes and a working setup with 1.6 :)

If you want you can also have a look at ignition this is a more reliable way to provision ContainerLinux than using CloudConfig (just as a hint).

Feel free to push this into Foreman upstream, as I said currently I don't use Foreman.

If you are using CentOS or Ubuntu the easiest way to get started with Kubernetes would be kubeadm -> https://kubernetes.io/docs/getting-started-guides/kubeadm which handles all the setup tasks for Kubernetes (all components will be running as a container). Currently there is no "official" kubeadm for CoreOS kubernetes/kubernetes#34134

[skloeckner]

Yeah, I am aware of kubeadm and wish it would work with CoreOS. I like its simplicity in being a docker consumer and it's simpler to provision on a bare metal machine rather than bringing a 3.6 GB base install of Ubuntu or Centos.

I've heard of ignition. Does that replace cloudconfig completely?

[johscheuer]

It works with CoreOS -> https://github.com/xakraz/kubeadm-installer there are just some "pitfalls". The Ubuntu Server image is only ~500M MB but I get your point :)

AFAIK it's the preferred and more reliable way to provision CoreOS.

[skloeckner]

Thanks again for the links, I think this has been the most helpful issue I've raised in more than just one topic 👍

I do have one last question regarding using that docker image, do I have to create an etcd cluster for the bare metal machines I plan on using first? Or does Kubernetes configure that for me because all the minions end up pointing to the master, anyway?

[johscheuer]

If you use the kubeadm installer kubeadm will bootstrap one etcd node (no HA at the moment) on the kubernetes master. As an alternative you can supply etcd endpoints if you want etcd to run in HA mode -> https://kubernetes.io/docs/admin/kubeadm but this means that you have an running etcd cluster (you can easily setup the cluster via cloud-config/ignition).

Depending on the size/workload of your cluster you probably want the etcd cluster separated from the Kubernetes cluster.

AFAIK the kubelet doesn't talk directly to etcd it rather talks over the API server with the etcd (for many design decision).