Extensionpoints for Kamaji

[hwaastad]

Hi,
been testing a lot with Kamaji, terraform and suse/rancher stack (elemental/elemental-operator/rancher manager) and a question came to mind.
Today, tcp TCP provides the kubeadm and kubelet-config configmaps in target cluster, but no way really for customizing configuration. For cluster provisioning through other toolchains like ansible, terraform etc this off course can be managed but since TCP already provides kubeadm config it would be nice to consider how this information can be configured. For instance, as for #476, a configmap needs to be provisioned.

Im not sure if building support for different types of solutions into kamaji is a good idea (tech depth and what not) but maybe some kind of dynamic extensionpoint for cluster resources can be added? Im just thinking out loud here.

As for SoC, either be able to customize in some way or let kamaji only handle the controller and other toolchains handle the rest. the latter would probably require option to disable any extra resources created in the target cluster (backward compatibility)

/hw

[prometherion]

Hey @hwaastad, thanks for the heads up and bring up the discussion of this proposal.

I've always tried to stick to the kubeadm bootstrap process since considered it as the de-facto standard for the vanilla Kubernetes flavour. Recently, kubeadm from v1.31 introduced the support for patching generated resources such as the kubelet configuration and the kubeadm ones.

Sticking to kubeadm has been put in place to keep consistent with the general approach of generating clusters: @bsctl can add his thoughts but we didn't want to invent yet another bootstrap provider for Kubernetes, despite the limitations of the tools (just thinking of the kube-proxy and CoreDNS bootstrap phases by kubeadm which are considered just to get a viable cluster up and running).

Considering also the feedback from several adopters, advanced configurations of addons are managed using a CD approach, and wondering if the same could be achieved for other components as you're suggesting: the UNIX philosophy rule of thumb do one thing and do it well is always relevant to me, and wondering if we really need to burden Kamaji of this duty, or rather, having something externally managed in a Platform/Product concept.