WIP: Integrate Kubernetes deployment requirements for ink! playground

[achimcc]

This is a very much WIP PR to move away from running the Dockerfile in a sysbox runtime to a full Kubernetes to add a Kubernetes feature for deployment. As discussed in https://github.com/paritytech/devops/issues/2526.

I'd prefer to keep the working Docker config, as it could be useful self-hosting and testing purposes.

I started to implement a kubernetes feature flag in the backend crate, which allows the main/webserver pod to call into the Kubernetes AI through kube instead of using the sysbox docker runtime.

Supposed to solve:

#704
https://github.com/paritytech/devops/issues/2526

[JoshOrndorff]

Excellent start! Keep it up!

FYI, those paritytech/devops links are 404 (probably a private repo). NBD, just wanted you to know.

[achimcc]

Architecture update:

• We will use Kubernetes jobs instead of pods for Smart Contract compilation.
• We assemble a thin compiler image which contains cargo contract compilation machinery but not any compiled artifacts.
• We instantiate a permanent base compilation volume, which gets populated by the main ink-playground Webserver pod at its first instantiation.
• On every new Job instantiation we create a temporary overlay over the base compilation volume, which stores the result from the requested Smart Contract compilation.

With the Kubernetes implementation it should also be trivial to orchestrate different base images for Smart Contract compilation for job instantiations, so that we can finally also implement the versioning of ink! Smart Contracts which was requested by @cmichi in #427 and continued in #460 .

Thanks for support and discussion @ArshamTeymouri !

We make the Kubernetes namespace in the backend crate configurable with a namespace cli feature. We can pass this cli feature through an internal .env variable of the ink-playground Dockerfile with default value "namespace", so that @paritytech/devops can provide their custom 'namespace' there.

[deuszx]

As I understand, this PR is supposed to solve current issues with https://ink-playground.substrate.io/ ?

Any ETA on it?

[kamalbuilds]

+1 to @deuszx cc @achimcc