Renaming flavors which include board and re-structure of our overlay and docker files

[mauromorales]

After working a bit with it for the Ubuntu RPi flavors I found a few issues with this setup, and would like to propose a change and would like to hear your opinion

This is of course a draft and can be adapted with your improvements 🙌

Current flavor differentiation

• opensuse-leap -> for the amd64 version
• opensuse-leap-arm-rpi -> for the generic arm image and the specific RPi board image

And the same applies for the other flavors which has support for arm RPi

Current structure of the overlay files

• generic files -> always applied
• flavor+board (e.g rpi64) specific -> only applied if flavor matches (current flavors include the board (e.g. opensuse-leap-arm-rpi)

Current naming of the Dockerfiles

• Dockerfile.opensuse-leap
• Dockerfile.opensuse-leap-arm-rpi

Related issues

• Maintenance of the flavor specific files is harder e.g.: when changing a generic file, we normally don't evaluate if the change is needed also in the specific one. So we rely on the tests or a user reporting an issue to update them and even if things don't break it means the delta between the two becomes bigger over time and it's hard to tell if the difference is necessary or just a result of the lack of maintenance
• The amount of files that need to be maintained will just keep growing with more flavors and boards. In the short term we will introduce Nvidia boards to the support matrix, that means that for a single flavor we will have opensuse-leap, opensuse-leap-arm-rpi and something like opensuse-leap-arm-nvidia.
• The fact that when you're building a generic image, you're still passing as --FLAVOR=opensuse-leap-arm-rpi but the rpi suffix, doesn't make sense and related RPi packages should not be installed (see next point)
• The dockerfile might have conflicting packages and configuration. e.g. for the ubuntu-arm-rpi flavor, there are two different types of kernels that should be installed, one for the generic image and a different one for the RPi. In this case they are conflicting ones so we do some logic to ensure that only the right one is, but if they are not conflicting we just keep both of them bloating the image

Proposal

1. We remove the the board from the flavor name e.g. opensuse-leap-arm-rpi becomes only opensuse-leap-arm. (Maybe even remove the architecture and pass that with another flag?)
2. For overlay files there are two options:
   a. We keep the logic inside the generic overlay file but add logic there to decide what to do depending on the board
   b. We have functionality specific overlay files, and Earthly is in charge of doing the evaluation and deciding which ones to copy depending on the flavor+board
3. We change to have only one Dockerfile per flavor i.e. in the case of opensuse-leap-arm-rpi and opensusse-leap, we remove the former and keep the latter. In the one we keep, we use advanced multi-stage build patterns (see ubuntu-arm-rpi for an example) to only run a certain layer depending on the MODEL flag that was passed. IMO this is a clean way to keep all the logic in one place and ensures that we don't apply changes just to one Dockerfile and forget to add to another making maintenance easier

Progress

• Using a single dockerfile per distro
• overlay files have been moved to packages (this is different than the proposal above but marking as done for this context)
• Renaming of the flavors to only include the distro + version e.g. opensuse-tumbleweed or ubuntu-20-lts

[Itxaka]

IMO, all sounds good to me. Less files, the better, build stuff done on the builder definition earthfile+dockerfiles.

The only thing is, will it get much more complex? We may just want to add comments to stuff to make it clear what is happenig but for me this proposition looks good!

[mauromorales]

yeah good point, I see for example that we are not using comments on Earthly to define targets, maybe we could make use of that a bit more