Extend precompile to support a DAG #792
Conversation
Signed-off-by: Brian H <[email protected]>
|
/cc @jprendes as well who is doing some larger changes to the engine linking a few issues for context: |
| @@ -62,7 +65,7 @@ pub trait Engine: Clone + Send + Sync + 'static { | |||
| /// The cached, precompiled layers will be reloaded on subsequent runs. | |||
| /// The runtime is expected to return the same number of layers passed in, if the layer cannot be precompiled it should return `None` for that layer. | |||
| /// In some edge cases it is possible that the layers may already be precompiled and None should be returned in this case. | |||
| fn precompile(&self, _layers: &[WasmLayer]) -> Result<Vec<Option<Vec<u8>>>> { | |||
| async fn precompile(&self, _layers: &[WasmLayer]) -> Result<Vec<PrecompiledLayer>> { | |||
There was a problem hiding this comment.
Since this now provides a way to do more than just pre-compiling such as composing I wonder if we would want to rename it
There was a problem hiding this comment.
Yeah that definitely seems reasonable! Happy to call this process_layers or whatever makes sense.
|
a quick look and it is about what I was expecting for changes. It was helpful to see how it was used in a non trivial example with spinkube/containerd-shim-spin#259 |
There was a problem hiding this comment.
I have some concerns around the GC references, and how we link layers with precompiled artifacts, but I'm no expect on any of that, so feel free to correct me.
I would eventually like to split the precompilation from the Engine, which would be a significant breaking change. If you think there's a more drastic change that could help with this work, I'd like to hear it :-)
| @@ -62,7 +65,7 @@ pub trait Engine: Clone + Send + Sync + 'static { | |||
| /// The cached, precompiled layers will be reloaded on subsequent runs. | |||
| /// The runtime is expected to return the same number of layers passed in, if the layer cannot be precompiled it should return `None` for that layer. | |||
| /// In some edge cases it is possible that the layers may already be precompiled and None should be returned in this case. | |||
| fn precompile(&self, _layers: &[WasmLayer]) -> Result<Vec<Option<Vec<u8>>>> { | |||
| async fn precompile(&self, _layers: &[WasmLayer]) -> Result<Vec<PrecompiledLayer>> { | |||
There was a problem hiding this comment.
Return impl Future and remove the dependency on async_trait
| async fn precompile(&self, _layers: &[WasmLayer]) -> Result<Vec<PrecompiledLayer>> { | |
| fn precompile(&self, _layers: &[WasmLayer]) -> impl Future<Output = Result<Vec<PrecompiledLayer>>> + Send { async move { |
| } | ||
|
|
||
| let layers = all_layers.values().cloned().collect::<Vec<_>>(); |
There was a problem hiding this comment.
| let layers = all_layers.values().cloned().collect::<Vec<_>>(); | |
| let layers = all_layers.into_values().collect::<Vec<_>>(); |
|
|
||
| let gc_label = | ||
| format!("containerd.io/gc.ref.content.precompile.{child_digest}"); | ||
| parent_layer.labels.insert(gc_label, child_digest.clone()); |
There was a problem hiding this comment.
IIUC, this means that as long as at least one parent is present, the precompilation won't be GCd.
It seems to be that should be that as soon as one parent is GCd, the precompilation should be GCd as well.
Could this lead to the situation where we have a very popular layer that parents many components (e.g., a virtual FS layer), and prevents all the precompilations from being GCd?
@jsturtevant, you know more about this than me :-)
| self.update_info(original_layer).await?; | ||
| // Update the original layers with a gc label which associates the original digests that | ||
| // were used to process and produce the new layer with the digest of the precompiled content. | ||
| for parent_idx in parents { |
There was a problem hiding this comment.
for the future: As an optimization, I guess all of these could be done in parallel.
| .labels | ||
| .into_iter() | ||
| .filter_map(|(key, child_digest)| { | ||
| if key.starts_with(&format!("{precompile_id}/child")) { |
There was a problem hiding this comment.
IIUC, this label gets added to all parents of a precompiled layer.
If a popular layer is shared by many images (e.g., a layer for a virtual FS) and parents many precompilations, will this result in the precompiled artifact for all those images to be loaded here?
Still WIP while I address some lingering TODOs but wanted to get this up for 👀 before too long.
This PR effectively breaks the assumption that layers input to precompile map 1:1 with layers returned. This is necessary to support things like component dependencies in the spin shim via composition where there is not a clear mapping of original to precompiled layers.
I tried to evolve the precompile API in the most straightforward way without requiring shim developers to have to manage a DAG themselves.
NOTE: have to update the other shims to conform to the new API which will be straightforward but i'd like to get feedback on this approach here first.
cc/ @Mossaka @jsturtevant @kate-goldenring