Porch does not render subpackages correctly

[liamfallon]

When Porch runs its pipeline on a package with subpackages, Porch recognizes the subpackages and runs the mutators correctly on some subpackages (The subpackages at the botton of the tree). However, once it steps up out of the bottom level, the mutation pipeline breaks down.

The test package used is in this PR:
nephio-project/catalog#91

Under kpt fn render this package renders correctly:

cp network-function-with-subpackages porchctl -n porch-demo rpkg pull porch-test-481dfb1517c08660a142eee9f5c4988c10362ccb pulled
liam@saor subpackage % p -p
liam@saor subpackage % cp -pr network-function-with-subpackages network-function-with-subpackages-test
liam@saor subpackage % kpt fn render network-function-with-subpackages-test
Package "network-function-with-subpackages-test/sub1/sub1sub1": 
[RUNNING] "gcr.io/kpt-fn/apply-replacements:v0.1.1"
[PASS] "gcr.io/kpt-fn/apply-replacements:v0.1.1" in 200ms
[RUNNING] "gcr.io/kpt-fn/apply-replacements:v0.1.1"
[PASS] "gcr.io/kpt-fn/apply-replacements:v0.1.1" in 100ms
Package "network-function-with-subpackages-test/sub1/sub1sub2": 
[RUNNING] "gcr.io/kpt-fn/apply-replacements:v0.1.1"
[PASS] "gcr.io/kpt-fn/apply-replacements:v0.1.1" in 100ms
[RUNNING] "gcr.io/kpt-fn/apply-replacements:v0.1.1"
[PASS] "gcr.io/kpt-fn/apply-replacements:v0.1.1" in 100ms
Package "network-function-with-subpackages-test/sub1": 
[RUNNING] "gcr.io/kpt-fn/apply-replacements:v0.1.1"
[PASS] "gcr.io/kpt-fn/apply-replacements:v0.1.1" in 200ms
[RUNNING] "gcr.io/kpt-fn/apply-replacements:v0.1.1"
[PASS] "gcr.io/kpt-fn/apply-replacements:v0.1.1" in 200ms
Package "network-function-with-subpackages-test/sub2/sub2sub1": 
[RUNNING] "gcr.io/kpt-fn/apply-replacements:v0.1.1"
[PASS] "gcr.io/kpt-fn/apply-replacements:v0.1.1" in 100ms
[RUNNING] "gcr.io/kpt-fn/apply-replacements:v0.1.1"
[PASS] "gcr.io/kpt-fn/apply-replacements:v0.1.1" in 100ms
Package "network-function-with-subpackages-test/sub2/sub2sub2": 
[RUNNING] "gcr.io/kpt-fn/apply-replacements:v0.1.1"
[PASS] "gcr.io/kpt-fn/apply-replacements:v0.1.1" in 100ms
[RUNNING] "gcr.io/kpt-fn/apply-replacements:v0.1.1"
[PASS] "gcr.io/kpt-fn/apply-replacements:v0.1.1" in 100ms
Package "network-function-with-subpackages-test/sub2": 
[RUNNING] "gcr.io/kpt-fn/apply-replacements:v0.1.1"
[PASS] "gcr.io/kpt-fn/apply-replacements:v0.1.1" in 200ms
[RUNNING] "gcr.io/kpt-fn/apply-replacements:v0.1.1"
[PASS] "gcr.io/kpt-fn/apply-replacements:v0.1.1" in 200ms
Package "network-function-with-subpackages-test": 
[RUNNING] "gcr.io/kpt-fn/apply-replacements:v0.1.1"
[PASS] "gcr.io/kpt-fn/apply-replacements:v0.1.1" in 200ms
[RUNNING] "gcr.io/kpt-fn/apply-replacements:v0.1.1"
[PASS] "gcr.io/kpt-fn/apply-replacements:v0.1.1" in 200ms

Successfully executed 14 function(s) in 7 package(s).

However, when this package is pushed into Porch, the rendering pipeline breaks down:

Log:

 % porchctl -n porch-demo rpkg init network-function-with-subpackages --repository=porch-test --workspace=myws                 
porch-test-481dfb1517c08660a142eee9f5c4988c10362ccb created
% porchctl -n porch-demo rpkg pull porch-test-481dfb1517c08660a142eee9f5c4988c10362ccb network-function-with-subpackages-porch
% cp -pr network-function-with-subpackages/* network-function-with-subpackages-porch
% porchctl -n porch-demo rpkg push porch-test-481dfb1517c08660a142eee9f5c4988c10362ccb network-function-with-subpackages-porch
Error: Internal error occurred: fn.render: pkg /:
	pkg.render: pkg /sub1:
	pipeline.run: error: function failure

Error occurred rendering package in kpt function pipeline.
Package has NOT been pushed to remote.
Please fix package locally (modify until 'kpt fn render' succeeds) and retry. 

Log on Porch server:

I0106 15:07:14.914319       1 render.go:62] Package "sub1/sub1sub1": 
I0106 15:07:14.914912       1 render.go:62] [RUNNING] "gcr.io/kpt-fn/apply-replacements:v0.1.1"
I0106 15:07:14.914927       1 render.go:62] 
I0106 15:07:14.921172       1 render.go:62] [PASS] "gcr.io/kpt-fn/apply-replacements:v0.1.1" in 0s
I0106 15:07:14.921382       1 render.go:62] [RUNNING] "gcr.io/kpt-fn/apply-replacements:v0.1.1"
I0106 15:07:14.921395       1 render.go:62] 
I0106 15:07:14.927038       1 render.go:62] [PASS] "gcr.io/kpt-fn/apply-replacements:v0.1.1" in 0s
I0106 15:07:14.927147       1 render.go:62] 
I0106 15:07:14.927942       1 render.go:62] Package "sub1/sub1sub2": 
I0106 15:07:14.928395       1 render.go:62] [RUNNING] "gcr.io/kpt-fn/apply-replacements:v0.1.1"
I0106 15:07:14.928407       1 render.go:62] 
I0106 15:07:14.936049       1 render.go:62] [PASS] "gcr.io/kpt-fn/apply-replacements:v0.1.1" in 0s
I0106 15:07:14.936270       1 render.go:62] [RUNNING] "gcr.io/kpt-fn/apply-replacements:v0.1.1"
I0106 15:07:14.936277       1 render.go:62] 
I0106 15:07:14.941415       1 render.go:62] [PASS] "gcr.io/kpt-fn/apply-replacements:v0.1.1" in 0s
I0106 15:07:14.941506       1 render.go:62] 
I0106 15:07:14.941833       1 render.go:62] Package "sub1": 
I0106 15:07:14.942632       1 render.go:62] [RUNNING] "gcr.io/kpt-fn/apply-replacements:v0.1.1"
I0106 15:07:14.942651       1 render.go:62] 
I0106 15:07:14.956332       1 render.go:62] [FAIL] "gcr.io/kpt-fn/apply-replacements:v0.1.1" in 0s
I0106 15:07:14.956355       1 render.go:62]   Results:
    [error]: unable to find field "metadata.annotations.sub1-annotation1" in replacement target
E0106 15:07:14.956428       1 generictaskhandler.go:461] fn.render: pkg /:
	pkg.render: pkg /sub1:
	pipeline.run: error: function failure