WIP: add objectfilters

[duncan485]

I ran into the issue that we cannot set resourceNames in (cluster)Role.rules.resourceNames because the cache/watch includes all the object of a kind in the selected namespace. This PR will limit the cache to only the secrets and webhooks that are needed for the cert-controller.

I tested this locally and verified it works, any feedback would be much appreciated.;

[duncan485]

Thanks @JaydipGabani , I'd say this also closes #24

[duncan485]

@JaydipGabani Can this one be merged?

[JaydipGabani]

@maxsmythe @ritazh @sozercan PTAL

[JaydipGabani]

@JaydipGabani Can this one be merged?

waiting for one more review at least.

[maxsmythe]

LGTM

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 62.83%. Comparing base (190188d) to head (5f2f330).
Report is 26 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #228      +/-   ##
==========================================
+ Coverage   57.16%   62.83%   +5.67%     
==========================================
  Files           1        1              
  Lines         572      479      -93     
==========================================
- Hits          327      301      -26     
+ Misses        181      114      -67     
  Partials       64       64              

Flag	Coverage Δ
unittests	62.83% <100.00%> (+5.67%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[sozercan]

Suggested change

	ObjectFilers := make(map[client.Object]cache.ByObject)
	objectFilers := make(map[client.Object]cache.ByObject)

[sozercan]

@duncan485 can we add tests for this?

[duncan485]

While writing tests, I found that this doesn't work with multiple webhooks, it will only add the last added webhook in the filter.

I'm putting this back in WIP, and continue this when I get to it.

[dlipovetsky]

While writing tests, I found that this doesn't work with multiple webhooks, it will only add the last added webhook in the filter.

Are you talking about multiple webhooks of the same type?, The map ObjectFilers [sic] maps from one GVK to a webhook name. For example, if you have two or more validating webhooks, each will have its own name. However, you can only map the validating webhook GVK to one webhook name.

cert-controller/pkg/rotator/rotator.go

Lines 199 to 205 in 7e39bd7

	for _, webhook := range cr.Webhooks {
	wh := &unstructured.Unstructured{}
	wh.SetGroupVersionKind(webhook.gvk())
	ObjectFilers[wh] = cache.ByObject{
	Field: fields.SelectorFromSet(fields.Set{"metadata.name": webhook.Name}),
	}
	}

We can assign exactly one field selector per GVK, and since field selectors do not support set-based operators, we can only target one value.

[duncan485]

Yes exactly, that's while experimenting, what I found.

Taking a step back, my goal is to limit the the cache to only the web-hooks that are needed/configured for the cert-controller, so in the end I can setup the K8S role, limited by resourceNames.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: role
rules:
- apiGroups:
  - admissionregistration.k8s.io
  resources:
  - mutatingwebhookconfigurations
  verbs:
  - get
  - list
  - update
  - watch
  resourceNames:   <---
  - webhooka
  - webhookb

If you have a suggestion to how this could be accomplished, please let me know and I'd be happy to dig in.

[dlipovetsky]

Taking a step back, my goal is to limit the the cache to only the web-hooks that are needed/configured for the cert-controller, so in the end I can setup the K8S role, limited by resourceNames.
If you have a suggestion to how this could be accomplished, please let me know and I'd be happy to dig in.

To support name-based filters, I think controller-runtime would have to implement delegating caches for every resource name. Right now, it implements a delegating cache for every resource (identified by its GroupVersionKind) [1].

One option is to propose per-resource caching to contoller-runtime. But that proposal might not be accepted, since most controllers (I"m assuming here) built with controller-runtime reconcile a group of resources across namespaces, and they don't need the cache granularity you're looking to add to cert-controller.

Another option is to implement a per-resource cache in cert-controller.

And finally, there's a "hack" that would work: reference resource names in the RBAC, apply a label to the resources, and use a label selector in cert-controller. (This would break cert-controller when the label is applied to a resource that isn't referenced in RBAC).

1: https://github.com/kubernetes-sigs/controller-runtime/blob/main/designs/cache_options.md