OCPBUGS-304: exclude rhacs-operator namespace from resource limit rules #12307

Vincent056 · 2024-08-15T16:35:32Z

Added three new variables to be able to exclude namespaces:

var_daemonset_limit_namespaces_exempt_regex for rule resource_requests_limits_in_daemonset var_deployment_limit_namespaces_exempt_regex for rule resource_requests_limits_in_deployment var_statefulset_limit_namespaces_exempt_regex for rule resource_requests_limits_in_statefulset

rhacs-operator namespace has also been excluded by default.

Link to Jira Bug: https://issues.redhat.com/browse/OCPBUGS-304, https://issues.redhat.com/browse/CMP-2400

Additional namespace should be excluded using those variables

github-actions · 2024-08-15T16:37:05Z

Start a new ephemeral environment with changes proposed in this pull request:

Fedora Environment

Oracle Linux 8 Environment

github-actions · 2024-08-15T17:18:01Z

🤖 A k8s content image for this PR is available at:
ghcr.io/complianceascode/k8scontent:12307
This image was built from commit: 5798710

Click here to see how to deploy it

If you alread have Compliance Operator deployed:
utils/build_ds_container.py -i ghcr.io/complianceascode/k8scontent:12307

Otherwise deploy the content and operator together by checking out ComplianceAsCode/compliance-operator and:
CONTENT_IMAGE=ghcr.io/complianceascode/k8scontent:12307 make deploy-local

yuumasato · 2024-08-20T10:33:07Z

/test 4.15-e2e-aws-ocp4-moderate
/test 4.16-e2e-aws-ocp4-moderate

applications/openshift/general/resource_requests_limits_in_deployment/rule.yml

applications/openshift/general/resource_requests_limits_in_statefulset/rule.yml

yuumasato · 2024-08-20T10:42:58Z

Nit, not necessary to merge this PR.

The result of the jquery is quite uncanny. So I was playing with it and got to this:
https://jqplay.org/s/v33KOMdUaoM9X8j

yuumasato · 2024-08-20T12:14:08Z

CI issues related to the malformed condition, I think

     helpers.go:872: Result - Name: e2e-moderate-resource-requests-limits-in-daemonset - Status: PASS - Severity: medium
    helpers.go:1060: Rule e2e-moderate-resource-requests-limits-in-daemonset matched expected result
    helpers.go:872: Result - Name: e2e-moderate-resource-requests-limits-in-deployment - Status: FAIL - Severity: medium
    helpers.go:879: E2E-FAILURE: The expected result for the e2e-moderate-resource-requests-limits-in-deployment rule didn't match. Expected 'PASS', Got 'FAIL'
    helpers.go:872: Result - Name: e2e-moderate-resource-requests-limits-in-statefulset - Status: FAIL - Severity: medium
    helpers.go:879: E2E-FAILURE: The expected result for the e2e-moderate-resource-requests-limits-in-statefulset rule didn't match. Expected 'PASS', Got 'FAIL'

yuumasato · 2024-08-20T12:29:35Z

@Vincent056 Why is https://issues.redhat.com/browse/CMP-2400 linked?
Do you plan to address this rule?
https://github.com/ComplianceAsCode/content/blob/master/applications/openshift/general/resource_requests_quota_per_project/rule.yml

Vincent056 · 2024-08-27T14:19:19Z

@yuumasato thanks for the review!

@Vincent056 Why is https://issues.redhat.com/browse/CMP-2400 linked?
Do you plan to address this rule?
https://github.com/ComplianceAsCode/content/blob/master/applications/openshift/general/resource_requests_quota_per_>project/rule.yml

I have submitted a new pr for CMP 2400 here: #12344

Vincent056 · 2024-08-27T14:19:48Z

https://jqplay.org/s/v33KOMdUaoM9X8j

this is a nice tool!

yuumasato · 2024-08-27T16:04:41Z

/test 4.15-e2e-aws-ocp4-moderate
/test 4.16-e2e-aws-ocp4-moderate

BhargaviGudi · 2024-08-28T05:47:48Z

/hold for test

yuumasato · 2024-08-28T07:33:02Z

/packit retest-failed

rhmdnd · 2024-08-30T21:50:45Z

applications/openshift/general/resource_requests_limits_in_daemonset/rule.yml

-
-{{% set jqfilter = '[ .items[] | select(.metadata.namespace | startswith("kube-") or startswith("openshift-") | not) | select( .spec.template.spec.containers[].resources.requests.cpu == null  or  .spec.template.spec.containers[].resources.requests.memory == null or .spec.template.spec.containers[].resources.limits.cpu == null  or  .spec.template.spec.containers[].resources.limits.memory == null )  | .metadata.name ]' %}}
-
+{{% set jqfilter = '[ .items[] | select(.metadata.namespace | startswith("kube-") or startswith("openshift-") | not)  | select(.metadata.namespace != "rhacs-operator" and ({{if ne .var_daemonset_limit_namespaces_exempt_regex "None"}}.metadata.namespace | test("{{.var_daemonset_limit_namespaces_exempt_regex}}") | not{{else}}true{{end}})) | select( .spec.template.spec.containers[].resources.requests.cpu == null  or  .spec.template.spec.containers[].resources.requests.memory == null or .spec.template.spec.containers[].resources.limits.cpu == null  or  .spec.template.spec.containers[].resources.limits.memory == null )  | .metadata.name ]' %}}


I suppose we could bake the openshift-, kube-, and rhacs-operator strings into the variable as the default, since that's effectively what this is doing, just in a more rigid way.

That is an interesting idea, but that would / could complicate the user experience a little bit.
To keep the "default" behavior of ignoring namespaces begining with openshift- and kube- and exactly matching rhcas-operator they would have to add these same values on their tailorings. Otherwise, the rules would start to check the default list of ignored namespaces.

For example:

Kind: TailoredProfile ... spec: setValue: - name: var_daemonset_limit_namespaces_exempt_regex rationale: Set my org exceptions value: "my_namespace_1|my_namespace_2"

This would end up dropping the default skipped namespaces, instead of adding to the list of skipped namespaces.

Nit: However we could move the default skipped namespaces to a jq variable with the aim of simplifying the jq filter.

xiaojiey · 2024-09-02T12:28:28Z

verification pass with ghcr.io/complianceascode/k8scontent:12307

1. Create 2 namespaces test1 and test2, and create daemonset, deployment, statefulset without resource limits.
2. Create tp and ssb.
% cat tp.yaml 
apiVersion: compliance.openshift.io/v1alpha1
kind: TailoredProfile
metadata:
  name: limits-test
  namespace: openshift-compliance
spec:
  description: Test
  setValues:
  - name: upstream-ocp4-var-daemonset-limit-namespaces-exempt-regex
    value: "test1|test2"
    rationale: test
  - name: upstream-ocp4-var-deployment-limit-namespaces-exempt-regex
    value: "test1|test2"
    rationale: test
  - name: upstream-ocp4-var-statefulset-limit-namespaces-exempt-regex
    value: "test1|test2"
    rationale: test
  extends: upstream-ocp4-high
  title: My modified nist profile with a custom value
% oc apply -f tp.yaml 
tailoredprofile.compliance.openshift.io/limits-test created
% cat ssb_high_u.yaml 
apiVersion: compliance.openshift.io/v1alpha1
kind: ScanSettingBinding
metadata:
  name: cis-compliance
  namespace: openshift-compliance
profiles:
  - name: limits-test
    kind: TailoredProfile
    apiGroup: compliance.openshift.io/v1alpha1
settingsRef:
  name: default
  kind: ScanSetting
  apiGroup: compliance.openshift.io/v1alpha1
% cat ssb_high_u2.yaml 
apiVersion: compliance.openshift.io/v1alpha1
kind: ScanSettingBinding
metadata:
  name: high-profile-compliance
  namespace: openshift-compliance
profiles:
  - name: upstream-ocp4-high
    kind: Profile
    apiGroup: compliance.openshift.io/v1alpha1
settingsRef:
  name: default
  kind: ScanSetting
  apiGroup: compliance.openshift.io/v1alpha1
% oc apply -f ssb_high_u.yaml 
scansettingbinding.compliance.openshift.io/cis-compliance created
% oc apply -f ssb_high_u2.yaml 
scansettingbinding.compliance.openshift.io/high-profile-compliance created
3. Check the result:
% oc get ccr | grep resource-requests-limit
limits-test-resource-requests-limits-in-daemonset                            PASS     medium
limits-test-resource-requests-limits-in-deployment                           PASS     medium
limits-test-resource-requests-limits-in-statefulset                          PASS     medium
upstream-ocp4-high-resource-requests-limits-in-daemonset                     FAIL     medium
upstream-ocp4-high-resource-requests-limits-in-deployment                    FAIL     medium
upstream-ocp4-high-resource-requests-limits-in-statefulset                   FAIL     medium
% oc get rule upstream-ocp4-resource-requests-limits-in-daemonset  -o=jsonpath={.instructions}
Run the following command to retrieve a list of daemonsets that does not have resource requests and limits:
$  oc get daemonset.apps  --all-namespaces -o json  | jq '[ .items[] | select(.metadata.namespace | startswith("kube-") or startswith("openshift-") | not)  | select(.metadata.namespace != "rhacs-operator" and (true)) | select( .spec.template.spec.containers[].resources.requests.cpu == null  or  .spec.template.spec.containers[].resources.requests.memory == null or .spec.template.spec.containers[].resources.limits.cpu == null  or  .spec.template.spec.containers[].resources.limits.memory == null )  | .metadata.name ]'
Make sure that there is output nothing in the result.
Is it the case that Resource requests and limits is not set?%                                                                                                                                                  %  oc get daemonset.apps  --all-namespaces -o json  | jq '[ .items[] | select(.metadata.namespace | startswith("kube-") or startswith("openshift-") | not)  | select(.metadata.namespace != "rhacs-operator" and (true)) | select( .spec.template.spec.containers[].resources.requests.cpu == null  or  .spec.template.spec.containers[].resources.requests.memory == null or .spec.template.spec.containers[].resources.limits.cpu == null  or  .spec.template.spec.containers[].resources.limits.memory == null )  | .metadata.name ]'
[
  "hello-daemonset",
  "hello-daemonset",
  "hello-daemonset",
  "hello-daemonset"
]
% oc get rule upstream-ocp4-resource-requests-limits-in-deployment  -o=jsonpath={.instructions}
Run the following command to retrieve a list of deployments that does not have resource requests and limits:
$  oc get deployment.apps  --all-namespaces -o json  | jq '[ .items[] | select(.metadata.namespace | startswith("kube-") or startswith("openshift-") | not) | select(.metadata.namespace != "rhacs-operator" and (true)) | select( .spec.template.spec.containers[].resources.requests.cpu == null  or  .spec.template.spec.containers[].resources.requests.memory == null or .spec.template.spec.containers[].resources.limits.cpu == null  or  .spec.template.spec.containers[].resources.limits.memory == null )  | .metadata.name ]'
Make sure that there is output nothing in the result.
Is it the case that Resource requests and limits is not set?%                                                                                                                                                  % oc get deployment.apps  --all-namespaces -o json  | jq '[ .items[] | select(.metadata.namespace | startswith("kube-") or startswith("openshift-") | not) | select(.metadata.namespace != "rhacs-operator" and (true)) | select( .spec.template.spec.containers[].resources.requests.cpu == null  or  .spec.template.spec.containers[].resources.requests.memory == null or .spec.template.spec.containers[].resources.limits.cpu == null  or  .spec.template.spec.containers[].resources.limits.memory == null )  | .metadata.name ]'
[
  "nginx-deployment",
  "nginx-deployment"
]

% oc get rule upstream-ocp4-resource-requests-limits-in-statefulset  -o=jsonpath={.instructions}
Run the following command to retrieve a list of statefulsets that does not have resource requests and limits:
$  oc get statefulset.apps --all-namespaces -o json  | jq '[ .items[] | select(.metadata.namespace | startswith("kube-") or startswith("openshift-") | not) | select(.metadata.namespace != "rhacs-operator" and (true)) | select( .spec.template.spec.containers[].resources.requests.cpu == null  or  .spec.template.spec.containers[].resources.requests.memory == null or .spec.template.spec.containers[].resources.limits.cpu == null  or  .spec.template.spec.containers[].resources.limits.memory == null )  | .metadata.name ]'
Make sure that there is output nothing in the result.
Is it the case that Resource requests and limits is not set?%                                                                                                                                                  
% oc get statefulset.apps --all-namespaces -o json  | jq '[ .items[] | select(.metadata.namespace | startswith("kube-") or startswith("openshift-") | not) | select(.metadata.namespace != "rhacs-operator" and (true)) | select( .spec.template.spec.containers[].resources.requests.cpu == null  or  .spec.template.spec.containers[].resources.requests.memory == null or .spec.template.spec.containers[].resources.limits.cpu == null  or  .spec.template.spec.containers[].resources.limits.memory == null )  | .metadata.name ]'
[
  "hello-statefulset",
  "hello-statefulset"
]

xiaojiey · 2024-09-02T12:28:33Z

/unhold

yuumasato · 2024-09-03T23:52:15Z

@Vincent056 you might need to rebase for Gate / Build, Test on Fedora Latest (Container) (push) to pass

Added three new variable to be able to exclude namespaces in: `var_daemonset_limit_namespaces_exempt_regex` for rule `resource_requests_limits_in_daemonset` `var_deployment_limit_namespaces_exempt_regex` for rule `resource_requests_limits_in_deployment` `var_statefulset_limit_namespaces_exempt_regex` for rule `resource_requests_limits_in_statefulset` `rhacs-operator` namespace has also being excluded by default. Link to Jira Bug: https://issues.redhat.com/browse/OCPBUGS-304

codeclimate · 2024-09-04T15:57:31Z

Code Climate has analyzed commit 5798710 and detected 0 issues on this pull request.

The test coverage on the diff in this pull request is 100.0% (50% is the threshold).

This pull request will bring the total coverage in the repository to 59.5% (0.0% change).

View more on Code Climate.

Vincent056 force-pushed the resource_limit branch from f0a49fe to efb9940 Compare August 15, 2024 16:53

yuumasato self-assigned this Aug 19, 2024

jan-cerny added the OpenShift OpenShift product related. label Aug 20, 2024

yuumasato requested changes Aug 20, 2024

View reviewed changes

applications/openshift/general/resource_requests_limits_in_deployment/rule.yml Show resolved Hide resolved

applications/openshift/general/resource_requests_limits_in_statefulset/rule.yml Show resolved Hide resolved

yuumasato requested review from xiaojiey and BhargaviGudi August 27, 2024 16:10

yuumasato added this to the 0.1.75 milestone Aug 27, 2024

openshift-ci bot added the do-not-merge/hold Used by openshift-ci-robot bot. label Aug 28, 2024

rhmdnd reviewed Aug 30, 2024

View reviewed changes

openshift-ci bot removed the do-not-merge/hold Used by openshift-ci-robot bot. label Sep 2, 2024

yuumasato approved these changes Sep 3, 2024

View reviewed changes

Vincent056 force-pushed the resource_limit branch from 2de38b5 to 5798710 Compare September 4, 2024 15:16

yuumasato merged commit a172683 into master Sep 4, 2024
112 checks passed

yuumasato deleted the resource_limit branch September 4, 2024 17:18

yuumasato mentioned this pull request Sep 16, 2024

OCPBUGS-304: Fix jq filters for resource request limits #12382

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OCPBUGS-304: exclude rhacs-operator namespace from resource limit rules #12307

OCPBUGS-304: exclude rhacs-operator namespace from resource limit rules #12307

Vincent056 commented Aug 15, 2024 •

edited

Loading

github-actions bot commented Aug 15, 2024

github-actions bot commented Aug 15, 2024 •

edited

Loading

yuumasato commented Aug 20, 2024

yuumasato commented Aug 20, 2024 •

edited

Loading

yuumasato commented Aug 20, 2024

yuumasato commented Aug 20, 2024

Vincent056 commented Aug 27, 2024

Vincent056 commented Aug 27, 2024

yuumasato commented Aug 27, 2024

BhargaviGudi commented Aug 28, 2024

yuumasato commented Aug 28, 2024

rhmdnd Aug 30, 2024

yuumasato Sep 3, 2024 •

edited

Loading

yuumasato Sep 3, 2024 •

edited

Loading

xiaojiey commented Sep 2, 2024

xiaojiey commented Sep 2, 2024

yuumasato commented Sep 3, 2024

codeclimate bot commented Sep 4, 2024


		{{% set jqfilter = '[ .items[] \| select(.metadata.namespace \| startswith("kube-") or startswith("openshift-") \| not) \| select( .spec.template.spec.containers[].resources.requests.cpu == null or .spec.template.spec.containers[].resources.requests.memory == null or .spec.template.spec.containers[].resources.limits.cpu == null or .spec.template.spec.containers[].resources.limits.memory == null ) \| .metadata.name ]' %}}

		{{% set jqfilter = '[ .items[] \| select(.metadata.namespace \| startswith("kube-") or startswith("openshift-") \| not) \| select(.metadata.namespace != "rhacs-operator" and ({{if ne .var_daemonset_limit_namespaces_exempt_regex "None"}}.metadata.namespace \| test("{{.var_daemonset_limit_namespaces_exempt_regex}}") \| not{{else}}true{{end}})) \| select( .spec.template.spec.containers[].resources.requests.cpu == null or .spec.template.spec.containers[].resources.requests.memory == null or .spec.template.spec.containers[].resources.limits.cpu == null or .spec.template.spec.containers[].resources.limits.memory == null ) \| .metadata.name ]' %}}

OCPBUGS-304: exclude rhacs-operator namespace from resource limit rules #12307

OCPBUGS-304: exclude rhacs-operator namespace from resource limit rules #12307

Conversation

Vincent056 commented Aug 15, 2024 • edited Loading

github-actions bot commented Aug 15, 2024

github-actions bot commented Aug 15, 2024 • edited Loading

yuumasato commented Aug 20, 2024

yuumasato commented Aug 20, 2024 • edited Loading

yuumasato commented Aug 20, 2024

yuumasato commented Aug 20, 2024

Vincent056 commented Aug 27, 2024

Vincent056 commented Aug 27, 2024

yuumasato commented Aug 27, 2024

BhargaviGudi commented Aug 28, 2024

yuumasato commented Aug 28, 2024

rhmdnd Aug 30, 2024

Choose a reason for hiding this comment

yuumasato Sep 3, 2024 • edited Loading

Choose a reason for hiding this comment

yuumasato Sep 3, 2024 • edited Loading

Choose a reason for hiding this comment

xiaojiey commented Sep 2, 2024

xiaojiey commented Sep 2, 2024

yuumasato commented Sep 3, 2024

codeclimate bot commented Sep 4, 2024

Vincent056 commented Aug 15, 2024 •

edited

Loading

github-actions bot commented Aug 15, 2024 •

edited

Loading

yuumasato commented Aug 20, 2024 •

edited

Loading

yuumasato Sep 3, 2024 •

edited

Loading

yuumasato Sep 3, 2024 •

edited

Loading