🐛 [bug] - Image security / Jenkins + Tekton fails, invalid output format pretty

[alexpdp7]

📝 Description

🚶 Steps to reproduce

https://rht-labs.com/tech-exercise/#/3-revenge-of-the-automated-testing/7a-jenkins

, in the first step 3, the pipeline fails on set +x curl -k -L -H "Authorization: Bearer ${ROX_CREDS_PSW}" https://${ROX_CREDS_USR}/api/cli/download/roxctl-linux --output roxctl > /dev/null; chmod +x roxctl > /dev/null export ROX_API_TOKEN=${ROX_CREDS_PSW} ./roxctl image scan --insecure-skip-tls-verify -e ${ROX_CREDS_USR}:443 --image image-registry.openshift-image-registry.svc:5000/${DESTINATION_NAMESPACE}/${APP_NAME}:${VERSION} --format pretty:

Flag --format has been deprecated, please use --output/-o to specify the output format. NOTE: The new JSON / CSV format contains breaking changes, make sure you adapt to the new structure before migrating.

Error: invalid arguments: invalid output format "pretty" used. You can only specify json or csv

script returned exit code 1

[alexpdp7]

set +x curl -k -L -H "Authorization: Bearer ${ROX_CREDS_PSW}" https://${ROX_CREDS_USR}/api/cli/download/roxctl-linux --output roxctl > /dev/null; chmod +x roxctl > /dev/null export ROX_API_TOKEN=${ROX_CREDS_PSW} ./roxctl image scan --insecure-skip-tls-verify -e ${ROX_CREDS_USR}:443 --image image-registry.openshift-image-registry.svc:5000/${DESTINATION_NAMESPACE}/${APP_NAME}:${VERSION} --format json

Flag --format has been deprecated, please use --output/-o to specify the output format. NOTE: The new JSON / CSV format contains breaking changes, make sure you adapt to the new structure before migrating.

Scanning image failed: rpc error: code = Internal desc = image enrichment error: error getting metadata for image: image-registry.openshift-image-registry.svc:5000/fulanitos-test/pet-battle:1.3.1 error: error getting metadata from registry: "Autogenerated https://image-registry.openshift-image-registry.svc:5000 for cluster do500": Failed to get the manifest digest : Head "https://image-registry.openshift-image-registry.svc:5000/v2/fulanitos-test/pet-battle/manifests/1.3.1": http: non-successful response (status=401 body=""). Retrying after 3 seconds

Scanning image failed: rpc error: code = Internal desc = image enrichment error: error getting metadata for image: image-registry.openshift-image-registry.svc:5000/fulanitos-test/pet-battle:1.3.1 error: error getting metadata from registry: "Autogenerated https://image-registry.openshift-image-registry.svc:5000 for cluster do500": Failed to get the manifest digest : Head "https://image-registry.openshift-image-registry.svc:5000/v2/fulanitos-test/pet-battle/manifests/1.3.1": http: non-successful response (status=401 body=""). Retrying after 3 seconds

Scanning image failed: rpc error: code = Internal desc = image enrichment error: error getting metadata for image: image-registry.openshift-image-registry.svc:5000/fulanitos-test/pet-battle:1.3.1 error: error getting metadata from registry: "Autogenerated https://image-registry.openshift-image-registry.svc:5000 for cluster do500": Failed to get the manifest digest : Head "https://image-registry.openshift-image-registry.svc:5000/v2/fulanitos-test/pet-battle/manifests/1.3.1": http: non-successful response (status=401 body=""). Retrying after 3 seconds

Error: rpc error: code = Internal desc = image enrichment error: error getting metadata for image: image-registry.openshift-image-registry.svc:5000/fulanitos-test/pet-battle:1.3.1 error: error getting metadata from registry: "Autogenerated https://image-registry.openshift-image-registry.svc:5000 for cluster do500": Failed to get the manifest digest : Head "https://image-registry.openshift-image-registry.svc:5000/v2/fulanitos-test/pet-battle/manifests/1.3.1": http: non-successful response (status=401 body="")

script returned exit code 1

[alexpdp7]

The next step is also problematic:

set +x export ROX_API_TOKEN=${ROX_CREDS_PSW} ./roxctl image check --insecure-skip-tls-verify -e ${ROX_CREDS_USR}:443 --image image-registry.openshift-image-registry.svc:5000/${DESTINATION_NAMESPACE}/${APP_NAME}:${VERSION} --json --json-fail-on-policy-violations=false:

Flag --json has been deprecated, use the new output format which also offers JSON. NOTE: The new output format's structure has changed in a non-backward compatible way.

Flag --json-fail-on-policy-violations has been deprecated, use the new output format which will always fail with policy violations.

Checking image failed: rpc error: code = Internal desc = image enrichment error: error getting metadata for image: image-registry.openshift-image-registry.svc:5000/fulanitos-test/pet-battle:1.3.1 error: error getting metadata from registry: "Autogenerated https://image-registry.openshift-image-registry.svc:5000 for cluster do500": Failed to get the manifest digest : Head "https://image-registry.openshift-image-registry.svc:5000/v2/fulanitos-test/pet-battle/manifests/1.3.1": http: non-successful response (status=401 body=""). Retrying after 3 seconds

Checking image failed: rpc error: code = Internal desc = image enrichment error: error getting metadata for image: image-registry.openshift-image-registry.svc:5000/fulanitos-test/pet-battle:1.3.1 error: error getting metadata from registry: "Autogenerated https://image-registry.openshift-image-registry.svc:5000 for cluster do500": Failed to get the manifest digest : Head "https://image-registry.openshift-image-registry.svc:5000/v2/fulanitos-test/pet-battle/manifests/1.3.1": http: non-successful response (status=401 body=""). Retrying after 3 seconds

Checking image failed: rpc error: code = Internal desc = image enrichment error: error getting metadata for image: image-registry.openshift-image-registry.svc:5000/fulanitos-test/pet-battle:1.3.1 error: error getting metadata from registry: "Autogenerated https://image-registry.openshift-image-registry.svc:5000 for cluster do500": Failed to get the manifest digest : Head "https://image-registry.openshift-image-registry.svc:5000/v2/fulanitos-test/pet-battle/manifests/1.3.1": http: non-successful response (status=401 body=""). Retrying after 3 seconds

Error: rpc error: code = Internal desc = image enrichment error: error getting metadata for image: image-registry.openshift-image-registry.svc:5000/fulanitos-test/pet-battle:1.3.1 error: error getting metadata from registry: "Autogenerated https://image-registry.openshift-image-registry.svc:5000 for cluster do500": Failed to get the manifest digest : Head "https://image-registry.openshift-image-registry.svc:5000/v2/fulanitos-test/pet-battle/manifests/1.3.1": http: non-successful response (status=401 body="")

script returned exit code 1

[alexpdp7]

Also on Tekton:

Failure on task image-scan - check logs for details.

100 52.8M  100 52.8M    0     0   236M      0 --:--:-- --:--:-- --:--:--  236M
Getting roxctl
Flag --format has been deprecated, please use --output/-o to specify the output format. NOTE: The new JSON / CSV format contains breaking changes, make sure you adapt to the new structure before migrating.
Error: invalid arguments: invalid output format "pretty" used. You can only specify json or csv

I did not complete the rest of this section because I suspect it will have the same issues as the Jenkins bit.

[eformat]

updates roxctl command now. --pretty has been removed in latest version - its now 'table' format

we intentionally download the version of the cli from the matching installed ACS (which get minor updates as operator automatically updates)

the flag "--json-fail-on-policy-violations" when checking image scan is no longer supported (it always true now).

this is OK for -api image (tested OK against latest ubi 8.5) which i have rebuilt with latest libs for this cve
https://access.redhat.com/errata/RHSA-2021:4903

is going to be problematic for UI pet-battle image in Jekins since it is not regularly maintained.

the options i see are:

1. ignore the $? return ! which is what ""--json-fail-on-policy-violations=false" was doing - this is not ideal as its probably bad practice :) e.g

2. fix the upstream build of pet-battle to use patched/latest base image and libs so there are no critical/high warnings - this is preferable i think but a little work (@springdo @ckavili WDYT ?)

@alexpdp7 - should be good to retest now with these minor fixes now, i.e. won't fail on parsing/args now.

[eformat]

Option 3/4. we just document the

exit 0

strategy ... i have a feeling its the season for high vuln. just got caught by this literally this afternoon - broke my build

https://access.redhat.com/security/cve/cve-2021-37136

what we could do .. is also change the policy to break the build on critical only (instead on important)

[ckavili]

agree on breaking the build on critical only. But also let's do that little work, update the base image - I can take the ownership of that #75 :)

[eformat]

OK, i'm going to close this for now as the formatting and CLI has been fixed which was the original issue.