Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(cli): make opa test --exit-zero-on-skipped fail with failing test rules #6127

Merged
merged 1 commit into from
Jul 29, 2023
Merged

fix(cli): make opa test --exit-zero-on-skipped fail with failing test rules #6127

merged 1 commit into from
Jul 29, 2023

Conversation

fdaguin
Copy link
Contributor

@fdaguin fdaguin commented Jul 28, 2023

Why the changes in this PR are needed?

Closes #6126.

What are the changes in this PR?

  • Fixes the failed logic introduced in 54f203c to compute the exit code of a test run
  • Adds unit tests to cover this area

Further comments:

I consider this is a BREAKING-CHANGE: all passing CIs that were launching opa test --exit-zero-on-skipped including both skipped and failed test rules will now break.

ashutosh-narkar
ashutosh-narkar reviewed Jul 28, 2023
View reviewed changes
Copy link
Member

@ashutosh-narkar ashutosh-narkar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fdaguin thanks for this fix. I did some manual tests and the changes look good.

I consider this is a BREAKING-CHANGE: all passing CIs that were launching opa test --exit-zero-on-skipped including both skipped and failed test rules will now break.

Can you please add a note about this in the Change Log so that we can point this out in the next OPA release notes.

@netlify
Copy link

netlify bot commented Jul 28, 2023

✅ Deploy Preview for openpolicyagent ready!

Name Link
🔨 Latest commit 56019a7
🔍 Latest deploy log https://app.netlify.com/sites/openpolicyagent/deploys/64c4401ddcf0d30008a812e4
😎 Deploy Preview https://deploy-preview-6127--openpolicyagent.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@fdaguin
Copy link
Contributor Author

fdaguin commented Jul 28, 2023 •
edited
Loading

@fdaguin thanks for this fix. I did some manual tests and the changes look good.

I consider this is a BREAKING-CHANGE: all passing CIs that were launching opa test --exit-zero-on-skipped including both skipped and failed test rules will now break.

Can you please add a note about this in the Change Log so that we can point this out in the next OPA release notes.

Thanks for your review! I documented this breaking change in 56019a7.

ashutosh-narkar
ashutosh-narkar approved these changes Jul 28, 2023
View reviewed changes
Copy link
Member

@ashutosh-narkar ashutosh-narkar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for the fix @fdaguin!

@ashutosh-narkar
Copy link
Member

@fdaguin can you please squash your commits and we can get this in. You can use these guidelines to author your commit message.

@fdaguin
cmd: make opa test -z fail with failing tests
70eaf52 
Fixes the failed logic introduced in [1] to compute the exit code of a
test run. When used with the option `--exit-zero-on-skipped`, the
`opa test` command now returns the exit code 0 only if no failed tests
were found.

[1]: open-policy-agent@54f203c

Fixes: open-policy-agent#6126
BREAKING-CHANGE
Signed-off-by: Florian Daguin <[email protected]>
@fdaguin
Copy link
Contributor Author

fdaguin commented Jul 28, 2023

@ashutosh-narkar I squashed my commits 🙂

@ashutosh-narkar ashutosh-narkar merged commit ca7da22 into open-policy-agent:main Jul 29, 2023
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ashutosh-narkar ashutosh-narkar ashutosh-narkar approved these changes

Assignees
No one assigned
Labels
waiting-for-response
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

opa test --exit-zero-on-skipped always returns exit code 0 when tests are skipped
2 participants
@fdaguin @ashutosh-narkar