Security scan fails with an empty result list when used from a NuGet package. (OSOE-767)

[sarahelsaig]

When imported from a project reference, the ZAP report doesn't fail the test on low risk level or suppressed alerts. The same doesn't seem to be true when used form the NuGet package.

For example check out this run for OrchardCMS/OrchardCore.Commerce@b6d2897. If you download the failure dump, the ZAP report shows no actual alerts so it shouldn't have been created:

On the other hand, if you look through the 2024-01-09-ZAP-Report-localhost.json in the same archive it has warnings for id 10202, which has been suppressed here. We know this suppression works, because that's the reason why the entries don't show up in the HTML report. For now the only workaround I found was to also provide a custom assertion logic that detects and skips the alerts caused by the suppressed code.

Jira issue

[Piedone]

The linked code's comment mentions there being a discrepancy between the HTML and JSON reports (the workflow artifacts are expired, unfortunately). So, this seems to be rather a ZAP bug to me than a difference between the source and NuGet versions of the project.

[Piedone]

Found this BTW: OrchardCMS/OrchardCore.Commerce#404.

[Piedone]

It was an issue in our automation framework plans.