Provide a warning when matching against a group

[nikitawootten-nist]

User Story:

As a user, I would like a warning or error to be thrown when attempting to include a group.

As someone new to profile resolution, I would like explicit feedback that profile resolution cannot operate on groups.

As an example, consider the following excerpt:

{
    "profile": {
        "uuid": "c0dc468c-934e-4fe9-b5bf-9fc63f5a2915",
        "metadata": {
            "title": "Sample Security Profile *for Demonstration* and Testing",
            "last-modified": "2023-04-10T10:31:28.355446-04:00",
            "version": "1.0",
            "oscal-version": "1.04",
            "remarks": "The following document is used in the OSCAL Profile Tutorial and builds on the catalog created for the OSCAL Catalog Tutorial"
        },
        "imports": [
            {
                "href": "https://raw.githubusercontent.com/usnistgov/oscal-content/main/examples/catalog/json/basic-catalog.json",
                "include-controls": [
                    {
                        "with-ids": [
                            "s1.1"
                        ],
                        "with-child-controls": "yes"
                    }
                ]
            }
        ]
    }
}

The above profile excerpt will not yield a resolved catalog with all of the children of the group "s1". Instead, the current behavior is to fail silently and omit the import directive entirely:

{
  "catalog" : {
    "uuid" : "7661c744-1a8e-4151-beb3-3bd88e661a71",
    "metadata" : {
      "title" : "Sample Security Profile *for Demonstration* and Testing",
      "last-modified" : "2023-04-10T17:36:52.221809216Z",
      "version" : "1.0",
      "oscal-version" : "1.04",
      "props" : [ {
        "name" : "resolution-tool",
        "value" : "libOSCAL-Java"
      } ],
      "links" : [ {
        "href" : "file:///WORKING/basic-profile%20copy.json",
        "rel" : "source-profile"
      } ]
    }
  }
}%                                                                                                                                                                              

For catalogs that rely on groups of groups (such as the above example) this can cause quite a bit of confusion.

Goals:

• When a user attempts to resolve a profile and an "include-controls" or "exclude-controls" block references a group, throw a warning or error that this inclusion will be ignored/is invalid.

Dependencies:

N/A

Acceptance Criteria

• All website and readme documentation affected by the changes in this issue have been updated.
• A Pull Request (PR) is submitted that fully addresses the goals of this User Story. This issue is referenced in the PR.
• The CI-CD build process runs without any reported errors on the PR. This can be confirmed by reviewing that all checks have passed in the PR.

[nikitawootten-nist]

As @wendellpiez mentioned in usnistgov/OSCAL#1746, maybe it would even be cleaner to generalize a bit and throw a warning/error when an invalid control-id is specified.

[GaryGapinski]

If the inability to include a group (as opposed to a control) is seen as desirable, then throwing an error is appropriate.

This of course means that one cannot identify a group of controls in a profile resolution, and instead must identify each and every desired control within the group even when all are desired, further taking into account controls which are descendants of controls. So much for including all of group "PE", e.g., with a single imperative.

Were group inclusion to be countenanced, the name of include-controls/with-id would be retroactively unfortunate with respect to its parent. Absence of group inclusion reduces the utility of catalog/group (a catalog/group cannot be considered as a selection in profile resolution).

Possibly a consideration: the domain (and unique nature, namely "A human-oriented, locally unique identifier with instance scope that can be used to reference this control elsewhere in this and other OSCAL instances (e.g., profiles). This id should be assigned per-subject, which means it should be consistently used to identify the same control across revisions of the document") of id in a catalog does not seem to be constrained to all control attributes. If "per-subject" means what it appears to mean, then ids can be shared amongst disparate elements. Perhaps with-controls/with-id should have been with-control/@id?

[wendellpiez]

😁 Most RL catalogs would support profiles selecting groups implicitly, by using the matching syntax:

<include-controls>
  <matching pattern="ac.*"/>
</include-controls>

The requirement to use glob matching (which has other uses too, e.g. pattern="*.1" is what accounts for the cardinality of both matching and with-id under including-controls.

[aj-stein-nist]

To start, if we review the catalog, the ID s1.1 is a group ID, not a control ID. That said, your points around messaging this is something we could warn about has merit.

So there are two issues at play here, one on model enhancements, and the other being oscal-cli messaging that should be derived that.

1. We should likely add a constraint to the OSCAL model to check for profile include directives to look for a key in an index of all provided catalogs in a given profile with the set of all possible control IDs.
2. For oscal-cli, after that, we can potentially back the warnings requested here with that to more accurately identify nonexistent controls or incorrect use of group IDs as arguments to include or other directives.

More to follow.

[aj-stein-nist]

Dave also reported a different but related issue that the updated include-controls code to better align with the resolution spec's intent as changed for #1662 did not properly add a <group-as />. That is also related to this but additionally breaks schema gen code for metaschema-java/liboscal-java/oscal-cli stack.

https://github.com/usnistgov/OSCAL/blame/337ed9337ab088c5929183d55bbd65a605b843ae/src/metaschema/oscal_profile_metaschema.xml#L84

I will need to follow up and open an issue in the OSCAL project next week.

[david-waltermire]

Reviewing previous comments, I believe instead of warning that a group is referenced, the better thing to do is to add an error constraint that checks if the id does not match a control id in the referenced catalog or profile. In this way, if a group is referenced, then an error would be raised indicating that the referenced id is not for a control in the referenced source or resolved catalog.