UCO semi-open vocabularies need to become enumerations of xsd:strings

[ajnelson-nist]

Background

UCO gives sets of strings that suggest values to use for certain properties. UCO has called this the "Semi-open vocabulary" design pattern: While certain strings should be used, strings not included in UCO's ontology files can still be used instead when the user knows the value they are using is not yet in UCO, and/or might never be in UCO for whatever reason (e.g., confidentiality of non-public vocabularies).

UCO uses a syntax for its semi-open vocabularies that uses what has appeared to date to be an OWL design pattern for defining a datatype as a set of typed string values. However, it turns out the pattern used by UCO is invalid OWL syntax. Issue 593 covers half of the issues raised by an externally-developed OWL syntax reviewing tool, ROBOT¹. Issue 593's correction has no known impact on user data.

The other half of the syntax correction takes this form, which unfortunately differs from UCO's user data guidance since UCO's earliest examples. Given the known non-0 impact on user data, this is being treated as a non-fast-track proposal.

Here is (again, selected for its small size) the Bitness vocabulary, as implemented in UCO 1.3.0 and long before:

vocabulary:BitnessVocab
	a rdfs:Datatype ;
	rdfs:label "Bitness Vocabulary"@en-US ;
	rdfs:comment "Defines an open-vocabulary of word sizes that define classes of operating systems."@en ;
	owl:equivalentClass [
		a rdfs:Datatype ;
		owl:onDatatype xsd:string ;
		owl:oneOf (
			"32"^^vocabulary:BitnessVocab
			"64"^^vocabulary:BitnessVocab
		) ;
	] ;
	.

Issue 593 cut the triple [] owl:onDatatype xsd:string .. This Issue finishes the OWL syntax correction by changing the members to values typed only as xsd:string (which is the OWL-wide (IIRC, RDF-wide) default when no other type is stated):

vocabulary:BitnessVocab
        a rdfs:Datatype ;
        rdfs:label "Bitness Vocabulary"@en-US ;
        rdfs:comment "Defines an open-vocabulary of word sizes that define classes of operating systems."@en ;
        owl:equivalentClass [
                a rdfs:Datatype ;
                owl:onDatatype xsd:string ;
                owl:oneOf (
                        "32"
                        "64"
                ) ;
        ] ;
        .

The syntax error this addresses is that the enumeration attempted to assign a datatype to the enumeration's member-literals; but, that datatype is not yet defined at the time the enumeration is being consumed by the parsing process, because they are inside the referenced datatype's definition. ROBOT describes this as a cyclic reference in a definition, and calls it a syntax error. The "Bug description" section below goes into the details describing why the cycle is also a symptom of another syntax error.

Requirements

Requirement 1

UCO vocabularies must adhere to OWL syntactic requirements for custom datatypes.

Requirement 2

Any UCO vocabulary X must not use itself as a datatype in its enumerated literals, due to OWL syntax requirements (cited in "Bug description" section).

Risk / Benefit analysis

Benefits

• Presently, OWL tools using the OWL API library¹ declare UCO as having syntax errors (one per vocabulary) using the cyclic-reference pattern. (The error's call stack ends in this method.) From (offline) testing the patch with ROBOT, this change proposal addresses that error type, removing a barrier to compatibility for other OWL-based ontologies that might adopt UCO.
• Using plain xsd:string in vocabularies resolves a blocking design question on JSON-LD context dictionaries, on how to assign datatypes to semi-open vocabulary properties without inducing SHACL errors when a value outside the suggested set is used.
• This reduces programming languages' complexities in assigning datatypes to certain properties only when certain values are used. (This is a concern downstream from the UCO ontology, and applies to e.g. Python bindings.)

Risks

• All CASE and UCO data using semi-open vocabularies would need to revert vocabulary assignments to simple strings. This would be a backwards-incompatible change because 1.3.0-and-prior data would be flagged as erroneous, but this change can be implemented in stages with the warning pattern used for other proposals with pre-2.0.0 and post-2.0.0 effects.

Competencies demonstrated

Competency 1

A hash is being migrated from UCO 1.3.0 to UCO 1.4.0. The form in UCO 1.3.0 is:

{
    "@context": {
        "kb": "http://example.org/kb/",
        "types": "https://ontology.unifiedcyberontology.org/uco/types/",
        "vocabulary": "https://ontology.unifiedcyberontology.org/uco/types/"
    },
    "@graph": {
        "@id": "kb:Hash-9ed55c42-3204-4e45-ace7-7ae6bd7d8f38",
        "@type": "types:Hash",
        "types:hashMethod": {
            "@type": "vocabulary:HashNameVocab",
            "@value": "SHA3-256"
        },
        "types:hashValue": {
            "@type": "xsd:hexBinary",
            "@value": "36f028580bb02cc8272a9a020f4200e346e276ae664e45ee80745574e2f5ab80"
        }
    }
}

Competency Question 1.1

How is the hash supposed to be spelled with this proposal?

Result 1.1

Note that two lines are dropped: The "vocabulary" prefix from the context dictionary will never be needed by users, and hashMethod no longer needs the typed-literal JSON object in its spelling.

{
    "@context": {
        "kb": "http://example.org/kb/",
        "types": "https://ontology.unifiedcyberontology.org/uco/types/"
    },
    "@graph": {
        "@id": "kb:Hash-9ed55c42-3204-4e45-ace7-7ae6bd7d8f38",
        "@type": "types:Hash",
        "types:hashMethod": "SHA3-256",
        "types:hashValue": {
            "@type": "xsd:hexBinary",
            "@value": "36f028580bb02cc8272a9a020f4200e346e276ae664e45ee80745574e2f5ab80"
        }
    }
}

Competency Question 1.2

How is a hash using an algorithm name not in UCO supposed to be spelled?

Result 1.2

# UCO 1.4.0 form
kb:Hash-9ed55c42-3204-4e45-ace7-7ae6bd7d8f38
	a types:Hash ;
	types:hashMethod "FooHash" ;
	types:hashValue "098f6bcd4621d373cade4e832627b4f6"^^xsd:hexBinary ;
	.

Competency Question 1.3

How should semi-open vocabulary properties' ranges be spelled?

Result 1.3

These properties' ranges should continue to be spelled as the union of xsd:string and the UCO Vocabulary IRI. The OWL 2 Syntax document, Section 9.4 gives an example for a restricted string-property. UCO's semi-open vocabulary still permits general xsd:string, so an OWL Union'd range should be used in owl:DatatypePropertys, and vocabulary membership should be reviewed with SHACL.

Solution suggestion

No changes are necessary to the owl:DatatypeProperty OWL definitions currently using the semi-open vocabulary pattern, taking for example types:hashMethod:

types:hashMethod
	a owl:DatatypeProperty ;
	rdfs:label "hashMethod"@en ;
	rdfs:comment "A particular cryptographic hashing method (e.g., MD5)."@en ;
	rdfs:range [
		a rdfs:Datatype ;
		owl:unionOf (
			vocabulary:HashNameVocab
			xsd:string
		) ;
	] ;
	.

The SHACL shapes for the semi-open vocabulary pattern have been coming as trios of property shapes. They need to be adjusted as follows for the implementing next UCO SEMVER-minor release, so current data raises SHACL warnings instead of errors. Note that the sh:Info-severity "gentle suggestion" shape is now the shape that bears the enumeration.

types:Hash
	sh:property
		[
			sh:datatype xsd:string ;
			sh:message "As of UCO 1.4.0, the datatype to use for types:hashMethod should be xsd:string.  Not using xsd:string will be an error in UCO 2.0.0." ;
			sh:path types:hashMethod ;
			sh:severity sh:Warning ;
		] ,
		[
			sh:maxCount "1"^^xsd:integer ;
			sh:minCount "1"^^xsd:integer ;
			sh:nodeKind sh:Literal ;
			sh:path types:hashMethod ;
		] ,
		[
			sh:message "Value is not member of the vocabulary HashNameVocab." ;
			sh:in (
				"MD5"
				"MD6"
				"SHA1"
				"SHA224"
				"SHA256"
				"SHA3-224"
				"SHA3-256"
				"SHA3-384"
				"SHA3-512"
				"SHA384"
				"SHA512"
				"SSDEEP"
			) ;
			sh:path types:hashMethod ;
			sh:severity sh:Info ;
		]
		;
	.

(An aside: the new sh:Warning shape makes some new cases that previously failed now pass as warnings, like using an integer for hashMethod's value. Given the warnings, this feels like a low-risk relaxation.)

For UCO 2.0.0, the first two shapes would combine, leaving the implementation as follows.

types:Hash
	sh:property
		[
			sh:datatype xsd:string ;
			sh:maxCount "1"^^xsd:integer ;
			sh:minCount "1"^^xsd:integer ;
			sh:nodeKind sh:Literal ;
			sh:path types:hashMethod ;
		] ,
		[
			sh:message "Value is not member of the vocabulary HashNameVocab." ;
			sh:in (
				"MD5"
				"MD6"
				"SHA1"
				"SHA224"
				"SHA256"
				"SHA3-224"
				"SHA3-256"
				"SHA3-384"
				"SHA3-512"
				"SHA384"
				"SHA512"
				"SSDEEP"
			) ;
			sh:path types:hashMethod ;
			sh:severity sh:Info ;
		]
		;
	.

Due to a processing efficiency issue, one final modification from current practice is suggested: The "Gentle suggestion" property shape should change from an inlined, blank-node-identified sh:PropertyShape to a class-independent, IRI-identified sh:NodeShape that uses the targeting statement ... sh:targetObjectsOf types:hashMethod . The reason for this is that certain SHACL reporting practices, like those used in UCO's unit testing, inline all blank node property shapes in the SHACL result-sets. For some vocabularies, especially longer ones that suggest intensive extension (in particular, the 200+-member vocabulary:ObservableObjectRelationshipVocab), this could be hopelessly obscuring in the result sets whenever users engage in local vocabulary extension.

This pattern is suggested for the implementation instead, to offer users (1) a reference to an IRI they can look up on the documentation site instead of getting an inlined and long anonymous shape, and (2) the opportunity to opt out of any specific "Gentle suggestion" shapes by using sh:deactivate:

types:hashMethod-objects-in-shape
	a sh:NodeShape ;
	sh:in (
		"MD5"
		"MD6"
		"SHA1"
		"SHA224"
		"SHA256"
		"SHA3-224"
		"SHA3-256"
		"SHA3-384"
		"SHA3-512"
		"SHA384"
		"SHA512"
		"SSDEEP"
	) ;
	sh:message "Value is not member of the vocabulary HashNameVocab." ;
	sh:severity sh:Info ;
	sh:targetObjectsOf types:hashMethod ;
	.

The replacement for the class's sh:property links would be as follows, using an rdfs:seeAlso to preserve some reference on the documentation website without any other mechanical impact:

 types:Hash
+	rdfs:seeAlso types:hashMethod-objects-in-shape ;
 	sh:property
 		[
 			sh:datatype xsd:string ;
 			sh:maxCount "1"^^xsd:integer ;
 			sh:minCount "1"^^xsd:integer ;
 			sh:nodeKind sh:Literal ;
 			sh:path types:hashMethod ;
- 		] ,
- 		[
- 			sh:message "Value is not member of the vocabulary HashNameVocab." ;
- 			sh:in (
- 				"MD5"
- 				"MD6"
- 				"SHA1"
- 				"SHA224"
- 				"SHA256"
- 				"SHA3-224"
- 				"SHA3-256"
- 				"SHA3-384"
- 				"SHA3-512"
- 				"SHA384"
- 				"SHA512"
- 				"SSDEEP"
- 			) ;
- 			sh:path types:hashMethod ;
- 			sh:severity sh:Info ;
 		]
 		;
 	.

If a user wishes to opt out of the gentle suggestions, they can add this triple in their extension SHACL shapes:

types:hashMethod-objects-in-shape
	sh:deactivated true ;
	.

Bug description

While at least one tool reports that the current UCO vocabulary pattern is a cyclic definition error, the OWL syntax specification and RDF mapping documents together describe the problem as an inappropriate use of an OWL structure, DatatypeDefinition.

Looking through the OWL 2 Mapping to RDF document, Table 16 includes this mapping:

IF this pattern is in the RDF graph G:

*:x owl:equivalentClass y .
{ DR(*:x) ≠ ε amd DR(y) ≠ ε }

THEN the following axiom is added to the ontology OG:

DatatypeDefinition( DR(*:x) DR(y) )

For quick reference, the smbols above mean:

• *:x is an IRI
• y is either a blank node or an IRI
• DR(z) is a function mapping a node z to a DataRange structre in the ontology OG
  • Nodes for DR can be IRIs or blank nodes
• DR(z) ≠ ε states that DR(z) is defined (or equivalently, the function at the value z is not empty)

Taken together, owl:equivalentClass as a predicate between two certain nodes pertaining to DataRanges imply a DatatypeDefinition.

*:x being in a DatatypeDefinition imposes a certain constraint on *:x, with which UCO has been incongruous to date. Section 9.4 of the OWL 2 Syntax document defines that in the structure DatatypeDefinition ( DT DR ), the Datatype that is the first parameter (DT, or *:x, or uco-vocabulary:BitnessVocab) is a "synonym for DR..." And: "The datatypes defined by datatype definition axioms ... have empty lexical spaces and therefore they must not occur in literals."

The example in the OWL 2 syntax document that immediately follows that quote illustrates a datatype a:SSN, which is part of a datatype definition that would be spelled like this in Turtle:

# First axiom: "Declaration( Datatype( a:SSN ) )"
a:SSN
	a rdfs:Datatype ;
	.

# Second axiom: "DatatypeDefinition( ..."
a:SSN
	owl:equivalentClass [
		a rdfs:Datatype ;
		owl:onDatatype xsd:string ;
		owl:withRestrictions (
			[
				xsd:pattern "[0-9]{3}-[0-9]{2}-[0-9]{4}" ;
			]
		) ;
	] ;
	.

# Third axiom: "DataPropertyRange( a:hasSSN a:SSN )"
a:hasSSN
	rdfs:range a:SSN ;
	.

Note the end of the example block states "there can be no literals of datatype a:SSN".

In UCO's case, there can be no literals of datatype vocabulary:BitnessVocab, vocabulary:HashNameVocab, etc.

Coordination

• Tracking in Jira ticket OCUCO-323
• Administrative review completed, proposal announced to Ontology Committees (OCs) on 2024-08-12
• Requirements to be discussed in OC meeting, 2024-08-20
• Requirements Review vote occurred, passing, on 2024-08-20
• Requirements development phase completed.
• Solution announced to OCs on TODO-date
• Solutions Approval to be discussed in OC meeting, date TBD
• Solutions Approval vote has not occurred

• Solutions development phase completed.
• Backwards-compatible implementation merged into develop for the next release
• develop state with backwards-compatible implementation merged into develop-2.0.0
• Backwards-incompatible implementation merged into develop-2.0.0 (or N/A)
• Milestone linked
• Documentation logged in pending release page
• Prerelease publication: CASE develop branch updated to track UCO's updated develop branch
• Prerelease publication: CASE develop-2.0.0 branch updated to track UCO's updated develop-2.0.0 branch

Footnotes

1. Disclaimer: Participation by NIST in the creation of the documentation of mentioned software is not intended to imply a recommendation or endorsement by the National Institute of Standards and Technology, nor is it intended to imply that any specific software is necessarily the best available for the purpose. ↩ ↩²

[ajnelson-nist]

This proposal also has an impact on the implementation of Issue 549, as that issue adds a new vocabulary.