In response to Competency Question 3, "Which content types carry zip?"

Answers to that question should return the various x formats of Microsoft Office files, e.g. .docx. That is, they should return any file with what would be recorded in today's UCO as:

...
{
  "@type": "observable:ContentDataFacet",
  ...
  "observable:mimeType": "application/vnd.openxmlformats-officedocument.wordprocessingml.document"
  ...
}
...

This would be discoverable via taxonomic knowledge that that MIME type is a narrower concept than application/zip. Many of these relationships with zip can be found by review of the IANA documenting page - here is the .docx page. Unfortunately, reviewing for just the string "zip" among those pages won't be a perfect zip-relationship finder, as application/gzip is not related.

One part of the Solution of this proposal has been posted, in PR 377. The earlier commits document the rationale for some class selection and extensions.

This PR includes demonstration in unit tests of how MIME taxons can be used while looking close to string-literal properties (see kb:file-8 in this file), define taxons as IANA, non-IANA, or only a more general dcterms:FileFormat found to have been used in other RDF data models (see the nodes in this file, or its corresponding tests in test_mime.py).

The MIME demonstrations in that PR also include demonstration of a hierarchy between Media Types, including some "aliasing" relationships. test_mime.py includes demonstrations of finding all application/gzip content, even if that content is encoded as narrower taxons such as application/tar+gzip.

I cannot yet share the repository that generates the IANA Media Registry, but here are a few examples of taxons included in its generated monolithic resource:

<http://purl.org/NET/mediatypes/application/gzip>
        a
                dcterms:FileFormat ,
                skos:Concept
                ;
        dcam:memberOf dcterms:IMT ;
        skos:exactMatch <https://taxonomy.unifiedcyberontology.org/uco/mime/0.0.1/application/gzip> ;
        .

<https://taxonomy.unifiedcyberontology.org/uco/mime/0.0.1/application/gzip>
        a
                prov:Entity ,
                uco-types:IANAMediaType
                ;
        rdfs:isDefinedBy <https://www.iana.org/assignments/media-types/application/gzip> ;
        skos:exactMatch <http://purl.org/NET/mediatypes/application/gzip> ;
        skos:inScheme uco-mime:MIMEScheme ;
        skos:notation "application/gzip" ;
        .

<https://taxonomy.unifiedcyberontology.org/uco/mime/0.0.1/application/tar+gzip>
        a uco-types:NonIANAMediaType ;
        rdfs:comment "The media type suffix +gzip is registered in RFC 8460, Section 6.3."@en ;
        rdfs:seeAlso <https://www.rfc-editor.org/rfc/rfc8460.html#section-6.3> ;
        skos:broader
                <https://taxonomy.unifiedcyberontology.org/uco/mime/0.0.1/application/gzip> ,
                <https://taxonomy.unifiedcyberontology.org/uco/mime/0.0.1/application/tar>
                ;
        skos:notation "application/tar+gzip" ;
        .

I selected those excerpts because:

• .tar.gz is a file extension encountered in supply chain security review, yet currently lacks a taxon in the IANA registry
• Dublin Core provides a class and prefix for Media Types in their metadata publishing guide, including the demonstration in Turtle ex:myPicture dcterms:format mime:jpeg ..
• The second UCO taxon defined in the above excerpt covers some uniqueness constraints found in the SKOS specification. In short, because of these constraints, it can lead to an inconsistent SKOS knowledge base if one tries applying SKOS annotations to concepts outside one's own namespaces. Hence, there is a set of Media Types UCO defines that is mostly parallel with the IANA+Dublin Core set.

There is also a prov:Entity declaration, which handles another "Aliasing" issue that happens to be relevant to UCO users. text/xml is defined in RFC 7303 as an alias of application/xml. There turns out to not be a SKOS mechanism to support one concept being an alias of another when both are in the same concept scheme, so the taxonomy resorts to prov:alternateOf to handle these aliasing cases.

At the end of the day, the (pending-release) taxonomy will permit discovery of all XML files in a knowledge base by including this in a SPARQL query:

?nMimeType
    skos:exactMatch* /
      skos:broaderTransitive* /
        prov:alternateOf* /
          skos:exactMatch* /
            skos:notation "text/xml" ;

As an aside, it was surprisingly complex to determine how to work with forward slashes in prefixed concepts between Turtle, JSON-LD, and SPARQL. Each syntax handles forward-slash differently. Further details for those curious are in this bug report and its attached PR. Fortunately, usage of skos:notation within the present proposal saves UCO from having to deal with that nuance, because what one would search for in SPARQL would be a string value, and what one would encode in JSON-LD can look like the Media Type name, with some kind of mime: prefix.

@ajnelson-nist What needs to be done before submitting this to solutions review?

The taxonomy repository has been posted here, with some notations that it is currently experimental and needs committee review:

https://github.com/ucoProject/UCO-Taxonomy-MIME

We should now have sufficient demonstration available to show whether these changes to observable:mimeType and related are suitable for adoption.

This continues to be an invalid change proposal under the CDO charter foundational principles as discussed at length for the issue proposing changing controlled vocabularies to use semantic individuals rather than strings.
This is simply a proposal to make such a change for a single vocabulary. It is invalid for the same reasons the broader issue is invalid.
This (mime types) is even one of the two example areas (the other being kindOfRelationship) explicitly discussed where the proposed changes will quickly lead to problems because they are areas that UCO can never assure that they are complete. Both of these areas will always include values where UCO individuals are not current or where specific parties may need to use values outside of any appropriate standardization within UCO. Ironically, to demonstrate this concern the official IANA media type registry has been modified/updated on Aug 22, 2022 since the MIME taxonomy proposed here was last updated meaning it is already out of date.
The proposal is invalid due to a requirement that a user ontologically define an individual for a mime type (for which there is a clear string format standard defined) they wish to use that is outside the current UCO taxonomy version which is an impractical burden for the large majority of likely adopters. In this case, if a user needs to reference a MIME type not currently in the UCO taxonomy, such as if they submitted it and it was approved for IANA registration but a new UCO taxonomy updated version has not been released, they will be forced to define their own ontological entry (an already impractical burden) but now when the UCO taxonomy is updated with the new IANA registry they will now have divergent content (the interim individual they created and the new "official" individual).

When the issue of the invalidity of the proposal for changing vocabs to semantic individuals was discussed the ball was placed in the proposers court to suggest an approach that allowed the rigor of semantic individuals while allowing the required flexibility of strings. I have seen no such suggestion and this specific change proposal does not offer such a suggestion but rather is simply a single subcase of the broader proposal that was rejected.

Re: Rejection of broader proposal

I do not believe the proposal was rejected, but rather withheld due to lack of technical knowledge. We have significantly more technical knowledge now.

Re:

... under the CDO charter foundational principles as discussed at length ...

Only one person has declared those principles foundational, and other similar-strength language, and we have already seen those principles (I suspect the act one you're referencing) be used to attempt to move UCO away from a technology that is essential for any interoperability with other RDF-based technologies, which would be a significant act of UCO's isolation if enacted.

The entire UCO design document is still under committee review.

Re: incompleteness

I agree on lag being a possibility. The taxonomy does create a need for an operational tempo. In support of a regular deployment tempo, maintenance of the taxonomy for adding is almost entirely scriptable. make already handles refreshing the taxonomy entirely (observe its CI). It doesn't yet handle stamping a new versionIRI, but that scripted step can be added as well. I'm not sure if Github steps on generating releases could or should be automated, but "Refreshes" could be a calisthenic for Github mechanics, which our community does need in discussions with the "Product Manager" role that has been left unfilled for a while.

Re:

The proposal is invalid due to a requirement that a user ontologically define an individual for a mime type (for which there is a clear string format standard defined) they wish to use that is outside the current UCO taxonomy version which is an impractical burden for the large majority of likely adopters.

You missed some details where the semi-open vocabulary enforcement pattern of gently (sh:Info-severity) suggesting using individuals stronger than dcterms:FileFormat is used.

mimeType is changed to the class dcterms:FileFormat. Those individuals exist outside of UCO, and don't need to exist as defined by Dublin Core. (Indeed, today they aren't deployed, which is a graph-provisioning matter to discuss with their team. I've had some pleasant maintenance interaction with them recently, though, so I think this is approachable.)
observable:mimeType-class-types-MIMEFormat implements that gentle suggestion.

A firmer sh:Warning-severity shape requests whatever individuals used have skos:Notation so the text form can be available. I don't think that should be made into a sh:Violation-severity shape, because if the individuals do get provided by Dublin Core, they aren't guaranteed to use SKOS concepts.

Re: Keeping mimeType a string

I have not seen arguments against the benefits of this proposal that strings make more difficult.

• Does the MIME type for JPEG pictures have the e in jpeg or no? How do we enable interoperability when two significant knowledge bases disagree?
• What is the MIME type string for tarballs? I found a few candidate spellings. How would UCO encode that this set of non-IANA-recognized values mean "tarball"?
  • application/tar+gzip
  • application/x-tar+gzip
• Many formats are specialized zips. How could we encode that application/vnd.openxmlformats-officedocument.wordprocessingml.document entails an additional MIME type of application/zip?

I still believe UCO's current pattern pattern of semi-open vocabularies will hit a technical insurmountability with "Extension" vocabularies that are sets of strings that somehow use a non-public namespace. I admit the implementation in this MIME proposal does not yet get all the way to using the pattern I think the other properties will need. Other properties, especially kindOfRelationship as a string, need to be anchored off of unique string labels of concepts tied into classes in UCO. For the MIME proposal, SHACL-SPARQL seems to be the only functional mechanism.

If this proposal is not accepted today, I will adjust it. The MIME taxonomy does by itself serve a need in CASE-Corpora (DCAT uses dcterms:FileFormat), so we may need to look to demonstration in CASE-Corpora to understand when the implementation is satisfactory.

Only one person has declared those principles foundational

This is absolutely NOT one person declaring those principles foundational.
They are explicitly defined as such in the CDO charter and the charter specifically states that changes can not be made by the TSC or OCs that violate them.
It is simply not an option and not valid to hold a requirements review or solutions approval vote on.

As discussed before, I fully recognize the benefits fo having a formally defined taxonomy (some of which you identify above) and would love to have one. However, we cannot require users of UCO to refer to values directly as members of such a formal taxonomy for all the reasons discussed. It is very impractical for the large majority of UCO target users to have ontological knowledge and more specifically knowledge of how UCO uses ontology to define taxonomy individuals simply to express values outside what UCO has defined, especially for values such as MIMEType and kindOfRelationship where any UCO taxonomy or any other is guaranteed to be incomplete.
I have no objection to including the taxonomy as a definition reference for string values used but it must not be the direct mechanism for expressing those values.
At a minimum, we need to be able to express values as strings.
It would be better to have that with a reference taxonomy of known (vocab) string values.
It would be best to find a way to more formally bridge between flexible string values and the reference taxonomy when used values align to reference definitions for those values.
I have not yet seen a proposal for how to do that last part.

This was not voted on today for a few reasons. Not necessarily all of them are listed here:

• There is an operations issue where the MIME taxonomy would need to have a designated maintenance team.
• There is a solution possible using a new object-property, something like observable:mimeTaxon, that would let workflows that have stronger needs for graph individuals use those types rather than the string-typed mimeType.
• There is likely a more general design solution to semi-open vocabularies, that could let this proposal turn into a backwards-compatible change, and not require implementing a string-enumerant rdf:List in the uco-vocabulary namepace. The solution would likely involve a SHACL-SPARQL constraint.

A further note on the current uncontrolled nature of mimeType strings: This property currently has no restrictions on its values. A proposal was raised in the committee call this morning to implement a vocabulary, and I opposed it for an unfortunate technical reason. There is a performance issue with rdf-toolkit.jar where it suffers a significant slowdown when working with rdf:Lists of any appreciable length. I'm not sure what the exact number for "Appreciable" is, but the observable relationship vocabulary currently in the uco-vocabulary namespace is past that number, and it currently takes over a minute to normalize that file's contents. If this is a super-linear slowdown, a mimeType controlled vocabulary set could make normalizing the uco-vocabulary namespace untenable. This is an unfortunate issue that needs to be worked through with the maintainers of rdf-toolkit.

Looking forward, I still believe IRI-identified graph individuals will be a necessity as a full replacement for the current UCO semi-open vocabulary design. I think the resolution, using classes developed for this proposal, would involve something like this query as part of a sh:PropertyShape with target sh:targetObjectsOf observable:mimeType:

SELECT $this
WHERE {
    FILTER NOT EXISTS {
        ?nTaxon
            a/rdfs:subClassOf* types:MIMEFormat ;
            skos:notation $this ;
            .
    }
}

I think that is a correct way, but using a NOT EXISTS makes me wonder if it's a sufficiently fast way.

I think this will need to be exercised further before returning for committee review. Likely, it will be tried in CASE-Corpora as that graph grows, particularly with an eye towards "govdocs1".