-
Notifications
You must be signed in to change notification settings - Fork 8
Mark Wilkinson's comments
----------- REVIEW -----------
The authors describe the "Crop Ontology", which appears to be all of (a) a website for hosting and browsing crop-related ontologies, (b) a set of community-generated ontologies, and (c) a set of tools for creating and visualizing ontologies and commenting on them.
Given this broad set of things and activities, it seems a somewhat odd title. It seems that something like "The Crop Ontology Project" might be a more appropriate title? In addition, given this broad scope, the paper probably should have been submitted as a long-paper, rather than the 8 pages, since it was difficult to really understand what the authors had achieved, and what significance these contributions have in the context of other initiatives, based on reading the text (e.g. the "features" section doesn't explain how the authors achieved any of these features - did they take existing tools and bring them together in this interface, or did they design each tool from scratch?)
I ended up spending considerable time exploring both their interface, and the data behind it. It is difficult to separate my generally positive feelings in support of what the authors are attempting to do (and the difficulty of doing so!) from my generally negative feelings about the current technical/ontological quality of the output of the system. I understand that these are community-generated ontologies, and that we cannot expect them to have any ontological rigor, but at the same time the aggrigation of low-quality resources without somehow improving them isn't necessarily helping the community... Yes, the API does help the community find things more easily, but if what they find is of questionable quality, then you're really just enabling bad annotation to be faster.
So, let me split my comments into three main areas: Interface, Ontologies, and technical aspects
INTERFACE: The interface was very well-done!! It worked exactly as-advertised, provided very clear navigation, and was laid-out in a very intuitive manner. I not only found it easy to use, I enjoyed using it! I liked the simplicity of their code-level API also; however I had some discomfort in that they appear to be duplicating much of the effort of the OBO API. In fact, I found myself wondering why there was a need to have an ontology repository for many crop-related terminologies outside of the main global ontology repository at OBO, and moreover, one that duplicates much of OBOs functionality. This absolutely needs to be justified in the manuscript text.
Ontologies: I am hesitant to blame the authors for the quality of community-generated content. Unfortunately, the authors say "Moderators of the system make sure everything is done correctly, using good semantic practices" so they have begged for the reviewers to challenge that assertion! Here are some examples of semantic oddness that left me extremely perplexed:
:CO_338:0000076 a rdfs:Class; rdfs:label "Pod length"@en;
:CO_338:0000119 a rdf:Property; rdfs:label "Average number of pods from 3 or 5 randomly selected plants"@en;
Ontologies all appear to be sub-classed from something called "ROOT", which (when you attempt to resolve it's URL) gives a 404. Good practice wouldn't do that... and moreover, this will break many/most semweb tools.
In the Solanaceae phenotype ontology I found the following, very worrisome set of assertions: leaf variegation - is_a - leaf color - is_a - leaf - is_a - Solanaceae phenotype ontology
...and there were dozens of other examples of not using "good semantic practices" in many other ontologies I looked at. This is going to be particularly problematic if, as the authors suggest, they begin migranting from "loose" semantics into something like OWL, where words have very strict meanings.
These are just a few examples, but there were many others even from a superficial examination of the content of their repository, so I might suggest that the authors be a bit more circumspect in talking about things being "done correctly" and "using good semantic practices".
Technical problems:
Ontologies that are claimed to be owl ontologies are not - they are rdfs ontologies. For example: http://www.cropontology.org/ontology/CO_338 a owl:Ontology;
there are no owl axioms in that ontology whatsoever. This may or may not be a problem, depending on which parser you are using, but it's technically not correct.
the reverse of that problem is also true. For example:
http://www.cropontology.org/rdf/CO_321:0000118 resolves to a fragment of RDF that contains OWL axioms, but does not declare itself to be an OWL ontology. This is problematic because there is an owl:Annotation element, which (as far as I am aware) is not allowed outside of an owl:Ontology/ block. As such, these fragments of RDF/XML break OWL parsers such as Protege (though they are valid RDF, they are not valid OWL, and since only OWL parsers can make use of OWL axioms, breaking those parsers is a serious problem!)
using the same example http://www.cropontology.org/rdf/CO_321:0000118 contains some other oddness. First, the RDF does not contain the same information that the Web page of the same term contains (http://www.cropontology.org/terms/CO_321:0000118/) which is not technically wrong, but it is a bit disappointing. ((As an aside, many ontology projects are now using the RDF as the web page, and formatting it nicely using a style-sheet, in order that these kinds of differences don't happen; moreover, there are now two URIs that represent the same thing - one is the web page of the thing, and the other is the RDF node of the thing... which one of these are we supposed to use in our Linked Data? This, again, is not "good semantic practice"))
using the same example again: http://www.cropontology.org/rdf/CO_321:0000118 contains an OBO dbXref axiom, but doesn't contain any cross-reference information. It seems to contain a duplication of the term definition.
The turtle that is provided from the RDF links on the website does not parse into Protege (e.g. http://www.cropontology.org/ontology/CO_338/Chickpea/ttl). I wasn't able to trouble-shoot that problem...
So, in summary, the project has the potential to be extremely useful to the community. It has a very user-friendly and well-thought-out interface (albeit with many features that appear to duplicate the functionality of OBO), but unfortunately the interface leads to data that is highly suspect, both logically and technically. The manuscript is well-written, but considering the scope of the project, is overly superficial (this needed to be a long paper), and moreover, promises a quality higher than what is delivered.
I do look forward to seeing future iterations of both the project and this manuscript, as it is clearly going to be a valuable resource when the technical problems are resolved!
Best wishes to the authors,
Mark Wilkinson