OIP records are encoded with protobuff (v3) messages. These messages are like metadata schema that contains fields and specifications of the type of data that can fit in each field. Every schema of a valid OIP record type must be published in the blockchain prior to its use and it can be used by anyone.
Here we will define the first OIP record types and will explain the hierarchy between them.
Here we define 2 primitive record types: profile
and relationship
. All record types must belong to either these two primitive types.
Type of record to store metadata of things.
This type has four fields only: name
, description
, validRelationships
and notes
. They are mostly self-explanatory, but one requires more digging.
Briefly, validRelationships
is cross-validation that a relationship in the chain is accepted by all parties. For example, in a relationship of employment, the company issues the employment record and the employee, to accept the job, could add that record identifier to the list of validRelationships
. We will be talking more about this in the future.
Type of records to store metadata of relationships between records of the type profile
.
This primitive record has the basic elements of a relationship: name
, description
of the relationship, since
to indicate when the relationship started and two other fields: parties
and roles
.
The field parties
list all the profiles involved in the relationship. The field roles
is a list in the same order as parties
with the role
of each profile listed in parties
.
We will talk more about relationships later.
To build a new record type, we must first define what is its parent type. The OIP record parent type can be any existing record type already published.
Because OIP uses protobuff (v3) to encode the information in the chain we cannot extend the fields of a parent type automatically. Instead, every field in the message of the new type must be reinstated. However, we use the commented line with a specific format in the message to let OIP know the parent type:
// parentType: profile
The new OIP record type must be compatible with its parent and this has implications to how we will write the new message. To make the message compatible, we need to follow the protobuff guidelines.
First, every field in the parent type must be present in the exact form and number in the child type.
Renaming an existing field just because of semantics is not a problem, protobuff will not break, neither the OIP.
Adding new fields are also not a problem, just never change the field number of the parent type.
Also, deleting a field is not a problem. Just make sure to mark the field number as reserved to avoid conflict. You can do that using the keyword reserved
.
The data encoded with this type won't be correctly decoded by the protobuff message of the parent type. Thus, because OIP will skip the error silently, the consequence is that records posted using this record type won't show in searches on the parent type - which is the whole reason why we use hierarchy between records.
If we need to significantly change a record type, we should consider to fork it instead of parent it. In forking a profile we only need to make our type compatible with the parent type of the profile we fork.
DDEMd is a distributed database of electron micrographs datasets that are generated by assays on the electron microscope taking several electron micrographs. The most fundamental part of this database is each individual electron micrograph, which in turn it is a particular type of image.
We decided to build a record type for images, as simple as possible for now. Images are things so we use profile
as the parent type and we add just two more fields: takenBy
and date
to say who took the image and when the image was produced.
Electron micrographs are a particular type of image that has some peculiar metadata associated with it. So we will build the record type electronMicrograph
instead of specifying a field type
in image
.
A general rule of thumb to use the OIP as we design is to make a child type every time we feel tempted to use the field type
without a limited number of choice. For example, blood type is ok, an image of the type electronMicrograph isn't.
Another rule of thumb is that when designing a record type that you think it might be useful to others, make it as simple as possible so more people can use it as a parent type. Don't worry about namespace because in real life each record has its own unique identifier once is placed in the blockchain and it is part of its name.
Here we just named them by the human-readable name "part", but in reality the record type we call image
is named image_9243fcc2d540
where 9243fcc2d540
is the first 15 digits of the transaction id that published the image
record schema.
Also, we sometimes refer to a record type by the immediate parent type followed by the type's name. For example: image.electronMicrograph
. Also we may extend to include further parents: profile.image.electronMicrograph
depending on what we want to call the readers attention to.
The new fields in electronMicrographs is takenWith
, magnification
and defocus
. The latter two fields are specific to electron micrographs, the third one is to describe the instrument that took the image.
The takenWidth
field should be filled with the identifier for the profile of the electron microscope that took the electron micrograph.
For that, we need to make the microscope's profile.
before we do that, we should first search for a parent type that makes sense. Microscopes are devices with a specific function. Thus, we should first search for a parent type that describes devices. Since this is the first project, we will make the record type device
We parent profile
and add three fields to this record type: serial
, manufactures
and parts
.
The field manufactures
takes identifiers of the companies or groups involved in making the device. We will talk about the record type company
very soon.
The field parts
takes identifiers of the records of the type part
. We are not going through that rabbit hole now, so we will leave it as is and hope that someday, someone will build a record type part
to describe components of devices.
Now we can build the record type microscope
.
A microscope is a type of device, that is it. So no fields added.
An electron microscope is a type of microscope, so we build microscope.electron
The same thing as previously but this opens a space for a remark about hierarchy.
We want someone searching for all devices to be able to find a record that describes an electron microscope. But we don't want them to find the record if they are searching for cameras. Some microscopes have cameras but aren't cameras. Thus, camera
is a sister record type of microscope
both with the same device
parent type.
Obviously, there are multiple types of microscopes and if someone is searching for a confocal microscope, we don't want OIP to list electron microscopes.
This is why we make new record types referencing parent types, this will improve the speed of searches within a certain scope.
Ok, now that we can safely fill the field takenWith
on the record electronMicrograph
, we will continue to design our schema.
As a scientific database, DDEMd is not just a set of images, they were taken by specific samples, using certain protocols and etc. We could put all this info in each electronMicrograph but then we thought it would be hard to make it obvious that a subset of these images was collected together, in a single assay.
For this reason, we built the record type assay
to hold the metadata of scientific assays. assay
is a child type of profile
and add the following fields:
samples
, protocols
, executeBy
and data
. Those are things that should be in any assay.
Before we go on explaining these fields, obviously we are not interested in posting a record of the generic assay
type, we want to make electronMicroscopy
assays. As we did before we just build electronMicroscopy
as a copy of assay
that parents assay
.
A few comments here.
First, the field data
is where we will put all the identifiers of each of the records of the type image.electronMicrograph
that was produced as a result of this assay.
Then we will build the record type protocol
to hold information about the protocol used to take these images. Then we will build sample
that will be used to publish records that hold information about the sample: where it was collected, by who, when and etc.
We will save the reader from going in the details of all the other record types involved in here, but it gives a segway for us to talk about relationships between records of type profile
.
We just say that records of the type assay
need to include who executed the assay.
In this field, we should reference the people who were involved in making the assay. For that reason, we created the record type profile.people
to be used to publish metadata about individual people.
Also, this field should hold identifiers of laboratories, universities, companies and any other institution responsible for the assay.
Notice again the pattern here: laboratories, universities, and companies are all institutions.
So we created the profile.institution
record type and also the types profile.institution.laboratory
, profile.institution.university
and profile.institution.company
.
Now we can finally talk about relationships.
Say that Bob's is working on his Ph.D. under the supervision of Prof. Alice at the ACME University.
So Alice is an employee of the ACME Univesity. Alice has a profile.people
record on OIP and so ACME University has a profile.institution.university
.
How do we tell the world that Alice works for ACME-U? Simple, ACME-U must publish a record of type relationship.employment
saying so.
In order to be thorough, we also built those record types.
Although this record type has the exact same fields as the record type relationship
, we need to specify two roles: employer and employee.
These roles are records of the type profile.role
that we built as an exact copy of profile
. It simply has a name and a description of what it means.
Let's give an example.
ACME-U wants to tell the world that Alice is now an employee. The first need to explain what they mean by "employer" and by "employee".
First, they publish two records of the type role
: one with the name "employee" and a description "Performs work for someone and get paid for it". Then another one with the name "employer" and description "Tell employees what to do and pay them for the work they have done".
Now, ACME-U makes a relationship.employment
record wherein the field parties
we have [ACME-U, Alice]
and in field roles
we have [employer, employee]
. Now obviously, that OIP will put the transaction ids of the records of ACME-U, Alice, employer, and employee in place before it publishes.
Ok, now that ACME-U publishes the employment, it is like a job offer. To accept the offer, Alice must add the transaction id of that record to the fields validRelationships
in her profile.
Now we all know that ACME-U offered a job to Alice and she took it. If they decide to terminate that relationship, ACME-U makes the record of employment with Alice inactive and Alice to accept the termination takes it out form her validRelationships
The same needs to happen between Alice's lab and Bob so the world knows that Bob is a member
of Alice's lab. Notice that we mentioned Alice's lab, a record of type laboratory
not Alice herself, a record of type people
.
In the record type profile.institution.laboratory
Alice is listed as a PI. Obviously, she has full control of her lab and these things could be interchangeable. In OIP it is not and we will learn about relationship.permission
and its child types later when we approach OIP record ownership. By then we will see how ACME-U can make a record inactive and also all the other things that can be done to an existing record by the owner or by the profile holding the permission to do so.
How will it propagate if the parent uses field numbers as already in chain child types without breaking everything?
Updates of records in OIPd are done using version control, record schemas are records in the chain and their updates are also done using version control.
A change in a parent type won't break the existing child types because it will be a new version and the current child types are still compatible with the older version.
When searching for information, OIPd reads all the records built with all versions of the record type.
So... no breaks.
- talk about the confusion between private keys and ownership of OIP.
- How this is related to permission given to profile but in fact to the private key that owns that record.
- must own all records during merge
- if we are doing the spliting on behalfOf someone else, who owns the splitted records?
- Special type of relationship: introducing
reserved
keyword of protobuff.
- Plain text to be index with ES. Referencing will slow down JOIN searches to be done in the application side.
...
- There should be a field to explain what the record type tries to accomplish.