Skip to content

Commit

Permalink
API docs about OData Dataset Service (#788)
Browse files Browse the repository at this point in the history 
* API docs about OData Dataset Service

* Added more details to select/filtering

* Docs update in response to comments
  • Loading branch information
@ktuite
ktuite authored Mar 6, 2023
1 parent 7b7deff commit da0926d
Showing 1 changed file with 217 additions and 3 deletions.
220 changes: 217 additions & 3 deletions docs/api.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,9 @@ Here major and breaking changes to the API are listed by version.

### ODK Central v2023.2

**Added**:
- New [OData Dataset Service](#reference/odata-endpoints/odata-dataset-service) for each `Dataset` that provides a list of `Entities`.

**Changed**:
- The response of `GET`, `POST`, `PUT` and `PATCH` methods of [Submissions](#reference/submissions/listing-all-submissions-on-a-form) endpoint has been updated to include metadata of the `currentVersion` of the Submission.

Expand Down Expand Up @@ -3428,15 +3431,15 @@ While OData itself supports data of any sort of structure, Power BI and Tableau

In general, the OData standard protocol consists of three API endpoints:

* The **Service Document** describes the available resources in the service. We provide one of these for every `Form` in the system. In our case, these are the tables we derive from the `repeat`s in the given Form. The root table is always named `Submissions`.
* The **Service Document** describes the available resources in the service. We provide one of these for every `Form` in the system. As of version 2023.2, we also provide one for every `Dataset`.
* The **Metadata Document** is a formal XML-based EDMX schema description of every data object we might return. It is linked in every OData response.
* The actual data documents, linked from the Service Document, are a simple JSON representation of the submission data, conforming to the schema we describe in our Metadata Document.
* The actual data documents, linked from the Service Document, are a simple JSON representation of the submission data or entity, conforming to the schema we describe in our Metadata Document.

As our focus is on the bulk-export of data from ODK Central so that more advanced analysis tools can handle the data themselves, we do not support most of the features at the Intermediate and above conformance levels, like `$sort` or `$filter`.

## OData Form Service [/v1/projects/{projectId}/forms/{xmlFormId}.svc]

ODK Central presents one OData service for every `Form` it knows about. Each service might have multiple tables related to that Form. To access the OData service, simply add `.svc` to the resource URL for the given Form.
ODK Central presents one OData service for every `Form` it knows about. To access the OData service, simply add `.svc` to the resource URL for the given Form.

+ Parameters
+ projectId: `7` (number, required) - The numeric ID of the Project
Expand Down Expand Up @@ -3658,6 +3661,217 @@ Because this `/#/dl` path returns a web page that causes a file download rather

(html markup data)


## OData Dataset Service [/v1/projects/{projectId}/datasets/{name}.svc]

ODK Central presents one OData service for every `Dataset` as a way to get an OData feed of `Entities`. To access the OData service, simply add `.svc` to the resource URL for the given Dataset.

+ Parameters
+ projectId: `7` (number, required) - The numeric ID of the Project

+ `name`: `trees` (string, required) - The `name` of the `Dataset` whose OData service you wish to access.

### Service Document [GET]

The Service Document provides a link to the main source of information in this OData service: the list of `Entities` in this `Dataset`, as well as the Metadata Document describing the schema of this information.

This document is available only in JSON format.

+ Response 200 (application/json; charset=utf-8; odata.metadata=minimal)
+ Body

{
"@odata.context": "https://your.odk.server/v1/projects/7/datasets/trees.svc/$metadata",
"value": [
{
"kind": "EntitySet",
"name": "Entities",
"url": "Entities"
},
]
}

+ Response 403 (application/json)
+ Attributes (Error 403)

+ Response 406 (application/json)
+ Attributes (Error 406)

### Metadata Document [GET /v1/projects/{projectId}/datasets/{name}.svc/$metadata]

The Metadata Document describes, in [EDMX CSDL](http://docs.oasis-open.org/odata/odata-csdl-xml/v4.01/odata-csdl-xml-v4.01.html), the schema of all the data you can retrieve from the OData Dataset Service in question. Essentially, these are the Dataset properties, or the schema of each `Entity`, translated into the OData format.

+ Response 200 (application/xml)
+ Body

<?xml version="1.0" encoding="UTF-8"?>
<edmx:Edmx xmlns:edmx="http://docs.oasis-open.org/odata/ns/edmx" Version="4.0">
<edmx:DataServices>
<Schema xmlns="http://docs.oasis-open.org/odata/ns/edm" Namespace="org.opendatakit.entity">
<ComplexType Name="metadata">
<Property Name="createdAt" Type="Edm.DateTimeOffset"/>
<Property Name="creatorId" Type="Edm.String"/>
<Property Name="creatorName" Type="Edm.String"/>
</ComplexType>
</Schema>
<Schema xmlns="http://docs.oasis-open.org/odata/ns/edm" Namespace="org.opendatakit.user.trees">
<EntityType Name="Entities">
<Key>
<PropertyRef Name="__id"/>
</Key>
<Property Name="__id" Type="Edm.String"/>
<Property Name="__system" Type="org.opendatakit.entity.metadata"/>
<Property Name="name" Type="Edm.String"/>
<Property Name="label" Type="Edm.String"/>
<Property Name="geometry" Type="Edm.String"/>
<Property Name="species" Type="Edm.String"/>
<Property Name="circumference_cm" Type="Edm.String"/>
</EntityType>
<EntityContainer Name="trees">
<EntitySet Name="Entities" EntityType="org.opendatakit.user.trees.Entities">
<Annotation Term="Org.OData.Capabilities.V1.ConformanceLevel" EnumMember="Org.OData.Capabilities.V1.ConformanceLevelType/Minimal"/>
<Annotation Term="Org.OData.Capabilities.V1.BatchSupported" Bool="false"/>
<Annotation Term="Org.OData.Capabilities.V1.CountRestrictions">
<Record>
<PropertyValue Property="Countable" Bool="true"/>
</Record>
</Annotation>
<Annotation Term="Org.OData.Capabilities.V1.FilterFunctions">
<Record>
<PropertyValue Property="NonCountableProperties">
<Collection>
<String>eq</String>
</Collection>
</PropertyValue>
</Record>
</Annotation>
<Annotation Term="Org.OData.Capabilities.V1.FilterFunctions">
<Record>
<PropertyValue Property="Filterable" Bool="true"/>
<PropertyValue Property="RequiresFilter" Bool="false"/>
<PropertyValue Property="NonFilterableProperties">
<Collection>
<PropertyPath>geometry</PropertyPath>
<PropertyPath>species</PropertyPath>
<PropertyPath>circumference_cm</PropertyPath>
</Collection>
</PropertyValue>
</Record>
</Annotation>
<Annotation Term="Org.OData.Capabilities.V1.SortRestrictions">
<Record>
<PropertyValue Property="Sortable" Bool="false"/>
</Record>
</Annotation>
<Annotation Term="Org.OData.Capabilities.V1.ExpandRestrictions">
<Record>
<PropertyValue Property="Expandable" Bool="false"/>
</Record>
</Annotation>
</EntitySet>
</EntityContainer>
</Schema>
</edmx:DataServices>
</edmx:Edmx>

+ Response 403 (application/json)
+ Attributes (Error 403)

+ Response 406 (application/json)
+ Attributes (Error 406)

### Data Document [GET /v1/projects/{projectId}/datasets/{name}.svc/Entities{?%24skip,%24top,%24count,%24filter,%24select}]

A data document is the straightforward JSON representation of all the `Entities` in a `Dataset`.

The `$top` and `$skip` querystring parameters, specified by OData, apply `limit` and `offset` operations to the data, respectively. The `$count` parameter, also an OData standard, will annotate the response data with the total row count, regardless of the scoping requested by `$top` and `$skip`. While paging is possible through these parameters, it will not greatly improve the performance of exporting data. ODK Central prefers to bulk-export all of its data at once if possible.

The [`$filter` querystring parameter](http://docs.oasis-open.org/odata/odata/v4.01/odata-v4.01-part1-protocol.html#_Toc31358948)can be used to filter by any data field in the system-level schema, but not the Dataset properties. The operators `lt`, `le`, `eq`, `ne`, `ge`, `gt`, `not`, `and`, and `or` are supported. The built-in functions `now`, `year`, `month`, `day`, `hour`, `minute`, `second` are supported.

The fields you can query against are as follows:

| Entity Metadata | OData Field Name |
| ------------------------| -------------------- |
| Entity UUID | `__id` |
| Entity Name (same as UUID) | `name` |
| Entity Label | `label` |
| Entity Creator Actor ID | `__system/creatorId` |
| Entity Timestamp | `__system/createdAt` |

Note that `createdAt` is a time component. This means that any comparisons you make need to account for the full time of the entity. It might seem like `$filter=__system/createdAt le 2020-01-31` would return all results on or before 31 Jan 2020, but in fact only entities made before midnight of that day would be accepted. To include all of the month of January, you need to filter by either `$filter=__system/createdAt le 2020-01-31T23:59:59.999Z` or `$filter=__system/createdAt lt 2020-02-01`. Remember also that you can [query by a specific timezone](https://en.wikipedia.org/wiki/ISO_8601#Time_offsets_from_UTC).

Please see the [OData documentation](http://docs.oasis-open.org/odata/odata/v4.01/odata-v4.01-part1-protocol.html#_Toc31358948) on `$filter` [operations](http://docs.oasis-open.org/odata/odata/v4.01/cs01/part1-protocol/odata-v4.01-cs01-part1-protocol.html#sec_BuiltinFilterOperations) and [functions](http://docs.oasis-open.org/odata/odata/v4.01/cs01/part1-protocol/odata-v4.01-cs01-part1-protocol.html#sec_BuiltinQueryFunctions) for more information.

The [`$select` query parameter](http://docs.oasis-open.org/odata/odata/v4.01/odata-v4.01-part1-protocol.html#_Toc31358942) will return just the fields you specify and is supported on `__id`, `__system`, `__system/creatorId` and `__system/createdAt`, as well as on user defined properties.

As the vast majority of clients only support the JSON OData format, that is the only format ODK Central offers.

+ Parameters
+ `%24skip`: `10` (number, optional) - If supplied, the first `$skip` rows will be omitted from the results.
+ `%24top`: `5` (number, optional) - If supplied, only up to `$top` rows will be returned in the results.
+ `%24count`: `true` (boolean, optional) - If set to `true`, an `@odata.count` property will be added to the result indicating the total number of rows, ignoring the above paging parameters.
+ `%24filter`: `year(__system/createdAt) lt year(now())` (string, optional) - If provided, will filter responses to those matching the query. Only [certain fields](/reference/odata-endpoints/odata-form-service/data-document) are available to reference. The operators `lt`, `le`, `eq`, `neq`, `ge`, `gt`, `not`, `and`, and `or` are supported, and the built-in functions `now`, `year`, `month`, `day`, `hour`, `minute`, `second`.
+ `%24select`: `__id, label, name` (string, optional) - If provided, will return only the selected fields.


+ Response 200 (application/json)
+ Body

{
"@odata.context": "https://your.odk.server/v1/projects/7/datasets/trees.svc/$metadata#Entities"",
"value": [
{
"__id": "0f56bde5-dd05-41f7-8175-c4114eab41c6",
"name": "0f56bde5-dd05-41f7-8175-c4114eab41c6",
"label": "25cm purpleheart",
"__system": {
"createdAt": "2022-12-09T19:41:16.478Z",
"creatorId": "39",
"creatorName": "Tree surveyor"
},
"geometry": "32.7413996 -117.1394617 53.80000305175781 13.933",
"species": "purpleheart",
"circumference_cm": "25"
},
{
"__id": "aeebd746-3b1e-4a24-ba9d-ed6547bd5ff1",
"name": "aeebd746-3b1e-4a24-ba9d-ed6547bd5ff1",
"label": "345cm mora",
"__system": {
"createdAt": "2022-11-21T19:17:36.348Z",
"creatorId": "8",
"creatorName": "[email protected]"
},
"geometry": "47.722581 18.562111 0 0",
"species": "mora",
"circumference_cm": "345"
},
{
"__id": "eacb9844-2f88-48b5-b7c0-6333263fe639",
"name": "eacb9844-2f88-48b5-b7c0-6333263fe639",
"label": "123cm wallaba",
"__system": {
"createdAt": "2022-11-21T18:22:43.759Z",
"creatorId": "8",
"creatorName": "[email protected]"
},
"geometry": "",
"species": "wallaba",
"circumference_cm": "123"
}
]
}

+ Response 403 (application/json)
+ Attributes (Error 403)

+ Response 406 (application/json)
+ Attributes (Error 406)

+ Response 501 (application/json)
+ Attributes (Error 501)


## Draft Testing [/v1/projects/{projectId}/forms/{xmlFormId}/draft.svc]

_(introduced: version 0.8)_
Expand Down

0 comments on commit da0926d

Please sign in to comment.