Avoid examples that are incoherent, or that are too abstract

[jpmckinney]

Users prefer specific examples. We now have many specific examples on deeper pages (e.g. for the Map step).

However, much of the older content still has generic/abstract examples – or no examples.

We should review the documentation to check for examples that need to be added or improved. We can then list the pages as a to-do list in this issue, and close the issue once a sufficient number of pages have been updated.

[jpmckinney]

cc @sabahfromlondon to identify/prioritize the pages on which to add/improve examples.

[odscjen]

@jpmckinney is it ok if I start working on this or shall I leave it for Sabah?

[sabahfromlondon]

@jpmckinney I think I missed this! Sorry.

I'll compile the list this week.

Thanks for flagging @odscjen

[sabahfromlondon]

@jpmckinney I've completed a review of the documentation pages. I also thought it was worth checking the depper pages, even though that was not quite within the scope of this ticker. It didn't take much longer and I have some thoughts / suggestions it might be worth discussing..

Here's the link to the spreadsheet.

Shall we discuss on our call on Wednesday?

cc: @odscjen

[jpmckinney]

Thank you! We discussed and answered some of the questions in the sheet. I'll leave this with analysts to find appropriate examples.

[neelima-j]

The sheet has identified 2 required tasks and 9 potential tasks

Example Required

• Include a real world example of what OCDS looks like in JSON and CSV, below the green box at How does the OCDS work?
• Have real world examples of a release and record from the same dataset. Use the dropdown for the user to toggle between the two as per other pages in the documentation, at How is OCDS data published?

@jpmckinney for the 'Maybe' examples, is the sheet the best place to ask clarifying questions/ have discussions, or would you rather we brought it here as tasks?

[jpmckinney]

@neelima-j I think your first link is meant to be to https://standard.open-contracting.org/latest/en/primer/how/?

For the maybe examples, let's move it here as notifications are easier to track.

[odscjen]

Maybe examples copied in from the spreadsheet. Initial comments also from the spreadsheet, italic comments added by me as I transferred this list to here:

• Awards and contracts - The JSON example is an external link probably becuase it is long - suggestion is to leave as it. There are also tabular examples, which should also be in JSON?
• Electronic catalogs - Could be helpful, but possibly no examples for this?
• Updates and amendments - The examples look fictional, but still very easy to understand. I assume the task is to see if there are any real examples which could be used instead?
• Milestones - The examples look fictional, but still very easy to understand. Perhaps a tender milestones example could be added? Again assume the maybe is around looking for real world examples to use instead here?
• Buyers and suppliers - Example could be in JSON in addition to/instead of tabular. The example is for the Department of Health and Social Care, but UK could also be added to the description.
• Organization references - Perhaps we should change the name of the organisation in the example to a name users will recognise as a possible tenderer and then supplier e.g. Microsoft? Maybe we could add an example of how the JSON looks for multiple organisations and with different roles?
• Record reference - Real world examples are welcome here
• Merging - Real world examples are welcome here
• Identifiers - It's possible that users might want to copy what another publisher has done, so may want to look up the whole dataset i.e. via the registry so it would be good to have real world examples throughout this page

[jpmckinney]

@yolile Which of these would you prioritize for added/improved examples? Are any good enough as-is?

[yolile]

Include a real world example of what OCDS looks like in JSON and CSV, below the green box at How does the OCDS work?

We often hear and need to show partners what OCDS data looks like, so this sounds good. The only thing to consider when preparing the example is to make sure that we state that this is just an example and not all the example fields are required nor does the example contain all the existing fields etc.

Have real world examples of a release and record from the same dataset. Use the dropdown for the user to toggle between the two as per other pages in the documentation, at How is OCDS data published?

For this one, I think we have worked examples at the Merging section and the Change History guidance. Maybe we can just add a sentence saying that an example is available on the Change History page?

Awards and contracts - The JSON example is an external link probably becuase it is long - suggestion is to leave as it. There are also tabular examples, which should also be in JSON?

The tabular examples are tabular on purpose to easily demonstrate the issue and solution. I think we can leave it as it is.

Electronic catalogs - Could be helpful, but possibly no examples for this?

Leave it as it is as there is no special mapping to show

Buyers and suppliers - Example could be in JSON in addition to/instead of tabular.

Agree that the example should be in JSON and not in tabular format.

Identifiers - It's possible that users might want to copy what another publisher has done, so may want to look up the whole dataset i.e. via the registry so it would be good to have real world examples throughout this page

The identifiers section has a lot of things to be improved and it has its own issues already documented.

For all the others, do we really want/need real-world examples? I can see some value in having a real example to follow, but I'm afraid of 1) broken links in the future and 2) the reader following any bad practice that the referenced publisher might have. We could avoid both things by only adding a copy of the real JSON file but I don't see much difference with having fictional examples (other than knowing who else is publishing that in OCDS)

[jpmckinney]

@yolile When this issue was created, we had identified that users find it harder to relate to an example if it is totally fictional, e.g. "Department of Bubble Gum buys 10 blister packs of chewing gum from Happy Smiles Inc."

If our fictional examples are close enough to "real", then they are fine. If there are deficiencies in a publisher's data, we can just correct them in the documentation - we aren't making a promise that the data is identical.

Edit: Another reason for using real data was to avoid creating incoherent examples. We sometimes had such examples in the past, where we wanted to demonstrate a lot of fields at once, but it started to make no sense in reality.

[odscjen]

For the release and record of the same dataset, Ghana publish records which include versioned and compiled releases so they're a good example for this e.g use ocds-uhveoc-680235 from https://www.ghaneps.gov.gh/ocds/services/recordpackage/getrecordpackage/2023/10 which has 2 releases. However the data will need cleaned up, e.g. they're not used parties.ids correctly and we probably don't need to include all of the fields they have. This could replace the fictional example in merging and then be linked to in How is OCDS data published along with Change history as suggested by @yolile in #1449 (comment).

Ebonyi State in Nigeria could be used for the json and csv example, they provide both json and excel downloads, e.g. https://ebonyieprocure.eb.gov.ng/do_award_contract_details.php?id=ocds-zinqhl-014665-EB/WTR/EP002-NG. Again this will need corrected, e.g. id's are not being used correctly, and we'll remove some of the fields so that it's not an overly long example.

I've picked African examples for both of these as:
a) most of the current examples seem to be either from the UK, Moldova, or Latin America
b) they're both publishers that are still publishing OCDS

@jpmckinney @yolile if you're happy with those examples I'll get started on drafting them up taking into consideration the need to make it clear that the actual data may not be identical to the data the publisher has published.

Re. the other points are you happy with:

• leaving the Buyers and suppliers example as fictional as long as it's converted into json? (it seems like a coherent example to me using real companies)
• the fictional example in Milestones left as is as it looks coherent and close to real
• the fictional example in amendements also gets left as is as it looks coherent and close to real
• leaving as is the example used in Organization references as it is a real company (although a small one), I'm not sure I see the need to change it to a large multinational (although I may be biased in this particular case as ODS is the example company!)

I've updated the above comment that lists the maybes to strike out those that @yolile has already confirmed can be left as is

[jpmckinney]

@yolile I defer to you.

[yolile]

All sounds good to me, thanks @odscjen

[duncandewhurst]

I'm following up on @jpmckinney's request in #1661 (comment) for me to see if any of the existing examples mentioned in this issue are deficient, and if so, propose some improvement.

Regarding avoiding totally fictional or incoherent examples (the purpose of this issue), I propose the following improvements:

• Amendments guidance (example 1) and Records reference (record structure) (Done in Make examples minimal #1680)
  • Deficiencies:
    • The buyer (Open Data Services) is not a government agency and appears elsewhere in the documentation as a supplier, which might be confusing to readers.
    • The object of the procurement (a data merging tool) closely relates to the subject of the example (updates and amendments), which might be confusing for readers.
  • Proposed improvements:
    • Replace the buyer with a government agency
    • Replace the object of the procurement with something unrelated to the OCDS releases and records model, e.g. office supplies.
• Records reference (embedded releases) (Done in Make examples minimal #1680)
  • Deficiencies:
    • The supplier is totally fictional (AnyCorp)
    • The example doesn't follow a common thread with the earlier examples on the page
  • Proposed improvements:
    • Replace with an embedded releases version of the example featured earlier in the page (noting the improvements proposed below)

Having reviewed the examples mentioned in this issue, I think there are many other opportunities for improvement. In most cases, with respect to the concepts they are supposed to illustrate, the examples are either unnecessarily complex or include fields which are irrelevant, or both.

My other observation from reviewing the changes in #1661 is that it's more trouble than it's worth to find real OCDS data and to try to correct errors and trim it down for brevity and clarity so that it can serve as example data.

I think that we should:

1. Get 1449 improve examples #1661 merged without adding to its scope
2. Update the contributor guidelines for authoring examples as follows:

• Explain what is meant by 'real' (i.e. featuring real organisations and plausible and coherent field values, not being real data from an OCDS publisher)
• Examples ought to feature the minimum level of complexity and minimum set of fields with respect to the concepts they are supposed to illustrate.
• Examples on the same page ought to follow a common thread/context/scenario.

3. Review and update all examples to conform to the guidelines. (incorporating Make JSON examples minimal #1666 so that we don't need to go through the examples twice).

I'm sharing my notes from reviewing the examples mentioned in this issue so that they aren't lost:

• Awards and contracts guidance (changes example):
  • Deficiencies: Lacks a JSON example to illustrate the text.
  • Proposed improvements: Add a minimal JSON example based on the text. The example will need to include awards.finalStatus, which will be added to the schema via release-schema: Add finalStatus to tender, award and contract. Clarify tender.datePublished. #1648
• Awards and contracts guidance (multiple decisions example):
  • Deficiencies:
    • The example data are tabular rather than JSON, which is inconsistent with most of the OCDS documentation.
    • The link to the real data in the Paraguay API is at risk of link rot.
    • The real data returned by the Paraguay API is likely to be overwhelming for readers because it contains many fields that are irrelevant to the subject of the example.
  • Proposed improvements:
    • Replace tabular examples with minimal JSON examples.
    • Remove the link to the Paraguay API.
• Amendments guidance (example 1) and Records reference (record structure) (Done in Make examples minimal #1680)
  • Deficiencies (further to those mentioned earlier in this comment):
    • The tender amendment release is unnecessarily complex: it amends two fields (tender.value and tender.period), when only one is needed to illustrate how amendments are modelled.
    • The example data contain many fields that are irrelevant to the subject of the example, e.g. tender.status, tender.procurementMethod, tender.awardPeriod etc.
  • Proposed improvements:
    • Update the tender amendment release so that only tender.period is amended
    • Remove irrelevant fields
• Amendments guidance (example 2)
  • Deficiencies:
    • The value of contracts.title ("4501062723") does not conform to the schema
    • The example data contains fields that are irrelevant to the subject of the example, e.g. contracts.items.classification.
  • Proposed improvements:
    • Update contracts.title to conform to the schema
    • Remove irrelevant fields
• Milestones guidance (planning example)
  • Deficiencies: The example data contains fields that are irrelevant to the subject of the example, e.g. planning.budget
  • Proposed improvements: Remove irrelevant fields
• Milestones guidance (project data example):
  • Deficiencies: The example is unnecessarily complex: it contains two milestones and two implementation update releases, when only one of each is required to illustrate how delivery milestones are modelled.
  • Proposed improvements: Update the example to cover only one implementation milestone and one implementation update release, reframe as 'delivery milestones'.
• Milestones guidance (delivery and payment data example)
  • Deficiencies: The example is unnecessarily complex: it covers two delivery and two payment milestones, when it need cover only single payment milestone because delivery milestones are already covered by the previous example.
  • Proposed improvements: Update the example to cover only a single payment milestone, reframe as 'payment milestones'.
• Organization reference guidance:
  • Deficiencies:
    • The tenderer/supplier (Open Data Services) appears elsewhere in the OCDS documentation as a buyer, which might be confusing to readers.
    • The example data contains fields that are irrelevant to the example, e.g. tender.status, awards.value, awards.contractPeriod etc.
    • The worked example is unnecessarily long and verbose given the simplicity of the concepts it concerns.
  • Proposed improvements:
    • Remove the worked example and instead expand the example in the OrganizationReference section of the schema reference to show both an organization reference and the referenced item in the parties array.

Also noting that the amendments guidance twice invites readers to use the Data Review Tool to explore changes in the contracting process. IIRC, the Data Review Tool no longer provides that functionality (OCDS Show) so I think those mentions should be removed.

[jpmckinney]

All sounds good to me – thank you, @duncandewhurst ! You can create an issue about "Update the contributor guidelines" in the handbook.

[jpmckinney]

IIRC, the Data Review Tool no longer provides that functionality (OCDS Show) so I think those mentions should be removed.

It does – we had planned to remove it as part of a full redesign, which hasn't occurred.

[duncandewhurst]

Moved to #1666 (comment)

[jpmckinney]

Moved to #1666 (comment)

[duncandewhurst]

I've updated the issue title to reflect the purpose expressed in #1661 (comment) and I've moved the previous two comments to #1666 (comment) (I'd forgotten I'd opened a separate issue about making examples minimal).

[duncandewhurst]

Whilst reviewing examples for #1666, I noted a couple more overly generic examples, which I'll update as part of #1680:

• Organization classifications - Buyer is 'Ciudad Ficticia'
• Serialization - tenders for 'standard development' and 'documentation authoring'

[jpmckinney]

This issue has some resolutions in #1680 so I won't mark #1661 as closing it.

[jpmckinney]

@duncandewhurst I think the second checklist item is not part of #1680, correct?

Is there anything else left from this issue, otherwise?

[duncandewhurst]

@duncandewhurst I think the second checklist item is not part of #1680, correct?

Whoops, yes. I should've listed it in #1666. Let me know if I should add it to #1680

Is there anything else left from this issue, otherwise?

Nope, I think we're done.

[jpmckinney]

Let me know if I should add it to #1680

Sure, should be quick.

[duncandewhurst]

Done in 6f058dc