CalmHub: Store Architecture Decision Records (ADR)

[jpgough-ms]

Feature Request

Description of Problem:

Whilst centralising the store of patterns and architectures, it also makes sense to centralise the store of architecture decision records that accompany these assets.

Potential Solutions:

• Pick a JSON structure to store ADRs
• Create a RESTful API for managing ADRs and changing their status

[jpgough-ms]

Proposal for what the API could look like:

# Retrieve all ADRs in a namespace
GET /calm/namespaces/{namespace}/adrs
- Fetch all ADRs within a specific namespace.

# Create a new ADR in draft state
POST /calm/namespaces/{namespace}/adrs
- Create a new ADR in the draft state. Returns an ADR id.

# Update ADR state to proposed
PUT /calm/namespaces/{namespace}/adrs/{id}/propose
- Transition the ADR to the "Propose" state.

# Update ADR state to accepted
PUT /calm/namespaces/{namespace}/adrs/{id}/accept
- Transition the ADR to the "Accepted" state.

# Mark ADR as superseded
PUT /calm/namespaces/{namespace}/adrs/{id}/supersede
- Mark an ADR as superseded by another ADR. Accepts the id of the superseding ADR in the request body.

# Mark ADR as outdated
PUT /calm/namespaces/{namespace}/adrs/{id}/outdate
- Mark an ADR as outdated.

# Reject ADR
PUT /calm/namespaces/{namespace}/adrs/{id}/reject
- Transition the ADR to the "Rejected" state.

[jpgough-ms]

I'd propose that the body content of these is the MADR specification for ADRs in JSON: https://adr.github.io/madr/

[yt-ms]

What is the difference between outdate and supersede?

[yt-ms]

I'd propose that the body content of these is the MADR specification for ADRs in JSON: https://adr.github.io/madr/

JSON to make it schema-verifiable? Markdown would be more human-friendly.

Also need to think about extensibility, as an organisation may want to add additional metadata, and potentially additional sections.

[yt-ms]

Presumably you'll also need GET /calm/namespaces/{namespace}/adrs/{id}...

[yt-ms]

What metadata would we expect to be returned for all ADRs? E.g. created by, last updated data, etc.

[yt-ms]

Do we need to support querying the version history?

GET /calm/namespaces/{namespace}/adrs/{id}/history (returns a list of versionIDs with some other info about each)
and
GET /calm/namespaces/{namespace}/adrs/{id}/history/{versionID}

[jpgough-ms]

What is the difference between outdate and supersede?

@yt-ms from my perspective, outdated means that there has been a change in circumstances/architecture/direction meaning that this ADR is no longer relevant. Superseded is a form of outdated, but there is now a specific ADR that should be used instead. Some ADRs are outdated and out of the control of the team.

[jpgough-ms]

What metadata would we expect to be returned for all ADRs? E.g. created by, last updated data, etc.

I'll have a go at defining a sample JSON and adding it to this issue. I'll make sure to incorporate room for additional metadata. I'll also consider the history aspect.

[yt-ms]

What is the difference between outdate and supersede?

@yt-ms from my perspective, outdated means that there has been a change in circumstances/architecture/direction meaning that this ADR is no longer relevant. Superseded is a form of outdated, but there is now a specific ADR that should be used instead. Some ADRs are outdated and out of the control of the team.

OK, but if the current ADR is now wrong (even if not due to this team) then people need to know what they should be doing instead, so that feels like it's still superseded to me.

Having said that, the MADR docs have deprecated as a status, so we should probably align with that instead.

[jpgough-ms]

Revised API Paths:

# Retrieve all ADRs in a namespace
GET /calm/namespaces/{namespace}/adrs
- Fetch all ADRs within a specific namespace.

# Retrieve a specific ADR by ID
GET /calm/namespaces/{namespace}/adrs/{id}
- Fetch details of a specific ADR by its ID.

# Retrieve the version history of an ADR
GET /calm/namespaces/{namespace}/adrs/{id}/history
- Fetch the version history of a specific ADR.
- Returns a list of version IDs along with metadata for each version (e.g., timestamp, status, updatedBy).

# Retrieve a specific version of an ADR
GET /calm/namespaces/{namespace}/adrs/{id}/history/{versionID}
- Fetch the details of a specific version of an ADR.

# Create a new ADR in draft state
POST /calm/namespaces/{namespace}/adrs
- Create a new ADR in the draft state. Returns an ADR ID.

# Update ADR state to proposed
PUT /calm/namespaces/{namespace}/adrs/{id}/propose
- Transition the ADR to the "Propose" state.

# Update ADR state to accepted
PUT /calm/namespaces/{namespace}/adrs/{id}/accept
- Transition the ADR to the "Accepted" state.

# Mark ADR as superseded
PUT /calm/namespaces/{namespace}/adrs/{id}/supersede
- Mark an ADR as superseded by another ADR. Accepts the ID of the superseding ADR in the request body.

# Mark ADR as deprecated
PUT /calm/namespaces/{namespace}/adrs/{id}/deprecate
- Mark an ADR as deprecated.

# Reject ADR
PUT /calm/namespaces/{namespace}/adrs/{id}/reject
- Transition the ADR to the "Rejected" state.

JSON Schema for content of ADR returned:

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "title": "MADR Schema",
  "type": "object",
  "properties": {
    "title": {
      "type": "string",
      "description": "Title of the ADR"
    },
    "status": {
      "type": "string",
      "enum": ["Draft", "Proposed", "Accepted", "Superseded", "Rejected", "Outdated"],
      "description": "Current state of the ADR"
    },
    "date": {
      "type": "string",
      "format": "date",
      "description": "Date when the ADR was created"
    },
    "contextAndProblemStatement": {
      "type": "string",
      "description": "Description of the problem or context driving the decision"
    },
    "decisionDrivers": {
      "type": "array",
      "items": {
        "type": "string"
      },
      "description": "List of decision drivers"
    },
    "consideredOptions": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "name": {
            "type": "string",
            "description": "Name of the option"
          },
          "description": {
            "type": "string",
            "description": "Description of the option"
          },
          "positiveConsequences": {
            "type": "array",
            "items": {
              "type": "string"
            },
            "description": "List of positive consequences"
          },
          "negativeConsequences": {
            "type": "array",
            "items": {
              "type": "string"
            },
            "description": "List of negative consequences"
          }
        },
        "required": ["name", "description"]
      },
      "description": "List of options considered"
    },
    "decisionOutcome": {
      "type": "object",
      "properties": {
        "chosenOption": {
          "type": "string",
          "description": "Chosen option"
        },
        "rationale": {
          "type": "string",
          "description": "Rationale for the decision"
        },
        "positiveConsequences": {
          "type": "array",
          "items": {
            "type": "string"
          },
          "description": "List of positive consequences"
        },
        "negativeConsequences": {
          "type": "array",
          "items": {
            "type": "string"
          },
          "description": "List of negative consequences"
        }
      },
      "required": ["chosenOption", "rationale"],
      "description": "Outcome of the decision"
    },
    "links": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "rel": {
            "type": "string",
            "description": "Relationship type (e.g., 'reference', 'related')"
          },
          "href": {
            "type": "string",
            "format": "uri",
            "description": "URL or path"
          }
        },
        "required": ["rel", "href"]
      },
      "description": "Links to related resources"
    },
    "metadata": {
      "type": "object",
      "description": "Custom metadata for organizational use",
      "additionalProperties": {
        "anyOf": [
          { "type": "string" },
          { "type": "array", "items": { "type": "string" } },
          { "type": "object" }
        ]
      }
    }
  },
  "required": ["title", "status", "date", "contextAndProblemStatement", "decisionDrivers", "consideredOptions", "decisionOutcome"],
  "additionalProperties": false
}

[yt-ms]

JSON schema has "outdated" rather than "deprecated".

[yt-ms]

Can we give "date" a more descriptive name? Is the creation date useful once it's been approved?

[yt-ms]

"positiveConsequences" and "negativeConsequences" is confusing two different things - the pros/cons of each option, and the consequences of the decision. Can we rename to be "pros" and "cons" maybe?

[yt-ms]

Can you get JSON schema to validate that the content of a field is valid Markdown? ;)

[grahampacker-ms]

Could the state changes be more generically defined. For example:

PUT /calm/namespaces/{namespace}/adrs/{id}/status/{status}
- Transition the ADR to the requested state, if possible.

This could return a descriptive error if an invalid transition is attempted.

[grahampacker-ms]

Should it be possible to edit an ADR in a draft state?

[grahampacker-ms]

Definition of the initial version I am going to work on.

Some assumptions I'm going to make:

• We aren't using semver for ADRs. I think of it more of a revision number. I will use sequential numbering.
• Status changes create new versions with the updated status
• The ADRs will be in JSON format as described above (comment here)
• No validation of the ADR schema will be done, this will be the responsibility of the CalmCLI, similar to what we've discussed for patterns and architectures

# Retrieve all ADRs in a namespace
GET /calm/namespaces/{namespace}/adrs
- Fetch all ADRs within a specific namespace.

# Retrieve a specific ADR by ID
GET /calm/namespaces/{namespace}/adrs/{id}
- Fetch details of the latest version of a specific ADR by its ID.

# Retrieve the version history of an ADR
GET /calm/namespaces/{namespace}/adrs/{id}/history
- Fetch the version history of a specific ADR.
- Returns a list of version IDs along with metadata for each version (e.g., timestamp, status, updatedBy).

# Retrieve a specific version of an ADR
GET /calm/namespaces/{namespace}/adrs/{id}/history/{versionID}
- Fetch the details of a specific version of an ADR.

# Create a new ADR in draft state
POST /calm/namespaces/{namespace}/adrs
- Create a new ADR in the draft state. Returns an ADR ID.

# Add a new version of an ADR
POST /calm/namespaces/{namespace}/adrs/{id}
- Create a new version of an ADR. Returns a location to get the ADR.

# Update ADR state to provided status. Creates a new version with the new status.
POST /calm/namespaces/{namespace}/adrs/{id}/status/{status}
- Transition the ADR to the provided state. Functionally a new revision with the new status

[jpgough-ms]

@grahampacker-ms

Status changes create new versions with the updated status

I thought you were calling this revision? 😄 So new revision of the same ADR ID?

The ADRs will be in JSON format as described above

Yeah store in JSON - CalmHub UI will render this correctly.

No validation of the ADR schema will be done, this will be the responsibility of the CalmCLI, similar to what we've discussed for patterns and architectures

I think this is where @yt-ms question comes in: what makes sure it is valid markdown? What does the transformation from markdown to JSON? Is that something easily achievable in the CLI? Originally, I'd thought about keeping the same document and updating the status on that single document. The requirement for an audit trail makes that trickier.

I'd also thought that one advantage of recording ADRs, means that the hub is useable without committing to CALM and the CLI straight away. There could be a minimal UI for creating and viewing ADRs that doesn't use the CLI, but that would mean understanding the ADR model on the server side. The thought process here is that teams that are interested in CALM will already be using ADRs, and this adds immediate lower level barrier to entry. I think we should strongly enforce the ADR model, unlike with our other exchanges with the CLI.

[grahampacker-ms]

Instead of using the version terminology and "consider it a revision" I'll use the revision terminology to be explicit.

I'll stub out validation of ADRs for now, we can discuss on WG/Office Hours to ensure we're deliberate in where we perform validations to avoid duplication of logic and also agree how strict to be. (For example, should we ensure every field is valid markdown, or render it as is)

We will also need to consider how to protect from XSS attacks as HTML can be embedded in markdown. I'll raise this as a separate task.