import confluence documents

[makelinux]

yaml example:

document:
  confluence:
    host: https://spaces.redhat.com
    spaces:
      - name: telco5gocp
        pages:
          - title: Debugging an OCP host
            version: 22

Closes #1152
Depends on: instructlab/schema#17

[tiran]

Thank you!

The lint failures are going to be fixed by #1162.

[russellb]

I placed a hold since this should depend on backend support, as well.

As for these changes, I think it would be nice to consider making this an optional dependency, since it would not be part of the most common flow.

[bjhargrave]

There is also the issue of knowledge stability. In the current git-repo-with-knowledge, a commit SHA is provided so the knowledge can be reliably collected at SDG/training time which matches the contribution time. This is important. The example here of using confluence do not seem to have a way to pin the knowledge to a specific revision like git does with a commit SHA. So I think this is a concern for repeatability and safety (since the knowledge could be poisoned after the contribution is made but before the SDG and training are done).

[russellb]

Adding a new knowledge source seems like a good candidate for a design doc in the dev-docs repo to answer some of these key design questions. The addition spans many components, so a project-wide design doc there makes sense to me.

[makelinux]

I've added Confluence versioning.

[leseb]

@makelinux What would a functional test for this look like? Thanks!

[hickeyma]

Thanks @makelinux for pushing this capability. As @russellb mentioned in #1160 (comment), do you mins pushing a design doc for this capability?

[makelinux]

Hi, @russellb , @hickeyma, here is the requested document Confluence document source design notes

[makelinux]

I placed a hold since this should depend on backend support, as well.
@russellb , Confluence pages usually are private, unlike Wikipedia. What do you mean by backend support?

[leseb]

Small nits otherwise this feels ready to go.

[russellb]

I placed a hold since this should depend on backend support, as well.
@russellb , Confluence pages usually are private, unlike Wikipedia. What do you mean by backend support?

I mean if there's an expectation that we would accept any PRs into instructlab/taxonomy with pointers to confluence, then we have some backend code that does weekly model builds out of that repo that would need to understand it. If the intent is that this would only be for private taxonomy forks, then I'd agree only this PR is necessary.

I see a design doc is up now (thank you!) so let's also get that merged first -- instructlab/dev-docs#64

[bjhargrave]

If the intent is that this would only be for private taxonomy forks, then I'd agree only this PR is necessary.

The rub is that ilab command and the public taxonomy repo share the same schema (from the schema repo). So to add this PR in ilab would require a schema update which would also be then in the public taxonomy repo which means people can submit such qna.yaml files which would not fail schema validation. Otherwise we would need to diverge on the schema such that ilab uses an superset schema than the public taxonomy repo. But even this can create disharmony since people could develop a qna.yaml which ilab diff approves of but will fail schema validation in a PR to the public taxonomy repo.

[github-actions]

This pull request has been automatically marked as stale because it has not had activity within 90 days. It will be automatically closed if no further activity occurs within 30 days.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. @makelinux please rebase it. https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork