API to either upload log file or instruct to download from copr

[kwk]

Hi,

thank you for working on this interesting project. I'd love to see an API that I can use which lets me:

1. upload a log file manually and annotate it with a given set of line ranges (e.g. [20-34, 100, 242-310]), and the rest of the fields that one needs to enter on the website.
2. instruct the log detective to download the logs for a given copr build id. The annotation is the same as with 1.

Background

We're running nightly snapshots of LLVM on Copr against the moving upstream 1. Sometimes we miss to list a new file in the %files section, or experience issues with the tests being temporarily broken. We've also seen network issues, copr timeouts, RPM dependency issues etc. If a long living upstream PR finally gets merged over night we will notice it the next day. Sometimes a patch needs to be applied only when targeting a special OS (e.g. RHEL vs. Fedora) or architecture. What I'm trying to say is that we have lots of clearly categorizable issues and sometimes we just hit a broken upstream version when our RPM spec files are actually working correctly.

For each build we list the affected OS, arch, project and we also have recently introduced a mechanism to identify an error cause. Here's an example of such an issue for a recent nightly build 2. Expand some of the messages in the main issue comment to see the persisted build log excerpt with the annotation of the type of error cause the automation thinks it is:

Or:

The error cause is identified with regular expressions done over the build log 3. We're going to extend the regular expressions over time and improve it to have lesser "unknown" errors. Also the dependency issue above is yet missing an important piece "No matching package to install: 'moreutils'". Instead the regex is currently focussed on the "Not all dependencies satisfy" part.

Suggestion:
In a semi automated fashion we could provide the log detective with our findings once a human (the assigned snapshot maintainer for the period) has confirmed that the error is what the automation claims to be. Directly from within a github issue we could transfer it over to the log detective. Is there an API one could use?

[nikromen]

hello, we have API for this - frontend uses it for fetching and submitting logs. Every endpoint is documented at log-detective.com/docs but because it's API for internal usage it's only automatically documented and not ideal. Would properly documenting the API with examples and mentioning somewhere that docs page exists solve your issue?

[nikromen]

also if users would use the log-detective's API, I would suggest renaming the /frontend prefix to /api so we don't raise eyebrows if users use these endpoints. WDYT @FrostyX ?

[kwk]

hello, we have API for this - frontend uses it for fetching and submitting logs. Every endpoint is documented at log-detective.com/docs but because it's API for internal usage it's only automatically documented and not ideal. Would properly documenting the API with examples and mentioning somewhere that docs page exists solve your issue?

That API is good enough for me. I think POST /frontend/contribute/copr/{build_id}/{chroot} is the right endpoint for me to use.

What do you mean by "internal"?

Proposal: Draft mode

Maybe this goes to far but when submitting a build to log-detective, is there the possibility to add it as a draft for people to confirm in a final stage? This would help in writing and improving automated producers of logs. A producer can submit it to log-detective and make sure a human approves it before it is added for good.

[FrostyX]

also if users would use the log-detective's API, I would suggest renaming the /frontend prefix to /api so we don't raise eyebrows if users use these endpoints. WDYT @FrostyX ?

Sounds good to me,
our frontend code is basically a web-based client using the backend API 😄. There is probably no reason to distinguish between it and other clients.