Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a coverage artifact to web-features releases #2021

Open
ddbeck opened this issue Oct 21, 2024 · 4 comments
Open

Add a coverage artifact to web-features releases #2021

ddbeck opened this issue Oct 21, 2024 · 4 comments
Labels
package:web-features tools and infrastructure Project internal tooling, such as linters, GitHub Actions, or repo settings

Comments

@ddbeck
Copy link
Collaborator

ddbeck commented Oct 21, 2024

We should have an artifact like coverage.json on web-features releases on GitHub that reports:

  • How many BCD keys exist (in scope for web-features, i.e., not webextensions)
  • How many BCD keys have been assigned to a feature

Eventually, we'll cover all of BCD and this will not be interesting, so I don't think we need to add it to the package itself.

Prompted by GoogleChrome/webstatus.dev#793.

@ddbeck ddbeck added tools and infrastructure Project internal tooling, such as linters, GitHub Actions, or repo settings package:web-features labels Oct 21, 2024
@jcscottiii
Copy link

Initially, I assumed this would be a temporary statistic that would be tracked. But I wanted to ask if you think this will be something that will be tracked long term (as more features are added)?

If it is something truly temporary, we can discuss some options like manually hosting a publicly accessible JSON blob. Then the webstatus.dev frontend can request it.

But if it is actually a long term statistic, we could build a coverage.json schema and release it alongside data.json. Then, I would modify my existing process to download and store coverage.json and create an endpoint to serve it.

Also, there are options in between these two approaches too.

@ddbeck
Copy link
Collaborator Author

ddbeck commented Oct 22, 2024

But I wanted to ask if you think this will be something that will be tracked long term (as more features are added)?

I don't think so. My expectation is that we'll (hopefully soon) get to the point where every BCD feature is tagged and new BCD features get tagged at inception (we're looking at ways to assign feature IDs before the feature ships). 100% coverage ought to be the norm.

If it is something truly temporary, we can discuss some options like manually hosting a publicly accessible JSON blob.

Honestly, I expect to reach >95% coverage in November or December (and the last fraction will likely be non-standard and deprecated features). This ought to be very short-lived. 😄

@jcscottiii
Copy link

jcscottiii commented Oct 23, 2024

If it's that short term, you could just tell me a number. Or put it in the release notes text to let everyone know the progress. It doesn't have to be machine readable. Either way, I'll grab it and update the score on the site.

@ddbeck
Copy link
Collaborator Author

ddbeck commented Oct 24, 2024

If it's that short term, you could just tell me a number. Or put it in the release notes text to let everyone know the progress

Can do. https://github.com/web-platform-dx/web-features/releases/tag/v2.2.0 includes the overall percentage in the release notes. I'll start doing this until we get to 100%.

Also, there are finer-grained stats in #788 if you want them, but they're done weekly, not pinned to a specific release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
package:web-features tools and infrastructure Project internal tooling, such as linters, GitHub Actions, or repo settings
Projects
None yet
Development

No branches or pull requests

2 participants