Find specs: get specs from validate-repos

[tidoust]

W3C/WICG repos now include a w3c.json file that could be used, filtering on w3c.report-type to detect new rec-track specs: https://w3c.github.io/validate-repos/report.json

One issue is that the report includes specs that are neither CSS or JS spec and that we may not want to add to the crawl as-is.

[foolip]

Lots of projects have a need for a spec list as input. Are you thinking that validate-repos would be the source of truth for lists to scrape, or just an input to add spec to the list here?

If a spec list was maintained elsewhere, what would be your requirements for depending on it? It seems like at least knowing whether there's IDL in the spec ahead of time, since reffy now does?

[tidoust]

I was more thinking of using it as an input to add the spec to the list, but I would certainly like to have a source of truth somewhere!

I'd say Reffy's requirements for such a list would be:

1. it contains URLs of all the specifications that define CSS/IDL content and that are developed in some kind of "official" group (right now, that means WHATWG, W3C CGs, W3C WGs and the Khronos Group)
2. it can be fetched or it exists as an NPM package
3. it gets updated when new specs get created
4. there is a way to get to the Editor's Draft from the URL in the list and to the published version on /TR/ when it exists (In practice, Reffy starts from /TR/ URLs but as long as that URL can be reconstructed, I don't really care what the source URL is)
5. at least in the beginning, it does not contain too many other types of specifications, or there is a way to identify specs that neither contain CSS or IDL. This may not be a real issue, I'm only adding that because we've never really run Reffy on other types of specs until now, but having such specs could actually be useful down the road for Reffy at least, e.g. to detect cross-spec references hiccups.

Right now, Reffy needs to know whether a particular spec contains CSS and/or IDL, but that's more to set expectations than an absolute requirement. It allows to assert things such as "This spec was supposed to contain IDL content, I could not find any, something is wrong!". Actual processing does not depend on these flags though, so I'm not including that as a requirement.

[foolip]

Thanks @tidoust, that's helpful! Do you know if there's anyone at the W3C working on a cross-SDO spec list like this?

[tidoust]

@dontcallmedom and I have been casually chatting about that for some time now.
@deniak, @jean-gui and @vivienlacourba maintain the W3C API and have, I believe, been discussing extending the list of specs to those not yet published to /TR/.

But I don't think there's anyone at the W3C specifically working on that.

[dontcallmedom]

re cross-SDO spec of list, there is the one maintained for MDN as well - I don't think it specifically aims to be exhaustive, but there is at least a requirement that any spec documented on MDN appears on that list.

[foolip]

I have a need for a web platform spec list (a list of specs that some browser implements or will implement) to compute some metrics on code review, related to w3c/validate-repos#27.

https://github.com/tobie/specref is already a standalone project, but it's a huge list and doesn't have a "matters to browsers" bit.

@dontcallmedom @tidoust what would you think of a standalone spec list that's a curated subset from known sources, which we could use as input to reffy and for my own purposes?

[foolip]

I see a 👍 from @dontcallmedom. If we do this, where should we host it? Maybe tidoust/reffy-specs?

[dontcallmedom]

I'm happy to create a @w3c repo for that - maybe w3c/browser-specs? I think this would be better than tidoust/reffy-specs for the following reasons:

• it's an org account, so has more granular persmission management
• its usefulness is presumably not limited to reffy

[foolip]

w3c/browser-specs SGTM. Do you want to create an empty repo, and we can use issues there to discuss structure?

[dontcallmedom]

https://github.com/w3c/browser-specs

[dontcallmedom]

w3c/browser-specs now includes features to find specs from a variety of sources, and since reffy now uses it as it sources of specification, I believe the issue can be closed