Add tests to compare base URL with release/nightly URL #72
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The base URL should either match the release URL if there is one or the nightly URL otherwise. New tests will fail when there are entries in `index.json` for which that is not the case. The first test should also help with identification of situations where a FPWD was published, and is known to Specref but not to us (w3c#62). Note that, as opposed to other tests that only depend on internal code logic, the second test may fail because external sources (Specref in practice) need to be fixed to return the right nightly URL. Fixing the problem at the source may take some time.
There was a problem hiding this comment.
I think that for tests that may fail for external reasons (and thus with undetermined delays before they get fixed), I would prefer us to use the approach we're considering in #71 where they get reported as issues rather than making the build fail.
|
I don't disagree. Let's leave this pull request pending while we figure out how to set things up for #71. |
tidoust
added a commit
to tidoust/browser-specs
that referenced
this pull request
Jun 12, 2020
- Use TR URL for geolocation-sensor - Use TR URL for reporting-1 - Use TR URL for feature-policy-1 - Use TR URL for accname-1.2 - Use TR URL for core-aam-1.2 - Add css-box-4 (on top of -3, which is still "active") - Add user-timing-3 (on top of -2, which is still the "default" for user-timing) - Replace webdriver1 with webdriver2 - Adjust shortname for webxr-ar-module-1 (fixed in W3C API) - Adjust shortname for webxr-gamepads-module-1 (fixed in W3C API)
tidoust
added a commit
to tidoust/browser-specs
that referenced
this pull request
Jun 12, 2020
- Use TR URL for geolocation-sensor - Use TR URL for reporting-1 - Use TR URL for feature-policy-1 - Use TR URL for accname-1.2 - Use TR URL for core-aam-1.2 - Move to W3C URL for badging - Add css-box-4 (on top of -3, which is still "active") - Add user-timing-3 (on top of -2, which is still the "default" for user-timing) - Replace webdriver1 with webdriver2 - Adjust shortname for webxr-ar-module-1 (fixed in W3C API) - Adjust shortname for webxr-gamepads-module-1 (fixed in W3C API)
tidoust
added a commit
that referenced
this pull request
Jun 12, 2020
- Use TR URL for geolocation-sensor - Use TR URL for reporting-1 - Use TR URL for feature-policy-1 - Use TR URL for accname-1.2 - Use TR URL for core-aam-1.2 - Move to W3C URL for badging - Add css-box-4 (on top of -3, which is still "active") - Add user-timing-3 (on top of -2, which is still the "default" for user-timing) - Replace webdriver1 with webdriver2 - Adjust shortname for webxr-ar-module-1 (fixed in W3C API) - Adjust shortname for webxr-gamepads-module-1 (fixed in W3C API)
|
Superseded by #192 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The base URL should either match the release URL if there is one or the nightly URL otherwise. New tests will fail when there are entries in
index.jsonfor which that is not the case.The first test should also help with identification of situations where a FPWD was published, and is known to Specref but not to us (#62).
Note that, as opposed to other tests that only depend on internal code logic, the second test may fail because external sources (Specref in practice) need to be fixed to return the right nightly URL. Fixing the problem at the source may take some time. Making tests fail certainly creates a strong incentive to fix the problem upstream, but means we're going to be stuck in a "some tests fail" situation in the meantime.
Typically, the tests currently choke on:
https://www.w3.org/TR/webxr-ar-module/: the W3C API says that the release URL ishttps://www.w3.org/TR/webxr-ar-module-1/but then the right shortname for the W3C API remainswebxr-ar-module. If we add-1to the base URL, the code won't be able to retrieve the info from the W3C API. Trying to see if this can be fixed in W3C systems.https://www.w3.org/TR/webxr-gamepads-module/: same reasonhttps://drafts.csswg.org/css-backgrounds-4/: Specref returnshttps://drafts.csswg.org/css4-background/for the nightly URL. Filed [biblio.ref] Update URLs of css-backgrounds-4 and css-page-template-1 csswg-drafts#5194 to address this at the sourcehttps://wicg.github.io/badging/: URL needs to be updated tohttps://w3c.github.io/badging/in specs.jsonhttps://wicg.github.io/geolocation-sensor/: URL needs to be updated tohttps://www.w3.org/TR/geolocation-sensor/in specs.jsonhttps://wicg.github.io/frame-timing/: spec was published as a /TR/ Note but moved back to WICG. The URL of the ED URL needs to be updated in W3C systems (request sent).PR should not be merged until these get fixed.