Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Some test group has empty list of tests #4

Closed
yanj-github opened this issue Jun 29, 2023 · 6 comments
Closed

Some test group has empty list of tests #4

yanj-github opened this issue Jun 29, 2023 · 6 comments

Comments

@yanj-github
Copy link
Collaborator

Some test groups have 0 number of tests selected. e.g: "2dcontext" on wmas2019
Will this cause issue please?
If we need all test groups have at least a number of tests selected, we can resolve it by adjusting the input parameters.

@yanj-github
Copy link
Collaborator Author

Notes from meeting 19/07/2023: it would be good that at least 1 test is selected for each group.

@yanj-github
Copy link
Collaborator Author

Further information:

wmas2018    
API Name Number of Common Passed Tests Selected Test Count
encrypted-media 0 0
fullscreen 0 0
css/compositing 0 0
css/css-animations 0 0
css/css-cascade 0 0
css/css-color 0 0
css/css-conditional 0 0
css/css-images 0 0
css/css-ui 1 0
css/css-values 1 0
css/css-writing-modes 0 0
css/mediaqueries 0 0
wave_extra 0 0
wmas2019    
API Name Number of Common Passed Tests Selected Test Count
2dcontext 723 0
css/css-namespaces 0 0
css/css-style-attr 0 0
css/mediaqueries 1 0
fullscreen 0 0
manifest 0 0
upgrade-insecure-requests 55 0
wmas2021    
API Name Number of Common Passed Tests Selected Test Count
media-capabilities 0 0

@JohnRiv
Copy link

JohnRiv commented Aug 16, 2023

If there is 0 common passed, it makes sense there are no selected tests. But I do agree it seems if there is at least 1 common passes test, there should be at least 1 selected test for that API.

@yanj-github
Copy link
Collaborator Author

This is blocked by cta-wave/WMAS#81
I am unable to read the common passed result anymore from the cloud hosted version.
It used to be working fine.

@yanj-github
Copy link
Collaborator Author

After fault fix now we have:

WMAS2018     Notes
API Name Number of Common Passed Tests Selected Test Count  
encrypted-media 0 0 0 common passed
fullscreen 0 0 0 common passed
css/compositing 0 0 0 common passed
css/css-animations 0 0 0 common passed
css/css-cascade 0 0 0 common passed
css/css-color 0 0 0 common passed
css/css-conditional 0 0 0 common passed
css/css-images 0 0 0 common passed
css/css-writing-modes 0 0 0 common passed
css/mediaqueries 0 0 0 common passed
wave_extra 0 0 0 common passed
       
WMAS2019      
API Name Number of Common Passed Tests Selected Test Count  
css/css-namespaces 0 0 0 common passed
css/css-style-attr 0 0 0 common passed
css/mediaqueries 1 0   note below
fullscreen 0 0 0 common passed
manifest 0 0 0 common passed
upgrade-insecure-requests 55 0  note below
       
WMAS2021      
API Name Number of Common Passed Tests Selected Test Count  
media-capabilities 0 0 0 common passed

upgrade-insecure-requests: all test in this group was created in 2020 which is after the WMAS2019 Publication cut off data that is why this is all removed, however, atual upgrade-insecure-requests https://w3c.github.io/webappsec-upgrade-insecure-requests/ is still a draft and 13 October 2022 is the latest release. If we do #3 we will have some tests selected.

css/mediaqueries: same for this one https://drafts.csswg.org/mediaqueries/

@yanj-github
Copy link
Collaborator Author

Now only WMAS2019 css/mediaqueries and upgrade-insecure-requests has empty test list
Fix of is blocked by issue #3. As this issue (#3) is a future improvement I will close the current issue. Last call for comments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants