-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Fix] Possible fix for matter-test-scripts issue #227: Remove PICS from OPSTATE tests #34290
base: master
Are you sure you want to change the base?
[Fix] Possible fix for matter-test-scripts issue #227: Remove PICS from OPSTATE tests #34290
Conversation
j-ororke
commented
Jul 10, 2024
- Removed PICS checks in TC_OpstateCommon.py module and replaced with attribute and command checks from connected endpoint during tests.
- Removed PICS checks and replaced with attribute and command checks from endpoint during tests.
PR #34290: Size comparison from 3c96d5b to d10612e Full report (11 builds for cc32xx, mbed, nrfconnect, qpg, stm32, tizen)
|
- Removed some variables that were no longer needed
PR #34290: Size comparison from 3c96d5b to 832661a Full report (85 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, mbed, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
- Removed automatable PICS checks and replaced with attributes available to be gathered from endpoint.
PR #34290: Size comparison from 3c96d5b to 53f6afb Full report (85 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, mbed, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
- Adding back in missing time import and test runner comments into test module
- Replaced input() with wait_for_user_input() in test script - Added back in short sleep to script, not sure why it got removed.
PR #34290: Size comparison from 3c96d5b to e30aefa Full report (3 builds for cc32xx, stm32)
|
PR #34290: Size comparison from 3c96d5b to 084a115 Increases above 0.2%:
Full report (85 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, mbed, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
- Had to remove "test_step" variable that was attempting to be called in wait_for_user_input() as it was not being defined earlier in the test module.
PR #34290: Size comparison from 3c96d5b to 3c2ba60 Full report (11 builds for cc32xx, mbed, nrfconnect, qpg, stm32, tizen)
|
- Minor change to remove unneeded f-string from wait_for_user_input().
PR #34290: Size comparison from 3c96d5b to ec31380 Increases above 0.2%:
Full report (85 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, mbed, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
PR #34290: Size comparison from 8750c55 to 1f8af58 Increases above 0.2%:
Full report (88 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
- Resolved linting errors
PR #34290: Size comparison from 941c1d2 to a441b26 Full report (19 builds for cc13x4_26x4, cc32xx, nrfconnect, nxp, qpg, stm32, tizen)
|
PR #34290: Size comparison from 941c1d2 to 6b61a00 Full report (88 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
PR #34290: Size comparison from 9c6c365 to fd714b8 Full report (88 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
PR #34290: Size comparison from 9c6c365 to f090844 Full report (88 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
- Debugging to find issue why test is failing in CI pipeline.
PR #34290: Size comparison from 9c6c365 to 78f6e4e Full report (88 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
- Continuing effort to resolve issue with CI pipeline
PR #34290: Size comparison from 9c6c365 to b972df9 Full report (88 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
… remove_PICS_from_OPSTATE_tests
PR #34290: Size comparison from 5c0c92e to 354f40a Full report (67 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
- Resolved issues with attributes_guard function in matter_testing support module
- changed verbosity in CI arguments to make it quieter.
PR #34290: Size comparison from 5c0c92e to bb54582 Increases above 0.2%:
Full report (67 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
- Updated attributes_guard function to make it async
PR #34290: Size comparison from 5c0c92e to 28f3676 Increases above 0.2%:
Full report (67 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
@@ -94,15 +94,16 @@ async def test_TC_RVCOPSTATE_2_1(self): | |||
asserts.fail("The --app-pid flag must be set when PICS_SDK_CI_ONLY is set") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can't comment up there, but having the endpoint be set is actually pretty important for this test because endpoint == None defaults to EP0, which is not what we want here. Can you add an assertion below that line that endpoint is actually set?
if self.attribute_guard(condition2_needs_to_be_false_to_skip_step): | ||
# skip step 2 if condition not met | ||
""" | ||
attr_condition = await asyncio.wait_for(should_run_test_on_endpoint(self, has_attribute(attribute_condition)), timeout=60) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The should_run_test_on_endpoint function is going to do a wildcard read on every call.
Given that we're moving now towards doing a lot of checks based on the device conformance, I think it would make sense to move that wildcard read up to the setup_class function and have it stored as self.wildcard and use that. Then you can just call has_attribute directly here (and swap over the should_run... to use the stored wildcard)