Skip to content

Commit

Permalink
python testing: Allow top-level PICS with automated selectors
Browse files Browse the repository at this point in the history
When this was originally written the idea was that the TH would
run these tests unconditionally and the test itself would indicate
if it ran. This works well in the CI and we still want to maintain
this mechanism since it will allow us to automate test selection
more easily down the road.

However, the time required to even start a test in the TH is
significant. With the way the test harness is set up currently,
adding unconditional tests adds a significant amount of time
to testing, even if the tests don't run. For now, let's bring
back at least cluster-level gating. We can still gate internally
on cluster elements for the CI. This isn't ideal, but it's where
we are now.
  • Loading branch information
cecille committed Dec 4, 2024
1 parent 5395249 commit e041b81
Show file tree
Hide file tree
Showing 2 changed files with 0 additions and 18 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -2212,8 +2212,6 @@ def run_if_endpoint_matches(accept_function: EndpointCheckFunction):
"""
def run_if_endpoint_matches_internal(body):
def per_endpoint_runner(self: MatterBaseTest, *args, **kwargs):
asserts.assert_false(self.get_test_pics(self.current_test_info.name),
"pics_ method supplied for run_if_endpoint_matches.")
runner_with_timeout = asyncio.wait_for(should_run_test_on_endpoint(self, accept_function), timeout=60)
should_run_test = asyncio.run(runner_with_timeout)
if not should_run_test:
Expand Down
16 changes: 0 additions & 16 deletions src/python_testing/test_testing/TestDecorators.py
Original file line number Diff line number Diff line change
Expand Up @@ -145,16 +145,6 @@ async def test_endpoints(self):
should_run = await should_run_test_on_endpoint(self, has_timesync_utc)
asserts.assert_false(should_run, msg)

# This test should cause an assertion because it has a pics_ method
@run_if_endpoint_matches(has_cluster(Clusters.OnOff))
async def test_endpoint_with_pics(self):
pass

# This method returns the top level pics for test_endpoint_with_pics
# It is used to test that test_endpoint_with_pics will fail since you can't have a per endpoint test gated on a PICS.
def pics_endpoint_with_pics(self):
return ['EXAMPLE.S']

# This test should be run once per endpoint
@run_if_endpoint_matches(has_cluster(Clusters.OnOff))
async def test_endpoint_cluster_yes(self):
Expand Down Expand Up @@ -243,12 +233,6 @@ def main():
if not ok:
failures.append("Test case failure: test_endpoints")

test_name = 'test_endpoint_with_pics'
test_runner.set_test('TestDecorators.py', 'TestDecorators', test_name)
ok = test_runner.run_test_with_mock_read(read_resp, hooks)
if ok:
failures.append(f"Did not get expected test assertion on {test_name}")

# Test should run once for the whole node, regardless of the number of endpoints
def run_check(test_name: str, read_response: Attribute.AsyncReadTransaction.ReadResponse, expect_skip: bool) -> None:
nonlocal failures
Expand Down

0 comments on commit e041b81

Please sign in to comment.