[TSTAT-2-1] [TSTAT-2-2] Setback is not selected in PICS but TH continues to test it

[raju-apple]

From chip-certification-tool created by pankore: CHIP-Specifications/chip-certification-tool#870

Description:

In PICS and zap, we unchecked the Setback related attributes, but TH continues to test for it, causing failure

Logs:

TC-TSTAT-2-1_2023_06_15_16_39_48.log

Additional Info:

v2.8.1 test harness
PICS tool v1.6.6

[raju-apple]

Looking at the the Test_TC_TSTAT_2_1.yaml we have:

-label: "Read OccupiedSetback attribute from the DUT"
      PICS: TSTAT.S.F02
      command: "readAttribute"
      attribute: "OccupiedSetback"
      response:
          constraints:
              type: int8u
              minValue: 0
              maxValue: 255

    - label: "Read OccupiedSetbackMin attribute from the DUT"
      PICS: TSTAT.S.F02
      command: "readAttribute"
      attribute: "OccupiedSetbackMin"
      response:
          constraints:
              type: int8u
              minValue: 0
              maxValue: 255

    - label: "Read OccupiedSetbackMax attribute from the DUT"
      PICS: TSTAT.S.F02
      command: "readAttribute"
      attribute: "OccupiedSetbackMax"
      response:
          constraints:
              type: int8u
              minValue: 0
              maxValue: 255

PICS values (from log):

...
TSTAT.S.F02=1
...

According your log we can see the PICS value TSTAT.S.F02=1 . So these tests steps aren't skipped.
Test steps (from log):

...
CHIPTOOL   | 2023-06-15 08:40:37.919422 | [1686818433.350033][25:33] CHIP:TOO:  ***** Test Step 53 : Read OccupiedSetback attribute from the DUT
...
CHIPTOOL   | 2023-06-15 08:40:39.888556 | [1686818434.031344][25:33] CHIP:TOO:  ***** Test Step 55 : Read OccupiedSetbackMax attribute from the DUT
...
CHIPTOOL   | 2023-06-15 08:40:38.950267 | [1686818433.690637][25:33] CHIP:TOO:  ***** Test Step 54 : Read OccupiedSetbackMin attribute from the DUT
...

[raju-apple]

@hiltonlima thanks for checking. However, in the specs, the conformance for OccupiedSetback, OccupiedSetbackMin and OccupiedSetbackMax is SB (setback), which is TSTAT.S.F04, not TSTAT.S.F02. So in my opinion, these steps should be skipped.

[raju-apple]

@pankore Fixed in PR 27011 please verify and close the ticket.

[fabiowmm]

@pankore is not in the org.