Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test Case Timeout for CI execution is causing failure for TC-CADMIN(MF) test cases #20838

Closed
kvikrambhat opened this issue Jul 16, 2022 · 6 comments
Closed

Test Case Timeout for CI execution is causing failure for TC-CADMIN(MF) test cases #20838

kvikrambhat opened this issue Jul 16, 2022 · 6 comments

Comments

@kvikrambhat
Copy link
Contributor

Problem

Test case timeout of 120 seconds specified for running tests on CI is cashing TC-CADMIN(MF) test cases to crash.

Proposed Solution

Increase timeout to 200 seconds to accommodate TC-CADMIN(MF) test cases

image
manjunath-grl added a commit to manjunath-grl/connectedhomeip that referenced this issue Jul 16, 2022
@manjunath-grl
Updating CI test case timeouts to avoid MF test error. [Issue project…
6e48815 
…-chip#20838]
kvikrambhat pushed a commit to manjunath-grl/connectedhomeip that referenced this issue Jul 16, 2022
@manjunath-grl @kvikrambhat
Updating CI test case timeouts to avoid MF test error. [Issue project…
7695a97 
…-chip#20838]
woody-apple added a commit that referenced this issue Jul 17, 2022
@manjunath-grl @restyled-commits @woody-apple
Manual and modified script update (#20821)
aa9457e 
* Modified tests
TC-APPLAUNCHER-1.3
TC-APPLAUNCHER-3.5
TC-APPLAUNCHER-3.6
TC-BOOL-2.1
TC-CC-3.3
TC-CHANNEL-1.6
TC-CHANNEL-5.1
TC-CHANNEL-5.2
TC-CHANNEL-5.3
TC-I-1.1
TC-LVL-2.1
TC-LVL-2.2
TC-LVL-3.1
TC-LVL-4.1
TC-LVL-5.1
TC-LVL-6.1
TC-MEDIAINPUT-1.4
TC-MEDIAINPUT-3.10
TC-MEDIAINPUT-3.11
TC-MEDIAINPUT-3.12
TC-MEDIAINPUT-3.13
TC-CADMIN-1.3
TC-CADMIN-1.4
TC-CADMIN-1.5
TC-CADMIN-1.6
TC-CADMIN-1.9
TC-CADMIN-1.10
TC-CADMIN-1.15
TC-OO-2.3
Modifed Manual scripts
TC-CADMIN-1.*
TC-MOD-1.*

* Auto generated files

* Restyled by whitespace

* Restyled by prettier-yaml

* Modified PICS.yaml file

* Moddified CADMIN-1.13 scripts

* auto genarated files

* Modified git files

* Restyled by whitespace

* Updating CI test case timeouts to avoid MF test error. [Issue #20838]

Co-authored-by: Restyled.io <[email protected]>
Co-authored-by: Justin Wood <[email protected]>
@bzbarsky-apple
Copy link
Contributor

@woody-apple @manjunath-grl @kvikrambhat The point is, tests that take longer than about 2 minutes to run should not be running in CI. They should be marked as "manual" tests. The whole point here was to avoid people adding new very laggy tests to CI.

Why was the timeout raised in #20821?

@bzbarsky-apple
Copy link
Contributor

We should not be raising this timeout. Multi-minute tests should not be running automatically in CI.

@kvikrambhat
Copy link
Contributor Author

kvikrambhat commented Jul 19, 2022 •
edited
Loading

@bzbarsky-apple We took the decided to increase the timeout to 3 minutes to accommodate the MF test scripts, We took the call as we had a few PRs lined up for SVE, we will disable these test and reduce the timeout in the next PR.
CC : @raju-apple

@raju-apple
Copy link
Contributor

@kvikrambhat Lets move that test case to manual list for CI. It will still be available for execution in TH since all the test steps are not disabled.

@bzbarsky-apple
Copy link
Contributor

we will disable these test and reduce the timeout in the next PR

I already did that in #20870

@kvikrambhat
Copy link
Contributor Author

Thanks you,will make sure we dont increase it going forward.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@kvikrambhat @bzbarsky-apple @raju-apple