Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gubernator sometimes fail to report failed tests #123

Closed
adrcunha opened this issue Sep 17, 2018 · 8 comments
Closed

gubernator sometimes fail to report failed tests #123

adrcunha opened this issue Sep 17, 2018 · 8 comments
Assignees

Comments

@adrcunha
Copy link
Contributor

e.g., in https://gubernator.knative.dev/build/knative-prow/logs/ci-knative-serving-continuous/1039348282343559168 the e2e tests failed but were not reported

@adrcunha adrcunha self-assigned this Sep 17, 2018
@adrcunha
Copy link
Contributor Author

#118 should solve it, or at least provide a hint on why the report on e2e tests was skipped.

@adrcunha adrcunha changed the title gubernator sometimes fail to reported failed E2E tests gubernator sometimes fail to report failed E2E tests Sep 17, 2018
@adrcunha
Copy link
Contributor Author

Once knative/serving#2042 is merged it will possible to check in knative/serving if this is really solved.

@adrcunha
Copy link
Contributor Author

Checked the CI flow for knative/build, and didn't find any missing results anymore.

@adrcunha
Copy link
Contributor Author

adrcunha commented Sep 26, 2018

Another instance where the results were not parsed correctly:

https://gubernator.knative.dev/build/knative-prow/pr-logs/pull/knative_serving/2063/pull-knative-serving-integration-tests/1045017388094853122/?log#log

I0926 18:47:04.525] FAIL	github.com/knative/serving/test/e2e	518.582s
W0926 18:47:04.626] mv: missing destination file operand after 'knative/serving/test/e2e'
W0926 18:47:04.626] Try 'mv --help' for more information.

@adrcunha
Copy link
Contributor Author

adrcunha commented Oct 1, 2018

Another example of missing failure:

https://storage.googleapis.com/knative-prow/pr-logs/pull/knative_serving/2120/pull-knative-serving-unit-tests/1046832725564067840/build-log.txt

I1001 18:53:55.385] - TestBreakerNoOverload :PASS:
I1001 18:53:55.467] - FAIL github.com/knative/serving/pkg/queue
I1001 18:53:55.478] - TestStoreBadConstructors :PASS:
...
I1001 18:53:57.690] === RUN   TestBreakerRecover
I1001 18:53:57.690] SIGQUIT: quit
...
I1001 18:53:57.699] *** Test killed with quit: ran too long (10m0s).
I1001 18:53:57.699] FAIL	github.com/knative/serving/pkg/queue	600.167s
...

The parser doesn't recognize SIGQUIT.

@adrcunha adrcunha changed the title gubernator sometimes fail to report failed E2E tests gubernator sometimes fail to report failed tests Oct 1, 2018
knative-prow-robot pushed a commit that referenced this issue Oct 11, 2018
Address #123.

Bonus: add tests for `report_go_test()`.
@adrcunha
Copy link
Contributor Author

In https://gubernator.knative.dev/build/knative-prow/logs/ci-knative-serving-continuous/1052590539603447808:

I1017 16:19:11.334] - FAIL github.com/knative/serving/test/e2e
I1017 16:19:11.338] *** INTERNAL ERROR: missing tests for knative/serving/test/e2e, got []
...
I1017 16:19:11.377] fatal	TestAutoscaleUpDownUp	e2e/autoscale_test.go:219	Unable to observe the Deployment named prodjetisbnn-00001-deployment scaling
...

Looks like the parser missed the fatal entry.

@adrcunha
Copy link
Contributor Author

The error above happened before library.sh was updated to the latest version, which had a few fixes for handling the fatal case. Also, the autoscale tests were updated as well. Since then there's no evidence in Testgrid that the parser is still missing failures.

@adrcunha
Copy link
Contributor Author

Considering this fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant