Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make conformance test failures more actionable #763

Closed
bobcatfish opened this issue Apr 27, 2018 · 1 comment
Closed

Make conformance test failures more actionable #763

bobcatfish opened this issue Apr 27, 2018 · 1 comment
Assignees
Labels
area/test-and-release It flags unit/e2e/conformance/perf test issues for product features kind/bug Categorizes issue or PR as related to a bug.

Comments

@bobcatfish
Copy link
Contributor

/area test-and-release
/kind bug
/assign @bobcatfish

Expected Behavior

When a conformance test fails, it should be as obvious as possible why the test failed so that folks submitting PRs which result in failing tests can get some idea of what might be going wrong.

Actual Behavior

Failures often look like this:

STEP: The Revision will be updated when it is ready to serve traffic

• Failure [134.472 seconds]
Route
/usr/local/google/home/christiewilson/Code/go/src/github.com/elafros/elafros/test/conformance/route_test.go:156
  Deploying an app with a pre-built container
  /usr/local/google/home/christiewilson/Code/go/src/github.com/elafros/elafros/test/conformance/route_test.go:193
    Creates a route, serves traffic to it, and serves traffic to subsequent revisions [It]
    /usr/local/google/home/christiewilson/Code/go/src/github.com/elafros/elafros/test/conformance/route_test.go:194

    Expected error:
        <*errors.errorString | 0xc42008b6b0>: {
            s: "timed out waiting for the condition",
        }
        timed out waiting for the condition
    not to have occurred

    /usr/local/google/home/christiewilson/Code/go/src/github.com/elafros/elafros/test/conformance/crd_polling.go:78
------------------------------

This means to debug them, folks have to run the tests and alongside run commands such as watch kubectl get revisions -o yaml --namespace=pizzaplanet to see what's going wrong.

Steps to Reproduce the Problem

  1. make a test fail, e.g. by changing the conditions to not match the actual spec, then run the tests

Additional Info

n/a

@google-prow-robot google-prow-robot added area/test-and-release It flags unit/e2e/conformance/perf test issues for product features kind/bug Categorizes issue or PR as related to a bug. labels Apr 27, 2018
@bobcatfish
Copy link
Contributor Author

The verbose logging added by @jessiezcc took care of this, thanks @jessiezcc !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/test-and-release It flags unit/e2e/conformance/perf test issues for product features kind/bug Categorizes issue or PR as related to a bug.
Projects
None yet
Development

No branches or pull requests

2 participants