Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sql: TestGetUserHashedPasswordTimeout failed #62948

Closed
cockroach-teamcity opened this issue Apr 1, 2021 · 8 comments · Fixed by #63010
Closed

sql: TestGetUserHashedPasswordTimeout failed #62948

cockroach-teamcity opened this issue Apr 1, 2021 · 8 comments · Fixed by #63010
Assignees
@yuzefovich
Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot.

Comments

@cockroach-teamcity
Copy link
Member

(sql).TestGetUserHashedPasswordTimeout failed on master@6bc054abae5d7e177666e7cc3bc736bee3a99d8b:

=== RUN   TestGetUserHashedPasswordTimeout
    test_log_scope.go:73: test logs captured to: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestGetUserHashedPasswordTimeout832820028
    test_log_scope.go:74: use -show-logs to present logs inline
-- sanity checks --
-- make ranges unavailable --
-- expect timeout --
    user_test.go:154: expected error during connection, got ERROR: password authentication failed for user foo (SQLSTATE XXUUU)
    panic.go:617: -- test log scope end --
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestGetUserHashedPasswordTimeout832820028
--- FAIL: TestGetUserHashedPasswordTimeout (0.65s)
More

Parameters:

  • TAGS=
  • GOFLAGS=-parallel=4
make stressrace TESTS=TestGetUserHashedPasswordTimeout PKG=./pkg/sql TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues
@cockroach-teamcity cockroach-teamcity added branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. labels Apr 1, 2021
@ajwerner
Copy link
Contributor

ajwerner commented Apr 1, 2021

https://teamcity.cockroachdb.com/viewLog.html?buildId=2839486&buildTypeId=Cockroach_UnitTests is another

@knz
Copy link
Contributor

knz commented Apr 1, 2021

This only started to fail recently, yet I have not looked into this for the past few months.

@rafiss could you perhaps route this to the more appropriate person?

@knz knz removed their assignment Apr 1, 2021
@cockroach-teamcity
Copy link
Member Author

(sql).TestGetUserHashedPasswordTimeout failed on master@1764c5f7c5642cff6dffe27432c90ab3a3484820:

=== RUN   TestGetUserHashedPasswordTimeout
    test_log_scope.go:73: test logs captured to: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestGetUserHashedPasswordTimeout422462952
    test_log_scope.go:74: use -show-logs to present logs inline
-- sanity checks --
-- make ranges unavailable --
-- expect timeout --
    user_test.go:154: expected error during connection, got ERROR: password authentication failed for user foo (SQLSTATE XXUUU)
    panic.go:617: -- test log scope end --
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestGetUserHashedPasswordTimeout422462952
--- FAIL: TestGetUserHashedPasswordTimeout (0.88s)
More

Parameters:

  • GOFLAGS=-json
make stressrace TESTS=TestGetUserHashedPasswordTimeout PKG=./pkg/sql TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@rafiss rafiss self-assigned this Apr 1, 2021
@rafiss
Copy link
Collaborator

rafiss commented Apr 1, 2021 •
edited
Loading

I bisected this failure to #62581

@yuzefovich would you mind seeing if you can reproduce this result, and if so, investigating further? (the test here uses the internal executor to fetch the hashedPassword, which seems to have been affected in some way by that PR).

my bisect was:

git bisect run make stress PKG=./pkg/sql TESTS=TestGetUserHashedPasswordTimeout 'STRESSFLAGS=-stderr -maxruns 500'

@rafiss rafiss assigned yuzefovich and unassigned rafiss Apr 1, 2021
@cockroach-teamcity
Copy link
Member Author

(sql).TestGetUserHashedPasswordTimeout failed on master@a5f21439d2c455fa78028d54db2cccf6d6f6f7a2:

=== RUN   TestGetUserHashedPasswordTimeout
    test_log_scope.go:73: test logs captured to: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestGetUserHashedPasswordTimeout392309448
    test_log_scope.go:74: use -show-logs to present logs inline
-- sanity checks --
-- make ranges unavailable --
-- expect timeout --
    user_test.go:154: expected error during connection, got ERROR: password authentication failed for user foo (SQLSTATE XXUUU)
E210401 21:55:31.040596 978422 kv/kvclient/kvcoord/txn_interceptor_committer.go:452  [-] 1  making txn commit explicit failed for "sql txn" meta={id=81a5fb8f pri=0.00233366 epo=0 ts=1617314130.823352485,0 min=1617314130.823352485,0 seq=6} lock=true stat=STAGING rts=1617314130.823352485,0 wto=false gul=1617314131.323352485,0 ifw=5: result is ambiguous (server shutdown)
    panic.go:617: -- test log scope end --
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestGetUserHashedPasswordTimeout392309448
--- FAIL: TestGetUserHashedPasswordTimeout (0.77s)
More

Parameters:

  • GOFLAGS=-json
make stressrace TESTS=TestGetUserHashedPasswordTimeout PKG=./pkg/sql TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@yuzefovich yuzefovich mentioned this issue Apr 1, 2021
Closed
ajwerner added a commit to ajwerner/cockroach that referenced this issue Apr 2, 2021
@ajwerner
sql: fix race condition in internal executor
010591f 
In the case where the finish() method was called because of a context
cancellation, the client might not have observed the context error and
instead would have assumed that execution completed successfully. The
code now prioritizes the context error if there is one.

Fixes cockroachdb#62948

Release note: None
@ajwerner ajwerner mentioned this issue Apr 2, 2021
Merged
ajwerner added a commit to ajwerner/cockroach that referenced this issue Apr 2, 2021
@ajwerner
sql: clean up ieResultChannel concepts and fix race condition
2d3c4aa 
The async and sync implementations were too close to justify two structs.
Also, the async behavior of not stopping the writer in case the reader
called close wasn't desireable. This commit unifies the implementation.
It also ensures that we propagate context errors in all cases triggered
by the closure of the done channel. It also makes closing the channel
idempotent.

Fixes cockroachdb#62948

Release note: None
yuzefovich pushed a commit to ajwerner/cockroach that referenced this issue Apr 2, 2021
@ajwerner @yuzefovich
sql: clean up ieResultChannel concepts and fix race condition
b606d7d 
The async and sync implementations were too close to justify two structs.
Also, the async behavior of not stopping the writer in case the reader
called close wasn't desireable. This commit unifies the implementation.
It also ensures that we propagate context errors in all cases triggered
by the closure of the done channel. It also makes closing the channel
idempotent.

Fixes cockroachdb#62948

Release note: None
@yuzefovich yuzefovich mentioned this issue Apr 2, 2021
Merged
@cockroach-teamcity
Copy link
Member Author

(sql).TestGetUserHashedPasswordTimeout failed on master@df07592e26735be7362ef2b815ee163ead00739e:

=== RUN   TestGetUserHashedPasswordTimeout
    test_log_scope.go:73: test logs captured to: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestGetUserHashedPasswordTimeout418243074
    test_log_scope.go:74: use -show-logs to present logs inline
-- sanity checks --
-- make ranges unavailable --
-- expect timeout --
    user_test.go:154: expected error during connection, got ERROR: password authentication failed for user foo (SQLSTATE XXUUU)
    panic.go:617: -- test log scope end --
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestGetUserHashedPasswordTimeout418243074
--- FAIL: TestGetUserHashedPasswordTimeout (0.96s)
More

Parameters:

  • GOFLAGS=-json
make stressrace TESTS=TestGetUserHashedPasswordTimeout PKG=./pkg/sql TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(sql).TestGetUserHashedPasswordTimeout failed on master@df07592e26735be7362ef2b815ee163ead00739e:

=== RUN   TestGetUserHashedPasswordTimeout
    test_log_scope.go:73: test logs captured to: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestGetUserHashedPasswordTimeout768325144
    test_log_scope.go:74: use -show-logs to present logs inline
-- sanity checks --
-- make ranges unavailable --
-- expect timeout --
    user_test.go:154: expected error during connection, got ERROR: password authentication failed for user foo (SQLSTATE XXUUU)
    panic.go:617: -- test log scope end --
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestGetUserHashedPasswordTimeout768325144
--- FAIL: TestGetUserHashedPasswordTimeout (0.52s)
More

Parameters:

  • TAGS=
  • GOFLAGS=-parallel=4
make stressrace TESTS=TestGetUserHashedPasswordTimeout PKG=./pkg/sql TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(sql).TestGetUserHashedPasswordTimeout failed on master@df07592e26735be7362ef2b815ee163ead00739e:

=== RUN   TestGetUserHashedPasswordTimeout
    test_log_scope.go:73: test logs captured to: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestGetUserHashedPasswordTimeout036009719
    test_log_scope.go:74: use -show-logs to present logs inline
-- sanity checks --
-- make ranges unavailable --
-- expect timeout --
    user_test.go:154: expected error during connection, got ERROR: password authentication failed for user foo (SQLSTATE XXUUU)
    panic.go:617: -- test log scope end --
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestGetUserHashedPasswordTimeout036009719
--- FAIL: TestGetUserHashedPasswordTimeout (0.91s)
More

Parameters:

  • GOFLAGS=-json
make stressrace TESTS=TestGetUserHashedPasswordTimeout PKG=./pkg/sql TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

craig bot pushed a commit that referenced this issue Apr 3, 2021
@yuzefovich
Merge #63047
6e5dfc4 
63047: sql: temporarily skip flaky test r=yuzefovich a=yuzefovich

Addresses: #62948.

Release note: None

Co-authored-by: Yahor Yuzefovich <[email protected]>
@craig craig bot closed this as completed in 651184b Apr 7, 2021
@yuzefovich yuzefovich mentioned this issue Apr 9, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@yuzefovich yuzefovich

Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot.
Projects
None yet
Milestone
No milestone
5 participants
@knz @rafiss @ajwerner @yuzefovich @cockroach-teamcity