roachtest: pgjdbc failed #39406

cockroach-teamcity · 2019-08-07T16:28:02Z

SHA: https://github.com/cockroachdb/cockroach/commits/a850466dfc4f8eed9e7f758a61f60b120798410f

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=pgjdbc PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1426564&tab=buildLog

The test failed on branch=release-19.1, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190807-1426564/pgjdbc/run_1
	pgjdbc.go:294,pgjdbc.go:305,test_runner.go:691: 
		Tests run on Cockroach v19.1.3-33-ga850466
		Tests run against pgjdbc REL42.2.6
		5786 Total Tests Run
		5768 tests passed
		18 tests failed
		0 tests passed unexpectedly
		4 tests failed unexpectedly
		0 tests expected failed, but not run
		For a full summary look at the pgjdbc artifacts 
		
		An updated blacklist (pgjdbcBlackList19_1) is available in the artifacts' pgjdbc log

The text was updated successfully, but these errors were encountered:

rafiss · 2019-08-07T19:08:01Z

Here is a blacklist we could use for 19.1

var pgjdbcBlackList19_1 = blacklist{
  "org.postgresql.test.jdbc2.ClientEncodingTest.setEncodingAscii[allowEncodingChanges=true]": "unknown",
  "org.postgresql.test.jdbc4.ClientInfoTest.testExplicitSetAppNameNotificationIsParsed": "unknown",
  "org.postgresql.test.jdbc42.GetObject310InfinityTests.test[binary = FORCE, expr = -infinity, pgType = date, klass = class java.time.LocalDate]": "unknown",
  "org.postgresql.test.jdbc42.GetObject310InfinityTests.test[binary = FORCE, expr = -infinity, pgType = timestamp with time zone, klass = class java.time.OffsetDateTime]": "unknown",
  "org.postgresql.test.jdbc42.GetObject310InfinityTests.test[binary = FORCE, expr = -infinity, pgType = timestamp, klass = class java.time.LocalDateTime]": "unknown",
  "org.postgresql.test.jdbc42.GetObject310InfinityTests.test[binary = FORCE, expr = -infinity, pgType = timestamp, klass = class java.time.OffsetDateTime]": "unknown",
  "org.postgresql.test.jdbc42.GetObject310InfinityTests.test[binary = FORCE, expr = infinity, pgType = date, klass = class java.time.LocalDate]": "unknown",
  "org.postgresql.test.jdbc42.GetObject310InfinityTests.test[binary = FORCE, expr = infinity, pgType = timestamp with time zone, klass = class java.time.OffsetDateTime]": "unknown",
  "org.postgresql.test.jdbc42.GetObject310InfinityTests.test[binary = FORCE, expr = infinity, pgType = timestamp, klass = class java.time.LocalDateTime]": "unknown",
  "org.postgresql.test.jdbc42.GetObject310InfinityTests.test[binary = FORCE, expr = infinity, pgType = timestamp, klass = class java.time.OffsetDateTime]": "unknown",
  "org.postgresql.test.jdbc42.GetObject310InfinityTests.test[binary = REGULAR, expr = -infinity, pgType = date, klass = class java.time.LocalDate]": "unknown",
  "org.postgresql.test.jdbc42.GetObject310InfinityTests.test[binary = REGULAR, expr = -infinity, pgType = timestamp with time zone, klass = class java.time.OffsetDateTime]": "unknown",
  "org.postgresql.test.jdbc42.GetObject310InfinityTests.test[binary = REGULAR, expr = -infinity, pgType = timestamp, klass = class java.time.LocalDateTime]": "unknown",
  "org.postgresql.test.jdbc42.GetObject310InfinityTests.test[binary = REGULAR, expr = -infinity, pgType = timestamp, klass = class java.time.OffsetDateTime]": "unknown",
  "org.postgresql.test.jdbc42.GetObject310InfinityTests.test[binary = REGULAR, expr = infinity, pgType = date, klass = class java.time.LocalDate]": "unknown",
  "org.postgresql.test.jdbc42.GetObject310InfinityTests.test[binary = REGULAR, expr = infinity, pgType = timestamp with time zone, klass = class java.time.OffsetDateTime]": "unknown",
  "org.postgresql.test.jdbc42.GetObject310InfinityTests.test[binary = REGULAR, expr = infinity, pgType = timestamp, klass = class java.time.LocalDateTime]": "unknown",
  "org.postgresql.test.jdbc42.GetObject310InfinityTests.test[binary = REGULAR, expr = infinity, pgType = timestamp, klass = class java.time.OffsetDateTime]": "unknown",
}

While looking at this I also realized that we weren't counting failures correctly. JUnit has a concept of tests that "error" in addition to ones that "fail." We were only counting the failed ones as failures. The correct count is 2421 failures out of 5786 total tests.

So in my fix I'll make sure we count everything.

cockroach-teamcity · 2019-08-08T08:38:27Z

SHA: https://github.com/cockroachdb/cockroach/commits/51a6fdedf0ce1d1329d40d801a7deaf8206b6b07

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=pgjdbc PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1428934&tab=buildLog

The test failed on branch=provisional_201908060405_v19.1.4, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190807-1428934/pgjdbc/run_1
	pgjdbc.go:294,pgjdbc.go:305,test_runner.go:691: 
		Tests run on Cockroach v19.1.3-31-g51a6fde
		Tests run against pgjdbc REL42.2.6
		5786 Total Tests Run
		5768 tests passed
		18 tests failed
		0 tests passed unexpectedly
		4 tests failed unexpectedly
		0 tests expected failed, but not run
		For a full summary look at the pgjdbc artifacts 
		
		An updated blacklist (pgjdbcBlackList19_1) is available in the artifacts' pgjdbc log

cockroach-teamcity · 2019-08-08T13:59:00Z

SHA: https://github.com/cockroachdb/cockroach/commits/1051f376924f18c5782887ed824ab5ede5027493

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=pgjdbc PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1429376&tab=buildLog

The test failed on branch=release-2.1, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190808-1429376/pgjdbc/run_1
	pgjdbc.go:294,pgjdbc.go:305,test_runner.go:691: 
		Tests run on Cockroach v2.1.8-6-g1051f37
		Tests run against pgjdbc REL42.2.6
		5786 Total Tests Run
		5783 tests passed
		3 tests failed
		12 tests passed unexpectedly
		1 test failed unexpectedly
		0 tests expected failed, but not run
		For a full summary look at the pgjdbc artifacts 
		
		An updated blacklist (pgjdbcBlackList2_1) is available in the artifacts' pgjdbc log

39375: exec: add null handling to default sorter r=rohany a=rohany Addresses part of #36880. Release note: None 39420: roachtest: fix pgjdbc tests r=rafiss a=rafiss The pgjdbc tests have two types of failures: one referred to as "failures" and the other one as "errors." We were only counting the former before. Now there are many more test failures that should be tracked in the blacklist. touches #39406 Release note: None 39430: workload/ycsb: implement read-modify-write r=jeffrey-xiao a=jeffrey-xiao Release note: None 39461: internal/sqlsmith: add vectorize option r=mjibson a=mjibson This option attempts to limits query generation such that vectorized execution will occur. Still a lot of work to do but this is a good start. Release note: None Co-authored-by: Rohan Yadav <[email protected]> Co-authored-by: Rafi Shamim <[email protected]> Co-authored-by: Jeffrey Xiao <[email protected]> Co-authored-by: Matt Jibson <[email protected]>

cockroach-teamcity · 2019-08-13T08:49:13Z

SHA: https://github.com/cockroachdb/cockroach/commits/51a6fdedf0ce1d1329d40d801a7deaf8206b6b07

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=pgjdbc PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1436116&tab=buildLog

The test failed on branch=provisional_201908060405_v19.1.4, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190812-1436116/pgjdbc/run_1
	pgjdbc.go:306,pgjdbc.go:317,test_runner.go:691: 
		Tests run on Cockroach v19.1.3-31-g51a6fde
		Tests run against pgjdbc REL42.2.6
		5786 Total Tests Run
		3361 tests passed
		2425 tests failed
		0 tests passed unexpectedly
		4 tests failed unexpectedly
		0 tests expected failed, but not run
		For a full summary look at the pgjdbc artifacts 
		
		An updated blacklist (pgjdbcBlackList19_1) is available in the artifacts' pgjdbc log

cockroach-teamcity · 2019-09-04T09:26:03Z

SHA: https://github.com/cockroachdb/cockroach/commits/179f29b066c266d14cfeac33ce29b2d18ba86c63

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=pgjdbc PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1465495&tab=buildLog

The test failed on branch=release-2.1, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190904-1465495/pgjdbc/run_1
	pgjdbc.go:306,pgjdbc.go:317,test_runner.go:688: 
		Tests run on Cockroach v2.1.8-8-g179f29b
		Tests run against pgjdbc REL42.2.6
		5786 Total Tests Run
		3360 tests passed
		2426 tests failed
		0 tests passed unexpectedly
		2412 tests failed unexpectedly
		0 tests expected failed, but not run
		For a full summary look at the pgjdbc artifacts 
		
		An updated blacklist (pgjdbcBlackList2_1) is available in the artifacts' pgjdbc log

jordanlewis · 2019-09-04T15:38:59Z

Something went pretty wrong here - 2412 unexpected failures. @rafiss do you know what happened?

rafiss · 2019-09-04T16:31:17Z

Yes, this is the first time that this suite was run against a 2.1 cluster. I will update the blacklist for 2.1.

cockroach-teamcity · 2019-09-05T07:41:43Z

SHA: https://github.com/cockroachdb/cockroach/commits/179f29b066c266d14cfeac33ce29b2d18ba86c63

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=pgjdbc PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1468506&tab=buildLog

The test failed on branch=provisional_201909042143_v2.1.9, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190905-1468506/pgjdbc/run_1
	pgjdbc.go:306,pgjdbc.go:317,test_runner.go:688: 
		Tests run on Cockroach v2.1.8-8-g179f29b
		Tests run against pgjdbc REL42.2.6
		5786 Total Tests Run
		3360 tests passed
		2426 tests failed
		0 tests passed unexpectedly
		2412 tests failed unexpectedly
		0 tests expected failed, but not run
		For a full summary look at the pgjdbc artifacts 
		
		An updated blacklist (pgjdbcBlackList2_1) is available in the artifacts' pgjdbc log

40335: coltypes: don't shallow copy decimals r=jordanlewis a=jordanlewis This commit fixes a correctness bug caused by illegal shallow copy of decimals. To copy a decimal, you must use the .Set method on it, and its memory must already be distinct from the source decimal. To facilitate this and generally simplify things, the "safe" GET method is removed and replaced by COPYVAL, which copies a scalar value to another scalar value safely. There were fewer than 5 users of GET, so this wasn't particularly disruptive. Closes #39777. Closes #39540. Release note: None 40481: roachtest: update pgjdbc blacklist for 2.1 r=rafiss a=rafiss The 2.1 blacklist was never filled in. closes #39406 Release note: None 40492: Docs: Add SHOW JOBS diagram r=lhirata a=lhirata Release note: none Co-authored-by: Rohan Yadav <[email protected]> Co-authored-by: Jordan Lewis <[email protected]> Co-authored-by: Rafi Shamim <[email protected]> Co-authored-by: Lauren <[email protected]>

cockroach-teamcity added this to the 19.2 milestone Aug 7, 2019

cockroach-teamcity assigned andreimatei Aug 7, 2019

cockroach-teamcity added C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. labels Aug 7, 2019

jordanlewis assigned rafiss and unassigned andreimatei Aug 7, 2019

rafiss mentioned this issue Aug 7, 2019

roachtest: fix pgjdbc tests #39420

Merged

kenliu mentioned this issue Aug 9, 2019

release: v19.1.4 #39341

Closed

18 tasks

rafiss mentioned this issue Sep 4, 2019

roachtest: update pgjdbc blacklist for 2.1 #40481

Merged

craig bot closed this as completed in 3bb9277 Sep 5, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

roachtest: pgjdbc failed #39406

roachtest: pgjdbc failed #39406

cockroach-teamcity commented Aug 7, 2019

rafiss commented Aug 7, 2019

cockroach-teamcity commented Aug 8, 2019

cockroach-teamcity commented Aug 8, 2019

cockroach-teamcity commented Aug 13, 2019

cockroach-teamcity commented Sep 4, 2019

jordanlewis commented Sep 4, 2019

rafiss commented Sep 4, 2019

cockroach-teamcity commented Sep 5, 2019

roachtest: pgjdbc failed #39406

roachtest: pgjdbc failed #39406

Comments

cockroach-teamcity commented Aug 7, 2019

rafiss commented Aug 7, 2019

cockroach-teamcity commented Aug 8, 2019

cockroach-teamcity commented Aug 8, 2019

cockroach-teamcity commented Aug 13, 2019

cockroach-teamcity commented Sep 4, 2019

jordanlewis commented Sep 4, 2019

rafiss commented Sep 4, 2019

cockroach-teamcity commented Sep 5, 2019