Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

roachtest: django failed #105598

Closed
cockroach-teamcity opened this issue Jun 27, 2023 · 12 comments · Fixed by #105866
Closed

roachtest: django failed #105598

cockroach-teamcity opened this issue Jun 27, 2023 · 12 comments · Fixed by #105866
Assignees
Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. T-sql-queries SQL Queries Team
Milestone

Comments

@cockroach-teamcity
Copy link
Member

cockroach-teamcity commented Jun 27, 2023

roachtest.django failed with artifacts on master @ 0b0a212ecbe514b55301f4a963edc92b16de9bab:

(orm_helpers.go:212).summarizeFailed: 
Tests run on Cockroach v23.2.0-alpha.00000000-dev-0b0a212ecbe514b55301f4a963edc92b16de9bab
Tests run against django cockroach-4.1.x
9951 Total Tests Run
9949 tests passed
2 tests failed
639 tests skipped
2 tests ignored
0 tests passed unexpectedly
2 tests failed unexpectedly
0 tests expected failed but skipped
0 tests expected failed but not run
---
--- FAIL: queries.tests.Queries1Tests.test_ticket2306 (unexpected)
--- FAIL: queries.tests.Queries1Tests.test_ticket4464 (unexpected)
--- SKIP: backends.mysql.test_creation.DatabaseCreationTests.test_create_test_db_unexpected_error due to MySQL tests (expected)
For a full summary look at the django artifacts 
An updated blocklist (djangoBlocklist) is available in the artifacts' django log
test artifacts and logs in: /artifacts/django/run_1

Parameters: ROACHTEST_arch=amd64 , ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

/cc @cockroachdb/sql-foundations

This test on roachdash | Improve this report!

Jira issue: CRDB-29116

@cockroach-teamcity cockroach-teamcity added branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions) labels Jun 27, 2023
@cockroach-teamcity cockroach-teamcity added this to the 23.2 milestone Jun 27, 2023
@rafiss
Copy link
Collaborator

rafiss commented Jun 27, 2023

======================================================================
FAIL: test_ticket2306 (queries.tests.Queries1Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/mnt/data1/django/tests/queries/tests.py", line 228, in test_ticket2306
    self.assertSequenceEqual(
AssertionError: Sequences differ: <QuerySet []> != [<Item: one>]

Second sequence contains 1 additional elements.
First extra element 0:
<Item: one>

- <QuerySet []>
+ [<Item: one>]

======================================================================
FAIL: test_ticket4464 (queries.tests.Queries1Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/mnt/data1/django/tests/queries/tests.py", line 259, in test_ticket4464
    self.assertSequenceEqual(
AssertionError: Sequences differ: <QuerySet []> != [<Item: one>]

Second sequence contains 1 additional elements.
First extra element 0:
<Item: one>

- <QuerySet []>
+ [<Item: one>]

----------------------------------------------------------------------

This might be a regression. @andyyang890 can you see if this reproduces? If so, we should find the query it's running, and potentially hand this off to another team.

@cockroach-teamcity
Copy link
Member Author

roachtest.django failed with artifacts on master @ 4a614f89cea81bf94674d6072c3bbf30502244d4:

(orm_helpers.go:212).summarizeFailed: 
Tests run on Cockroach v23.2.0-alpha.00000000-dev-4a614f89cea81bf94674d6072c3bbf30502244d4
Tests run against django cockroach-4.1.x
9951 Total Tests Run
9949 tests passed
2 tests failed
639 tests skipped
2 tests ignored
0 tests passed unexpectedly
2 tests failed unexpectedly
0 tests expected failed but skipped
0 tests expected failed but not run
---
--- FAIL: queries.tests.Queries1Tests.test_ticket2306 (unexpected)
--- FAIL: queries.tests.Queries1Tests.test_ticket4464 (unexpected)
--- SKIP: backends.mysql.test_creation.DatabaseCreationTests.test_create_test_db_unexpected_error due to MySQL tests (expected)
For a full summary look at the django artifacts 
An updated blocklist (djangoBlocklist) is available in the artifacts' django log
test artifacts and logs in: /artifacts/django/run_1

Parameters: ROACHTEST_arch=amd64 , ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_fs=ext4 , ROACHTEST_localSSD=true , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.django failed with artifacts on master @ 84a5961ebde5898d3c97aa97da59719c12a5ed16:

(orm_helpers.go:212).summarizeFailed: 
Tests run on Cockroach v23.2.0-alpha.00000000-dev-84a5961ebde5898d3c97aa97da59719c12a5ed16
Tests run against django cockroach-4.1.x
9951 Total Tests Run
9949 tests passed
2 tests failed
639 tests skipped
2 tests ignored
0 tests passed unexpectedly
2 tests failed unexpectedly
0 tests expected failed but skipped
0 tests expected failed but not run
---
--- FAIL: queries.tests.Queries1Tests.test_ticket2306 (unexpected)
--- SKIP: backends.mysql.test_creation.DatabaseCreationTests.test_create_test_db_unexpected_error due to MySQL tests (expected)
--- FAIL: queries.tests.Queries1Tests.test_ticket4464 (unexpected)
For a full summary look at the django artifacts 
An updated blocklist (djangoBlocklist) is available in the artifacts' django log
test artifacts and logs in: /artifacts/django/run_1

Parameters: ROACHTEST_arch=amd64 , ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_fs=ext4 , ROACHTEST_localSSD=true , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

This test on roachdash | Improve this report!

@rafiss
Copy link
Collaborator

rafiss commented Jun 30, 2023

From a brief look at the test code (https://github.com/django/django/blob/7eeadc82c2f7d7a778e3bb43c34d642e6275dacf/tests/queries/tests.py#L219), and based on the timing of when the failure started, this seems likely to be similar to #105608 (cc @DrewKimball)

@rafiss rafiss added T-sql-queries SQL Queries Team and removed T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions) labels Jun 30, 2023
@andyyang890
Copy link
Collaborator

andyyang890 commented Jul 1, 2023

I ran a git bisect and it does point to a87cfe6 again so it probably is the same root cause.

For the actual SQL queries, there are a lot of setup queries involved so I won't paste it all here but here are the actual SELECT queries for test_ticket2306 in case that's helpful in any way:

SELECT queries_tag.id
 FROM queries_tag INNER
   JOIN queries_item_tags
    ON (queries_tag.id = queries_item_tags.tag_id)
   WHERE queries_item_tags.item_id = _
 ORDER BY queries_tag.name ASC
SELECT queries_item.id, queries_item.name, queries_item.created, queries_item.modified, queries_item.creator_id,
queries_item.note_id
 FROM queries_item INNER
   JOIN queries_item_tags
    ON (queries_item.id = queries_item_tags.item_id)
   WHERE queries_item_tags.tag_id = _
 ORDER BY queries_item.name ASC
SELECT queries_item.id, queries_item.name, queries_item.created, queries_item.modified, queries_item.creator_id,
queries_item.note_id
 FROM queries_item INNER
   JOIN queries_item_tags
    ON (queries_item.id = queries_item_tags.item_id) INNER
   JOIN queries_item_tags AS t4
    ON (queries_item.id = t4.item_id) INNER
   JOIN queries_note
    ON (queries_item.note_id = queries_note.id)
   WHERE ((queries_item_tags.tag_id = _)
    AND (t4.tag_id = _))
 ORDER BY queries_note.note DESC, queries_item.name ASC

@DrewKimball
Copy link
Collaborator

Yes, that looks right to be the same bug as #105608.

@andyyang890
Copy link
Collaborator

Thanks for confirming! I'll reassign this issue to you then

@cockroach-teamcity
Copy link
Member Author

roachtest.django failed with artifacts on master @ df82b5306e945d396269f28874d98fa567f4ac32:

(orm_helpers.go:212).summarizeFailed: 
Tests run on Cockroach v23.2.0-alpha.00000000-dev-df82b5306e945d396269f28874d98fa567f4ac32
Tests run against django cockroach-4.1.x
9951 Total Tests Run
9949 tests passed
2 tests failed
639 tests skipped
2 tests ignored
0 tests passed unexpectedly
2 tests failed unexpectedly
0 tests expected failed but skipped
0 tests expected failed but not run
---
--- FAIL: queries.tests.Queries1Tests.test_ticket4464 (unexpected)
--- SKIP: backends.mysql.test_creation.DatabaseCreationTests.test_create_test_db_unexpected_error due to MySQL tests (expected)
--- FAIL: queries.tests.Queries1Tests.test_ticket2306 (unexpected)
For a full summary look at the django artifacts 
An updated blocklist (djangoBlocklist) is available in the artifacts' django log
test artifacts and logs in: /artifacts/django/run_1

Parameters: ROACHTEST_arch=amd64 , ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.django failed with artifacts on master @ aacba20d325e5702836e9a76be646b5f1bd922af:

(orm_helpers.go:212).summarizeFailed: 
Tests run on Cockroach v23.2.0-alpha.00000000-dev-aacba20d325e5702836e9a76be646b5f1bd922af
Tests run against django cockroach-4.1.x
9951 Total Tests Run
9949 tests passed
2 tests failed
639 tests skipped
2 tests ignored
0 tests passed unexpectedly
2 tests failed unexpectedly
0 tests expected failed but skipped
0 tests expected failed but not run
---
--- FAIL: queries.tests.Queries1Tests.test_ticket2306 (unexpected)
--- SKIP: backends.mysql.test_creation.DatabaseCreationTests.test_create_test_db_unexpected_error due to MySQL tests (expected)
--- FAIL: queries.tests.Queries1Tests.test_ticket4464 (unexpected)
For a full summary look at the django artifacts 
An updated blocklist (djangoBlocklist) is available in the artifacts' django log
test artifacts and logs in: /artifacts/django/run_1

Parameters: ROACHTEST_arch=amd64 , ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.django failed with artifacts on master @ aacba20d325e5702836e9a76be646b5f1bd922af:

(orm_helpers.go:212).summarizeFailed: 
Tests run on Cockroach v23.2.0-alpha.00000000-dev-aacba20d325e5702836e9a76be646b5f1bd922af
Tests run against django cockroach-4.1.x
9951 Total Tests Run
9949 tests passed
2 tests failed
639 tests skipped
2 tests ignored
0 tests passed unexpectedly
2 tests failed unexpectedly
0 tests expected failed but skipped
0 tests expected failed but not run
---
--- FAIL: queries.tests.Queries1Tests.test_ticket2306 (unexpected)
--- SKIP: backends.mysql.test_creation.DatabaseCreationTests.test_create_test_db_unexpected_error due to MySQL tests (expected)
--- FAIL: queries.tests.Queries1Tests.test_ticket4464 (unexpected)
For a full summary look at the django artifacts 
An updated blocklist (djangoBlocklist) is available in the artifacts' django log
test artifacts and logs in: /artifacts/django/run_1

Parameters: ROACHTEST_arch=amd64 , ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.django failed with artifacts on master @ 428dc9da6a320de218460de6c6c8807caa4ded98:

(orm_helpers.go:212).summarizeFailed: 
Tests run on Cockroach v23.2.0-alpha.00000000-dev-428dc9da6a320de218460de6c6c8807caa4ded98
Tests run against django cockroach-4.1.x
9951 Total Tests Run
9949 tests passed
2 tests failed
639 tests skipped
2 tests ignored
0 tests passed unexpectedly
2 tests failed unexpectedly
0 tests expected failed but skipped
0 tests expected failed but not run
---
--- SKIP: backends.mysql.test_creation.DatabaseCreationTests.test_create_test_db_unexpected_error due to MySQL tests (expected)
--- FAIL: queries.tests.Queries1Tests.test_ticket2306 (unexpected)
--- FAIL: queries.tests.Queries1Tests.test_ticket4464 (unexpected)
For a full summary look at the django artifacts 
An updated blocklist (djangoBlocklist) is available in the artifacts' django log
test artifacts and logs in: /artifacts/django/run_1

Parameters: ROACHTEST_arch=amd64 , ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.django failed with artifacts on master @ 34699bb9c1557fce449e08a68cd259efec94926f:

(orm_helpers.go:212).summarizeFailed: 
Tests run on Cockroach v23.2.0-alpha.00000000-dev-34699bb9c1557fce449e08a68cd259efec94926f
Tests run against django cockroach-4.1.x
9951 Total Tests Run
9949 tests passed
2 tests failed
639 tests skipped
2 tests ignored
0 tests passed unexpectedly
2 tests failed unexpectedly
0 tests expected failed but skipped
0 tests expected failed but not run
---
--- FAIL: queries.tests.Queries1Tests.test_ticket2306 (unexpected)
--- FAIL: queries.tests.Queries1Tests.test_ticket4464 (unexpected)
--- SKIP: backends.mysql.test_creation.DatabaseCreationTests.test_create_test_db_unexpected_error due to MySQL tests (expected)
For a full summary look at the django artifacts 
An updated blocklist (djangoBlocklist) is available in the artifacts' django log
test artifacts and logs in: /artifacts/django/run_1

Parameters: ROACHTEST_arch=amd64 , ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=false , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

This test on roachdash | Improve this report!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. T-sql-queries SQL Queries Team
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

4 participants