Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

roachtest,sqlinstance: fix multitenant_upgrade test #87111

Merged
merged 2 commits into from
Sep 1, 2022

Conversation

rafiss
Copy link
Collaborator

@rafiss rafiss commented Aug 30, 2022

fixes #81517
fixes #81506

roachtest: fix ordering of multitenant upgrade test step

The tenant must be initialized by a system tenant using an old binary in
order to be running on an old binary itself.

Also, we no longer finalize the secondary tenant upgrade before finalizing the
system tenant upgrade.

sqlinstance: use the correct timestamp when populating the cache

Previously, the sql instance cache would be populated using the
timestamp of the initial rangefeed scan. This was a bug since the
timestamp is used to determine which sql instance ID to use when there
are two IDs for the same address (i.e., it should use the most recent
one).

This caused problems if a tenant node was shutdown, then started with
the same address. Other layers like DistSQL would try to route requests
to an old sql instance, which would actually be the same as the
requesting node.

Release note: None

Release justification: test only change, and critical bug fix

@cockroach-teamcity
Copy link
Member

This change is Reviewable

@rafiss rafiss force-pushed the fix-multi-tenant-upgrade branch from 7e8848a to 11a6c9c Compare August 30, 2022 21:35
@rafiss rafiss marked this pull request as ready for review September 1, 2022 04:18
Copy link
Contributor

@ajwerner ajwerner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@@ -162,6 +162,7 @@ func (r *Reader) maybeStartRangeFeed(ctx context.Context) *rangefeed.RangeFeed {
updateCacheFn,
rangefeed.WithInitialScan(initialScanDoneFn),
rangefeed.WithOnInitialScanError(initialScanErrFn),
rangefeed.WithRowTimestampInInitialScan(true),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

honestly, this is the problem with functional options :(

@rafiss rafiss changed the title roachtest: fix multitenant_upgrade test roachtest,sqlinstance: fix multitenant_upgrade test Sep 1, 2022
The tenant must be initialized by a system tenant using an old binary in
order to be running on an old binary itself.

Also, we no longer finalize the secondary tenant upgrade before finalizing the
system tenant upgrade.

Release note: None

Release justification: test only change
Previously, the sql instance cache would be populated using the
timestamp of the initial rangefeed scan. This was a bug since the
timestamp is used to determine which sql instance ID to use when there
are two IDs for the same address (i.e., it should use the most recent
one).

This caused problems if a tenant node was shutdown, then started with
the same address. Other layers like DistSQL would try to route requests
to an old sql instance, which would actually be the same as the
requesting node.

No release note since this bug didn't affect anything prior to v22.2.

Release justification: high priority bug fix.

Release note: None
@rafiss rafiss force-pushed the fix-multi-tenant-upgrade branch from aebc71d to 3873bed Compare September 1, 2022 05:09
@rafiss
Copy link
Collaborator Author

rafiss commented Sep 1, 2022

tftr!

bors r=ajwerner

@craig
Copy link
Contributor

craig bot commented Sep 1, 2022

Build succeeded:

@craig craig bot merged commit d18ecbb into cockroachdb:master Sep 1, 2022
@rafiss rafiss deleted the fix-multi-tenant-upgrade branch September 1, 2022 18:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

roachtest: multitenant-upgrade failed roachtest: acceptance/multitenant failed
3 participants