Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

backupccl: allow backups with proto named BACKUP_MANIFEST #40643

Merged
merged 1 commit into from
Sep 27, 2019

Conversation

dt
Copy link
Member

@dt dt commented Sep 10, 2019

This does not change the writing of the current manifest file at the current name -- it only checks for the new name when reading the old one fails.

Adding this check now will allow a future-version to only write at the new name, but still play nice with any old nodes, as during the upgrade all nodes will look at both names.

Release justification: this is a low-risk change with high value, in that it establishes compatibility for a UX polish change we want to do in 20.1.

Release note: none.

@dt dt requested a review from thoszhang September 10, 2019 19:02
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@dt
Copy link
Member Author

dt commented Sep 10, 2019

This isn't critical to merge now and I could see declining to do so due to RC stability period rules. However if we do merge a small change to the read-path now to handle either name, it makes changing the write-path in 20.1 while not breaking mixed-version clusters a lot cleaner so I figured I'd send it out if that seems like a strong enough motivation.

Copy link
Contributor

@thoszhang thoszhang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @dt and @lucy-zhang)


pkg/ccl/backupccl/backup.go, line 59 at r1 (raw file):

	// BackupManifestName is a future name for the serialized
	// BackupDescriptor proto.
	BackupManifestName = BackupDescriptorName + "_MANIFEST"

Should this just be defined as "BACKUP_MANIFEST"? I don't think we use BackupDescriptorName as a prefix anywhere else.

Also, maybe we should just call these BackupManifestOldName and BackupManifestNewName or something, to make it clear that the files have the same content.


pkg/ccl/backupccl/backup.go, line 98 at r1 (raw file):

	backupDesc, err := readBackupDescriptor(ctx, exportStore, BackupDescriptorName)
	if err != nil {
		backupManifest, manifestErr := readBackupDescriptor(ctx, exportStore, BackupManifestName)

We'll have to attempt reading both filenames indefinitely into the future, right? At some point, I think having the if statements in the opposite order will make more sense.

Copy link
Member Author

@dt dt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @lucy-zhang)


pkg/ccl/backupccl/backup.go, line 59 at r1 (raw file):

Previously, lucy-zhang (Lucy Zhang) wrote…

Should this just be defined as "BACKUP_MANIFEST"? I don't think we use BackupDescriptorName as a prefix anywhere else.

Also, maybe we should just call these BackupManifestOldName and BackupManifestNewName or something, to make it clear that the files have the same content.

Done w.r.t. literal.

Agreed on the BackupManifestOldName but I'd rather do that when we actually change it for real -- my goal here is very much minimal churn required to just get the foot-in-the-door we need for upgrades later and no more.


pkg/ccl/backupccl/backup.go, line 98 at r1 (raw file):

Previously, lucy-zhang (Lucy Zhang) wrote…

We'll have to attempt reading both filenames indefinitely into the future, right? At some point, I think having the if statements in the opposite order will make more sense.

Yeah, I think a build that writes the new name would also optimistically read the new name first, but I think this order makes sense for the build that writes the old name.

This doesn't change the writing of the current manifest, but changes the
reading paths to also look for the new name so that in the 20.1 upgrade,
19.2 nodes will be aware of and correctly handle the new name.

Release justification: this is a low-risk change with high value, in that it establishes compatibility for a UX polish change we want to do in 20.1.

Release note: none.
@dt
Copy link
Member Author

dt commented Sep 26, 2019

@lucy-zhang I just realized I didn't merge this before I went OOO. Just wanted to re-check: do you still think this is worth merging for 19.2?

@thoszhang
Copy link
Contributor

Yeah, I think it's worth it.

@dt
Copy link
Member Author

dt commented Sep 26, 2019

bors r+

@dt
Copy link
Member Author

dt commented Sep 26, 2019

TFTR!

craig bot pushed a commit that referenced this pull request Sep 26, 2019
40643: backupccl: allow backups with proto named BACKUP_MANIFEST r=dt a=dt

This does *not* change the writing of the current manifest file at the current name -- it only checks for the new name when reading the old one fails.

Adding this check now will allow a future-version to only write at the new name, but still play nice with any old nodes, as during the upgrade all nodes will look at both names.

Release note: none.

Co-authored-by: David Taylor <[email protected]>
@craig
Copy link
Contributor

craig bot commented Sep 26, 2019

Build failed

@dt
Copy link
Member Author

dt commented Sep 27, 2019

Huh, I assumed the Release justification linter operated on just the commit messages but I guess it wants it in the GitHub PR text too?

bors r+

craig bot pushed a commit that referenced this pull request Sep 27, 2019
40643: backupccl: allow backups with proto named BACKUP_MANIFEST r=dt a=dt

This does *not* change the writing of the current manifest file at the current name -- it only checks for the new name when reading the old one fails.

Adding this check now will allow a future-version to only write at the new name, but still play nice with any old nodes, as during the upgrade all nodes will look at both names.

Release justification: this is a low-risk change with high value, in that it establishes compatibility for a UX polish change we want to do in 20.1.

Release note: none.

Co-authored-by: David Taylor <[email protected]>
@craig
Copy link
Contributor

craig bot commented Sep 27, 2019

Build succeeded

@craig craig bot merged commit 10d0a5b into cockroachdb:master Sep 27, 2019
@dt dt deleted the backup-manifest branch September 30, 2019 16:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants