Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

storageccl: don't update file registry when creating unencrypted files #66273

Merged
merged 1 commit into from
Jun 10, 2021

Conversation

andyyang890
Copy link
Collaborator

This patch changes PebbleFileRegistry to not store an entry
for an unencrypted file. This should lead to a performance
improvement since the file registry is rewritten every time
it is updated.

Fixes #65430.

Release note: None

@andyyang890 andyyang890 requested review from itsbilal, jbowens, sumeerbhola, a team and pbardea and removed request for a team June 9, 2021 19:11
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@andyyang890 andyyang890 force-pushed the improve_registry_unencrypted branch from 0615bb7 to 172638e Compare June 9, 2021 19:21
Copy link
Collaborator

@jbowens jbowens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @andyyang890, @itsbilal, @pbardea, and @sumeerbhola)


pkg/storage/pebble_file_registry.go, line 110 at r1 (raw file):

		return nil
	}

Do we need to verify that the file doesn't already exist in the registry? I believe the only caller of SetFileEntry is encryptedFS.Create. Calling Create on an existing path is permitted and truncates the file:

https://github.com/cockroachdb/pebble/blob/cbda11b8689ff2093da5868a84ff00b54120d21c/vfs/vfs.go#L45-L47

If the old file was encrypted and still in the registry, it looks like we'd have the incorrect registry state for the file unless we remove the existing entry.

@andyyang890 andyyang890 force-pushed the improve_registry_unencrypted branch from 172638e to 09ec487 Compare June 9, 2021 22:36
Copy link
Collaborator Author

@andyyang890 andyyang890 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @itsbilal, @jbowens, @pbardea, and @sumeerbhola)


pkg/storage/pebble_file_registry.go, line 110 at r1 (raw file):

Previously, jbowens (Jackson Owens) wrote…

Do we need to verify that the file doesn't already exist in the registry? I believe the only caller of SetFileEntry is encryptedFS.Create. Calling Create on an existing path is permitted and truncates the file:

https://github.com/cockroachdb/pebble/blob/cbda11b8689ff2093da5868a84ff00b54120d21c/vfs/vfs.go#L45-L47

If the old file was encrypted and still in the registry, it looks like we'd have the incorrect registry state for the file unless we remove the existing entry.

Good catch, fixed!

@pbardea
Copy link
Contributor

pbardea commented Jun 10, 2021

Removing myself as a reviewer, deferring to storage folks but did want to mention that if this is a performance improvement (and it looks like it is), it may warrant a performance improvement release note.

@pbardea pbardea removed their request for review June 10, 2021 13:34
Copy link
Collaborator

@jbowens jbowens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:LGTM:

Test looks good!
I think we'll want to backport this to 21.1 and 20.2.

Reviewed 1 of 2 files at r1, 1 of 1 files at r2.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @andyyang890, @itsbilal, @jbowens, and @sumeerbhola)


pkg/storage/pebble_file_registry.go, line 105 at r2 (raw file):

// SetFileEntry sets filename => entry in the registry map and persists the registry.
func (r *PebbleFileRegistry) SetFileEntry(filename string, entry *enginepb.FileEntry) error {
	// We choose not store an entry for unencrypted files since the absence of

nit: s/not store/not to store/

This patch changes PebbleFileRegistry to not store an entry
for an unencrypted file. This should lead to a performance
improvement since the file registry is rewritten every time
it is updated.

Release note: None
@andyyang890 andyyang890 force-pushed the improve_registry_unencrypted branch from 09ec487 to 0b806b4 Compare June 10, 2021 14:52
@andyyang890
Copy link
Collaborator Author

Removing myself as a reviewer, deferring to storage folks but did want to mention that if this is a performance improvement (and it looks like it is), it may warrant a performance improvement release note.

Noted, will add a release note to the backports but not this commit since it will be replaced by an upcoming revamp of encryption-at-rest.

@andyyang890
Copy link
Collaborator Author

bors r=jbowens

@craig
Copy link
Contributor

craig bot commented Jun 10, 2021

Build failed:

@andyyang890
Copy link
Collaborator Author

bors r=jbowens

@craig
Copy link
Contributor

craig bot commented Jun 10, 2021

Build succeeded:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

storageccl: Don't update file registry when unencrypted files are created
4 participants