-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
release-22.2: pkg/keys: implement SafeFormatter for roachpb.Key #87647
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Currently, when a key is logged, the entirety of the pretty-printed key is redacted, hindering observability when parsing through redacted logs (something that will become more common with upcoming compliance requirements). For example, prior to this patch, a pretty-printed key would appear in the following way for the unredacted/redacted cases, respectively: - unredacted: ‹/Table/42/1222/"index key"› - redacted: ‹x› This patch addresses this by implementing the SafeFormatter interface for `roachpb.Key` and `roachpb.RKey`, yielding the following result when looking at the same example above: - unredacted: /Table/42/1222/‹"index key"› - redacted: /Table/42/1222/‹x› While the index key itself remains redacted, the ability to see the specific table, index, and in the case of tenant tables, the tenant itself, provides much better observability into which table & index a log line is referring to than before. Note that this implementation is only partial. It currently only supports keys that fall in the `/Table` keyspace for application tenants and system tenants. Keyspaces such as Meta1, Meta2, Local, etc. are not yet supported, but can be added with much more ease in the future now that the example has been set. Finally, we remove the `maxLen` and related truncation logic from `StringWithDirs`, as this is no longer used. Furthermore, the truncation was invalid as it could have truncated a utf-8 sequence in the wrong place, making the result invalid utf-8. Release note (security update): redacted logs will now reveal pretty-printed keys, except for the index key values themselves. For example `/Table/42/1222/‹x›` will be shown instead of `‹x›` (which was shown previously). This improved redaction is available for the `/Table` keyspace for both system and application tenants. Other keyspaces such as `/Meta1`, `/Meta2`, `/Local`, etc. are not yet supported. Release justification: low risk, high benefit observability changes
blathers-crl
bot
force-pushed
the
blathers/backport-release-22.2-86813
branch
from
September 8, 2022 20:46
ff5ad33
to
6f93145
Compare
Thanks for opening a backport. Please check the backport criteria before merging:
If some of the basic criteria cannot be satisfied, ensure that the exceptional criteria are satisfied within.
Add a brief release justification to the body of your PR to justify this backport. Some other things to consider:
|
blathers-crl
bot
added
blathers-backport
This is a backport that Blathers created automatically.
O-robot
Originated from a bot.
labels
Sep 8, 2022
dhartunian
approved these changes
Sep 19, 2022
andreimatei
approved these changes
Sep 20, 2022
TFTR! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
blathers-backport
This is a backport that Blathers created automatically.
O-robot
Originated from a bot.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport 1/1 commits from #86813 on behalf of @abarganier.
/cc @cockroachdb/release
Currently, when a key is logged, the entirety of the pretty-printed
key is redacted, hindering observability when parsing through redacted
logs (something that will become more common with upcoming compliance
requirements).
For example, prior to this patch, a pretty-printed key would appear
in the following way for the unredacted/redacted cases, respectively:
This patch addresses this by implementing the SafeFormatter interface
for
roachpb.Key
androachpb.RKey
, yielding the following resultwhen looking at the same example above:
While the index key itself remains redacted, the ability to see the
specific table, index, and in the case of tenant tables, the tenant
itself, provides much better observability into which table & index
a log line is referring to than before.
Note that this implementation is only partial. It currently only
supports keys that fall in the
/Table
keyspace for applicationtenants and system tenants. Keyspaces such as Meta1, Meta2, Local,
etc. are not yet supported, but can be added with much more ease
in the future now that the example has been set.
Finally, we remove the
maxLen
and related truncation logic fromStringWithDirs
, as this is no longer used. Furthermore, thetruncation was invalid as it could have truncated a utf-8
sequence in the wrong place, making the result invalid utf-8.
This PR is a continuation of the work originally done by @kzh in
#67065. See the original PR for some initial discussions.
Release note (security update): redacted logs will now reveal
pretty-printed keys, except for the index key values themselves.
For example
/Table/42/1222/‹x›
will be shown instead of‹x›
(which was shown previously). This improved redaction is available
for the
/Table
keyspace for both system and application tenants.Other keyspaces such as
/Meta1
,/Meta2
,/Local
, etc. are notyet supported.
Release justification: low risk, high benefit observability changes
Addresses #86316
Release justification: