-
Notifications
You must be signed in to change notification settings - Fork 534
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(ioredis): only serialize non sensitive arguments in db statement attribute #1052
feat(ioredis): only serialize non sensitive arguments in db statement attribute #1052
Conversation
6f1528e
to
9f099e6
Compare
Codecov Report
@@ Coverage Diff @@
## main #1052 +/- ##
==========================================
- Coverage 95.91% 95.72% -0.19%
==========================================
Files 13 16 +3
Lines 856 1005 +149
Branches 178 200 +22
==========================================
+ Hits 821 962 +141
- Misses 35 43 +8
|
plugins/node/opentelemetry-instrumentation-ioredis/test/ioredis.test.ts
Outdated
Show resolved
Hide resolved
plugins/node/opentelemetry-instrumentation-ioredis/test/ioredis.test.ts
Outdated
Show resolved
Hide resolved
plugins/node/opentelemetry-instrumentation-ioredis/src/utils.ts
Outdated
Show resolved
Hide resolved
I went through the full list at https://redis.io/commands, and not only is there a lot of commands where we can just serialize everything, but the remainder should very often serialize more than just the first argument (as @blumamir pointed out). I've created a map of command names to number of serializable arguments so we can deal with these on a case-by-case basis. It's not perfect, and several commands will still be missing "safe" data that could be serialized, such as SET, which has a ton of optional arguments that come after the value. I've added |
Awsome @aptomaKetil , that is great work, thank you so much for doing it. I like your idea of adding I will look at the exact details later. If you want (optionally), it could be great to extract this logic into another new package maybe const defaultDbStatementSerializer: DbStatementSerializer = cmdName => cmdName; This PR is a great improvement on its own, so if you don't have the time or will to do this, let me know and I'll open a relevant issue |
Better create an issue for it, I won't have time to look at that right now. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM.
Thanks again for fixing this
Left few optional nits
plugins/node/opentelemetry-instrumentation-ioredis/src/utils.ts
Outdated
Show resolved
Hide resolved
plugins/node/opentelemetry-instrumentation-ioredis/src/utils.ts
Outdated
Show resolved
Hide resolved
plugins/node/opentelemetry-instrumentation-ioredis/src/utils.ts
Outdated
Show resolved
Hide resolved
plugins/node/opentelemetry-instrumentation-ioredis/src/utils.ts
Outdated
Show resolved
Hide resolved
Updated based on feedback from @blumamir and @vmarchaud. Fallback to only serialize first argument, for added safety. Change to a whitelist approach for commands that should serialize a different subset of commands. Use regexes for subset groups to keep things short. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@aptomaKetil @vmarchaud , What do you think about changing the default behavior to capture no arguments instead of one for all commands not in the white list? I think it will be safer
plugins/node/opentelemetry-instrumentation-ioredis/src/utils.ts
Outdated
Show resolved
Hide resolved
plugins/node/opentelemetry-instrumentation-ioredis/src/utils.ts
Outdated
Show resolved
Hide resolved
plugins/node/opentelemetry-instrumentation-ioredis/src/utils.ts
Outdated
Show resolved
Hide resolved
plugins/node/opentelemetry-instrumentation-ioredis/src/utils.ts
Outdated
Show resolved
Hide resolved
plugins/node/opentelemetry-instrumentation-ioredis/test/ioredis.test.ts
Outdated
Show resolved
Hide resolved
I don't think there is problem adding the command name since its already on the span name right ? |
plugins/node/opentelemetry-instrumentation-ioredis/src/utils.ts
Outdated
Show resolved
Hide resolved
If I am not mistaken, the current implementation will always record the command name, and for commands not in the white list it will additionally record the first command argument. |
Updated to serialize no arguments by default. |
Amazing, thank you for addressing everything. This is great work 🥇 |
Which problem is this PR solving?
The ioredis instrumentation is currently serializing the whole redis command, including potentially sensitive (or simply large) data.
Closes #1030.
Short description of the changes
Only serialize non-sensative command argument in
db.statement
attribute.Checklist
npm run test-all-versions
for the edited package(s) on the latest commit if applicable.