feat(node): Implement Sentry-specific http instrumentation #13763
Conversation
dev-packages/e2e-tests/test-applications/node-otel-without-tracing/src/instrument.ts
Outdated
Show resolved
Hide resolved
size-limit report 📦
|
❌ 1 Tests Failed:
View the top 1 failed tests by shortest run time
To view individual test run time comparison to the main branch, go to the Test Analytics Dashboard |
) Found this while working on #13763. Oops, the node-otel-without-tracing E2E test was not running on CI, we forgot to add it there - and it was actually failing since we switched to the new undici instrumentation :O This PR ensures to add it to CI, and also fixes it. The main change for this is to ensure we do not emit any spans when tracing is disabled, while still ensuring that trace propagation works as expected for this case. I also pulled some general changes into this, which ensure that we patch both `http.get` and `http.request` properly.
mydea
force-pushed
the
fn/custom-http-shim
branch
from
8cb19ad
to
a9fde4f
Compare
|
v1.11.1 of |
mydea
force-pushed
the
fn/custom-http-shim
branch
from
b7c8b37
to
00d177d
Compare
| @@ -14,8 +14,7 @@ | |||
| }, | |||
| "devDependencies": { | |||
| "@sentry-internal/test-utils": "link:../../../test-utils", | |||
| "@playwright/test": "^1.44.1", | |||
| "wait-port": "1.0.4" | |||
There was a problem hiding this comment.
cleaned this up as this is not needed anymore
| @@ -18,7 +18,7 @@ test('AWS Serverless SDK sends events in ESM mode', async ({ request }) => { | |||
| ); | |||
|
|
|||
| child_process.execSync('pnpm start', { | |||
| stdio: 'ignore', | |||
There was a problem hiding this comment.
was debugging a test here for some time, this made it much easier to debug this
There was a problem hiding this comment.
I'm trying to work out what Node version diagnostics_channel http events were added because they probably make this all possible without touching otel/import-in-the-middle. It has both start and end events for both incoming and outgoing requests.
UPDATE
Yep, just tested and these events are available in v14.17.0
|
Let me know if you want me to have a try at reworking this PR with |
|
Thanks for that Tim, I was not aware this was available for Node 14+! I will give it a quick look to see if I can easily use this instead in this PR, if so I'll update it right away. If it is more complex, maybe we merge this and then update it in a follow up :) |
mydea
force-pushed
the
fn/custom-http-shim
branch
from
6bf01c5
to
d0aac35
Compare
|
So for reference, it is not that easy to move this to diagnostics channel, because there we cannot wrap the incoming request in an easy way to ensure async local storage has the correct isolation scope etc. So I'm going to merge this like this, we can investigate in a follow up if we can refactor it! |
| @@ -25,6 +10,7 @@ type HttpOptions = Parameters<typeof originalHttpIntegration>[0]; | |||
| export const httpIntegration = ((options: HttpOptions = {}) => { | |||
| return originalHttpIntegration({ | |||
| ...options, | |||
| _instrumentation: RemixHttpIntegration, | |||
| // We disable incoming request spans here, because otherwise we'd end up with duplicate spans. | |||
| disableIncomingRequestSpans: true, | |||
There was a problem hiding this comment.
cc @lforst also relevant for Next after we merge this
This PR is a pretty big change, but it should help us to make custom OTEL support way better/easier to understand in the future.
The fundamental problem this PR is trying to change is the restriction that we rely on our instance of
HttpInstrumentationfor Sentry-specific (read: not span-related) things. This made it tricky/annoying for users with a custom OTEL setup that may includeHttpInstrumentationto get things working, because they may inadvertedly overwrite our instance of the instrumentation (because there can only be a single monkey-patch per module in the regular instrumentations), leading to hard-to-debug and often subtle problems.This PR fixes this by splitting out the non-span related http instrumentation code into a new, dedicated
SentryHttpInstrumentation, which can be run side-by-side with the OTEL instrumentation (which emits spans, ...).We make this work by basically implementing our own custom, minimal
wrapmethod instead of using shimmer. This way, OTEL instrumentation cannot identify the wrapped module as wrapped, and allow to wrap it again. While this is slightly hacky and also means you cannot unwrap the http module, we do not generally support this for the Sentry SDK anyhow.This new Instrumentation does two things:
With this change, in errors only mode you really do not need our instance of the default
HttpInstrumentationanymore at all, you can/should just provide your own if you want to capture http spans in a non-Sentry environment. However, this is sadly a bit tricky, because up to now we forced users in this scenario to still use our Http instance and avoid adding their own (instead we allowed users to pass their Http instrumentation config to our Http integration). This means that if we'd simply stop adding our http instrumentation instance when tracing is disabled, these users would stop getting otel spans as well :/ so we sadly can't change this without a major.Instead, I re-introduced the
spans: falseforhttpIntegration({ spans: false }). When this is set (which for now is opt-in, but probably should be opt-out in v9) we will only register SentryHttpInstrumentation, not HttpInstrumentation, thus not emitting any spans. Users can add their own instance of HttpInstrumentation if they care.One semi-related thing that I noticed while looking into this is that we incorrectly emitted node-fetch spans in errors-only mode. This apparently sneaked in when we migrated to the new undici instrumentation. I extracted this out into a dedicated PR too, but the changes are in this too because tests were a bit fucked up otherwise.
On top of #13765
This also includes a bump of import-in-the-middle to 1.11.2, as this includes a fix to properly allow double-wrapping ESM modules.