Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(tracing): Add hook for trace sampling function to SDK options #2820

Merged
merged 40 commits into from
Sep 9, 2020

Conversation

lobsterkatie
Copy link
Member

@lobsterkatie lobsterkatie commented Aug 14, 2020

Currently, tracing is either on, with a static sample rate for all transactions, or off. This doesn't let users either

a) sample different transactions at different rates or
b) filter out certain transactions all together.

This PR introduces a new SDK option, tracesSampler, whose value is a callback with the signature

tracesSampler(samplingContext: SamplingContext): number | boolean,

where samplingContext is a combination of automatically-provided data (varying by SDK/integration) and data passed by the user to startTransaction() as an optional third parameter. Given that data, the tracesSampler function can then compute the appropriate sample rate and return it to the sampling function.

Example use cases for variable sample rate:

  • Collect traces for all enterprise customers but only a fraction of regular customers
  • Collect a higher percentage of traces on rarely-visited pages, to ensure a decent sample size
  • Collect more traces on a page you recently revamped compared to one whose code you already have well-profiled

Example use cases for filtering:

  • Don't trace pre-flight CORS requests
  • Don't trace health check requests
  • Don't collect traces from users with a certain browser extension enabled, because it's known to break things
  • Don't trace navigation to deprecated parts of your web app

(A user can, of course, combine these ideas as well, returning a variable sample rate for some traces and 0 to filter out others.)

In order to support this addition, a few other changes needed to be made:

  • Move method for extracting node request data to @sentry/utils
  • Break up misc.ts to eliminate circular dependencies within @sentry/utils
  • Tweak some types related to domains and request data, and polyfill WorkerLocation type

Still to do:

  • Write a better PR description
  • Decide what can (and should) go into the SampleContext object, and put it there
  • Try it out
  • Write tests
  • Deal with circular dependency between packages
  • Fix linting and size check errors
  • Use isNaN in rate validator
  • Make sampler take precedence over parent's decision, pass parent's decision in sampleContext
  • Update docs (including mentioning what data is automatically passed in sampleContext depending on the SDK)

@github-actions
Copy link
Contributor

github-actions bot commented Aug 14, 2020

size-limit report

Path Size
@sentry/browser - CDN Bundle (gzipped) 17.69 KB (+0.1% 🔺)
@sentry/browser - Webpack 18.51 KB (+0.27% 🔺)
@sentry/react - Webpack 18.51 KB (+0.27% 🔺)
@sentry/browser + @sentry/tracing - CDN Bundle (gzipped) 23.47 KB (+2.91% 🔺)

@getsentry-bot
Copy link
Contributor

Warnings
⚠️ Please add a changelog entry for your changes.
Messages
📖

@sentry/browser bundle gzip'ed minified size: (ES5: 17.2637 kB) (ES6: 16.4004 kB)

Generated by 🚫 dangerJS against ba3bc24

@lobsterkatie lobsterkatie force-pushed the kmclb-add-traces-sampler-option branch 3 times, most recently from d157b23 to 0ba634a Compare August 26, 2020 05:03
Copy link
Contributor

@rhcarvalho rhcarvalho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My main concern here is whether we can have a clear and consistent type def for SamplingContext, not an arbitrary mapping of strings to anything.
That should make documentation easier and usage easier.

Looking at the impl. I'm also not convinced it has to be added to startTransaction as a new argument. If we consider Hub is an interface, changing method signatures is a breaking change.
Perhaps this is just because there's no usage example yet -- tests would help clarify what are the use cases being addressed.

tracesSampleRate: 1, // this is just to test, you should lower this in production

// A rate of 1 means all traces will be sent, so it's good for testing.
// In production, you'll likely want to either choose a lower rate or use `tracesSampler` instead.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like how clear and concise this is.

Noting that we may have to apply a similar change to other places that mention tracesSampleRate.
If this type of documentation is only in packages/gatsby, we could also consider replacing it with a link to the official docs, and then maintain only one copy of the example.

In addition, we could provide an example that uses tracesSampler instead of tracesSampleRate, so that users looking at this don't need to go figure out what "use tracesSampler instead" entails.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have updated the gatsby readme and will come back to the question of linking when I write the docs for this.

packages/hub/src/hub.ts Outdated Show resolved Hide resolved
@@ -84,7 +85,7 @@ export interface Options {
* 0.0 = 0% chance of a given trace being sent (send no traces)
* 1.0 = 100% chance of a given trace being sent (send all traces)
*
* Default: 0.0
* Can be omitted without disabling tracing if `tracesSampler` is defined. If neither is defined, tracing is disabled.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Too many conditions and negations for humans to parse here.

Please put tracesSampler close together to this option -- they are related. Then we can explain:

To enable tracing, either set tracesSampleRate to a non-zero value or set tracesSampler.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The linter insists on having all properties before any methods, so I moved tracesSampleRate to be the last property and tracesSampler to be the first method, so they're together. I also reworded the description. LMK if you feel like it's clearer now.

packages/types/src/transaction.ts Show resolved Hide resolved
packages/types/src/options.ts Outdated Show resolved Hide resolved
Comment on lines +110 to +132
// for all node servers that we currently support, we store the incoming request object (which is an instance of
// http.IncomingMessage) on the domain

// the domain members are stored as an array, so our only way to find the request is to iterate through the array
// and compare types
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sounds rather sketchy. Is this existing behavior? Does it work properly under concurrent requests?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is in fact the existing behavior, and my understanding is that scope/thread/concurrency safety the reason we still use the deprecated domain package, because we don't have another way to accomplish that. @kamilogorek is the expert here, though.

packages/tracing/src/hubextensions.ts Outdated Show resolved Hide resolved
const request = domain.members.find((member: any) => isInstanceOf(member, requestType));

if (request) {
defaultSampleContext.request = extractNodeRequestData(request);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we unify the type definition of SampleContext for node + browser?

SampleContext.url = ...
SampleContext.method = ...

What's the advantage of having request in node and location in browser?
The drawback include more complex documentation and less specific type definition for SampleContext (which ties back to documentation and helping people implement their custom samplers).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea is to give people as much data as is easily grabbed. Naturally, that data is going to be different depending on the context - browser window, service worker, or node server.

packages/tracing/test/hub.test.ts Outdated Show resolved Hide resolved
packages/tracing/test/hub.test.ts Outdated Show resolved Hide resolved
@lobsterkatie lobsterkatie force-pushed the kmclb-add-traces-sampler-option branch from 88596ae to 56f8b6e Compare September 2, 2020 01:36
@lobsterkatie
Copy link
Member Author

@rhcarvalho I've tried to address your concerns re: specificity (or lack thereof) in the definition of the SampleContext type.

  1. The type now consists of two halves, CustomSampleContext and added attributes (the defaults) so it's clear that we're combining two data sources. (I originally had the defaults in their own type, DefaultSampleContext, but I moved them to the main SampleContext type so that when you examine the SampleContext type (in VSCode or other), the default stuff shows up right there.)

  2. The defaults now have named keys so it's clear what gets passed automatically.

  3. The third parameter in startTransaction is now called customSampleContext, to make it clear that it's coming from the user.

LMK what you think.

/**
 * Context data passed by the user when starting a transaction, to be used by the tracesSampler method.
 */
export interface CustomSampleContext {
  [key: string]: any;
}

/**
 * Data passed to the `tracesSampler` function, which forms the basis for whatever decisions it might make.
 *
 * Adds default data to data provided by the user. See {@link Hub.startTransaction}
 */
export interface SampleContext extends CustomSampleContext {
  /**
   * Object representing the URL of the current page or worker script. Passed by default in a browser or service worker
   * context.
   */
  location?: Location | WorkerLocation;

  /**
   * Object representing the incoming request to a node server. Passed by default when using the TracingHandler.
   */
  request?: ExtractedNodeRequestData;
}

@lobsterkatie lobsterkatie force-pushed the kmclb-add-traces-sampler-option branch 3 times, most recently from 7150597 to ce227d0 Compare September 4, 2020 03:19
@lobsterkatie lobsterkatie marked this pull request as ready for review September 4, 2020 03:33
Copy link
Member

@HazAT HazAT left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Otherwise LGTM, good PR with a lot of refactorings 👍

Comment on lines +26 to +39
extensions?: {
/** Hack to prevent bundlers from breaking our usage of the domain package in the cross-platform Hub package */
domain?: typeof domain & {
/**
* The currently active domain. This is part of the domain package, but for some reason not declared in the
* package's typedef.
*/
active?: domain.Domain;
};
} & {
/** Extension methods for the hub, which are bound to the current Hub instance */
// eslint-disable-next-line @typescript-eslint/ban-types
[key: string]: Function;
};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

H: Are we sure that doesn't break in Browser environments?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will check again, for sure, but the change here isn't a behavior change but merely a TS one. We've always put the domain there, it's just that before we didn't include it in the type so we had to do a lot of eslint-disable @no-unsafe-member-access and eslint-disable @no-explicit-any. My understanding is that TS stuff all gets stripped?

Anyway, will check and update here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

UPDATE: Seems to build just fine, with no special accommodation in my set up.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lobsterkatie It seems like these changes broke @sentry/browser for TS users by requiring node types 😕 issue: #2927

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See #3050 for a fix.

Copy link
Member

@HazAT HazAT left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One more thing, why did we decide not to provide the Transaction in the SamplingContext?

@lobsterkatie
Copy link
Member Author

lobsterkatie commented Sep 7, 2020

One more thing, why did we decide not to provide the Transaction in the SamplingContext?

Okay, switched from passing parentSampled to passing transactionContext. I didn't pass the transaction itself because I felt like until it is returned from startTransaction(), it shouldn't be able to be mucked with from the outside (like by tracesSampler()).

That said, I think inheritance is a common and important enough concern that I'd like to be able to do better than "this existing decision probably came from the parent, but honestly, we don't know for sure." But in the interests of getting this through, I'll split my proposed solution there into a separate PR.

Okay, I lied. This should get decided before we merge tracesSampler. But to not confuse things too much, for the moment I pulled that work off into a new branch (off of this branch) so we can look at it/talk about it separately. Once we agree on what form it should or shouldn't take, I can meld it into this PR. 24bfd84...5d7845e

* The currently active domain. This is part of the domain package, but for some reason not declared in the
* package's typedef.
*/
active?: domain.Domain;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We shouldn't be using @types/node in an isomorphic package.

@dvins
Copy link

dvins commented Jan 5, 2021

@kamilogorek @lobsterkatie This feature introduced #2971 some months ago, the only solution for many is to down level the package version. It appears to have fallen off the radar despite a number of us customers having this problem. We're now pushing on three months with no feedback, let alone a fix.

@dvins
Copy link

dvins commented Jan 23, 2021

Can someone front Sentry at least provide some feedback, even if it's as trite as "We have no plans to resolve this..." so customers like us can make some decisions? @kamilogorek @lobsterkatie

@rhcarvalho
Copy link
Contributor

@dvins please try https://github.com/getsentry/sentry-javascript/releases/tag/6.0.4. #2971 is now closed, if there's anything else preventing you from upgrading please let us know in a new issue. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants