Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add sizing warning for single feature experiments #9641

Open
eliserichards opened this issue Oct 26, 2023 · 0 comments
Open

Add sizing warning for single feature experiments #9641

eliserichards opened this issue Oct 26, 2023 · 0 comments

Comments

@eliserichards
Copy link
Contributor

eliserichards commented Oct 26, 2023

Thread in #ask-experimenter for context

When one or more multifeature experiments are Live and a user goes to launch a new single feature experiment, they will need to adjust their population sizing to account for the users that are disqualified from being in the multifeature experiments.

We should add an explanation to experimenter.info (mozilla/experimenter-docs#521) and then create a warning (similar to this):

  • For users launching a single feature experiment where the targeted feature already has multifeature experiment(s) that are Live

  • Use the following formula to adjust the sizing:

    if data scientist run normal sizing for feature in “url bar” and get x% as desired typically, but if y% meeting the targeting criterion has been taken by active multiple-feature experiment where ‘url bar’ is one of the feature, they should then use x%/(1-y%) instead?

┆Issue is synchronized with this Jira Task

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant