-
Notifications
You must be signed in to change notification settings - Fork 104
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Configurable duration (in days) for AB Test cookie. #292
Configurable duration (in days) for AB Test cookie. #292
Conversation
Thanks @ikennaokpala, being able to set a duration for the experiment is a really useful feature, and was needed recently in some other work. So 👍 to the feature idea. Could you add a test for the new behaviour? There's some prior art for checking cookie values, so I think it should be easy. One small, non-blocking, point: Is days is the best unit for the duration? Recently we needed a duration in minutes, rather than days, which was possible with Maths, but makes me wonder if we should use seconds, would that be more consistent? If there's no consensus on that, i'd be more than happy to merge with days, but worth discussing until theres tests. |
Here, I have added a new option (cookieDuration), this option help to set the cookie expiration duration in days. When this option isn't supplied the the default cookie duration is set to expire in 30 days. The motivation here is, by reducing the cookie expiration will allow returning users to be assigned to different cohorts during A/B tests. According to the performance analyst, A/B tests performance is measured using sessions and with a default 30 day expiration, they will only see the one variant for that period. Being able to reduce this increases the likelihood of users seeing different variants across their sessions.
👍 i'm fine with the duration being in days, it's a sensible default and if someone needs to set a lower duration then the caller can calculate a fraction value. |
This commit bumps the front-end toolkit from 4.8.0 to 4.14.0 as a result of the recent update to the front-end toolkit. * alphagov/govuk_frontend_toolkit#296 * alphagov/govuk_frontend_toolkit#294 * alphagov/govuk_frontend_toolkit#292
This commit bumps the front-end toolkit from 4.8.0 to 4.14.0 as a result of the recent update to the front-end toolkit. * alphagov/govuk_frontend_toolkit#296 * alphagov/govuk_frontend_toolkit#294 * alphagov/govuk_frontend_toolkit#292
This commit bumps the front-end toolkit from 4.8.0 to 4.14.0 as a result of the recent update to the front-end toolkit. * alphagov/govuk_frontend_toolkit#296 * alphagov/govuk_frontend_toolkit#294 * alphagov/govuk_frontend_toolkit#292 SmartAnswers makes use of the following files from the frontend_toolkit: Stylesheets: - _conditionals.scss - _typography.scss - _colours.scss - _shims.scss - design-patterns/_buttons.scss - _grid_layout.scss Javascript: - govuk/multivariate-test Having reviewed the diff between 4.8.0 and 4.14.0 alphagov/govuk_frontend_toolkit@0cfc216...37d23c1. This diff composes of 82 commits, 34 Merge Commits and 48 concrete commits. I gone through each tracing the possible areas that this might affect smart answers. To the best of my knowledge I have found only cosmetic changes to the following files. Stylesheets: - _typography.scss e05ca517 , 70f53d27c - _colors.scss 11798945a , 3853a726c , 0052024b4b9 , 984d53292 , 2180a72e4 UI examination was carried by Anika and myself and we found nothing of concern. Already existing unit, regression and integration tests passed, giving assurance that the aforementioned commit/changes have had no (interfering) affect to the transit route or flow or other routes within check uk visa or other smart answers.
This commit bumps the front-end toolkit from 4.8.0 to 4.14.0 as a result of the recent update to the front-end toolkit. * alphagov/govuk_frontend_toolkit#296 * alphagov/govuk_frontend_toolkit#294 * alphagov/govuk_frontend_toolkit#292 SmartAnswers makes use of the following files from the frontend_toolkit: Stylesheets: - _conditionals.scss - _typography.scss - _colours.scss - _shims.scss - design-patterns/_buttons.scss - _grid_layout.scss Javascript: - govuk/multivariate-test Having reviewed the diff between 4.8.0 and 4.14.0 alphagov/govuk_frontend_toolkit@0cfc216...37d23c1. This diff composes of 82 commits, 34 Merge Commits and 48 concrete commits. I gone through each tracing the possible areas that this might affect smart answers. To the best of my knowledge I have found only cosmetic changes to the following files. Stylesheets: - _typography.scss e05ca517 , 70f53d27c - _colors.scss 11798945a , 3853a726c , 0052024b4b9 , 984d53292 , 2180a72e4 UI examination was carried by Anika and myself and we found nothing of concern. Already existing unit, regression and integration tests passed, giving assurance that the aforementioned commit/changes have had no (interfering) affect to the transit route or flow or other routes within check uk visa or other smart answers.
This commit bumps the front-end toolkit from 4.8.0 to 4.14.0 as a result of the recent update to the front-end toolkit. * alphagov/govuk_frontend_toolkit#296 * alphagov/govuk_frontend_toolkit#294 * alphagov/govuk_frontend_toolkit#292 SmartAnswers makes use of the following files from the frontend_toolkit: Stylesheets: - _conditionals.scss - _typography.scss - _colours.scss - _shims.scss - design-patterns/_buttons.scss - _grid_layout.scss Javascript: - govuk/multivariate-test Having reviewed the diff between 4.8.0 and 4.14.0 alphagov/govuk_frontend_toolkit@0cfc216...37d23c1. This diff composes of 82 commits, 34 Merge Commits and 48 concrete commits. I gone through each tracing the possible areas that this might affect smart answers. To the best of my knowledge I have found only cosmetic changes to the following files. Stylesheets: - _typography.scss e05ca517 , 70f53d27c - _colors.scss 11798945a , 3853a726c , 0052024b4b9 , 984d53292 , 2180a72e4 UI examination was carried by Anika and myself and we found nothing of concern. Already existing unit, regression and integration tests passed, giving assurance that the aforementioned commit/changes have had no (interfering) affect to the transit route or flow or other routes within check uk visa or other smart answers.
This commit bumps the front-end toolkit from 4.8.0 to 4.14.0 as a result of the recent update to the front-end toolkit. * alphagov/govuk_frontend_toolkit#296 * alphagov/govuk_frontend_toolkit#294 * alphagov/govuk_frontend_toolkit#292 SmartAnswers makes use of the following files from the frontend_toolkit: Stylesheets: - _conditionals.scss - _typography.scss - _colours.scss - _shims.scss - design-patterns/_buttons.scss - _grid_layout.scss Javascript: - govuk/multivariate-test Having reviewed the diff between 4.8.0 and 4.14.0 alphagov/govuk_frontend_toolkit@0cfc216...37d23c1. This diff composes of 82 commits, 34 Merge Commits and 48 concrete commits. I gone through each tracing the possible areas that this might affect smart answers. To the best of my knowledge I have found only cosmetic changes to the following files. Stylesheets: - _typography.scss e05ca517 , 70f53d27c - _colors.scss 11798945a , 3853a726c , 0052024b4b9 , 984d53292 , 2180a72e4 UI examination was carried by Anika and myself and we found nothing of concern. Already existing unit, regression and integration tests passed, giving assurance that the aforementioned commit/changes have had no (interfering) affect to the transit route or flow or other routes within check uk visa or other smart answers.
This commit bumps the front-end toolkit from 4.8.0 to 4.14.0 as a result of the recent update to the front-end toolkit. * alphagov/govuk_frontend_toolkit#296 * alphagov/govuk_frontend_toolkit#294 * alphagov/govuk_frontend_toolkit#292 SmartAnswers makes use of the following files from the frontend_toolkit: Stylesheets: - _conditionals.scss - _typography.scss - _colours.scss - _shims.scss - design-patterns/_buttons.scss - _grid_layout.scss Javascript: - govuk/multivariate-test Having reviewed the diff between 4.8.0 and 4.14.0 alphagov/govuk_frontend_toolkit@0cfc216...37d23c1. This diff composes of 82 commits, 34 Merge Commits and 48 concrete commits. I gone through each tracing the possible areas that this might affect smart answers. To the best of my knowledge I have found only cosmetic changes to the following files. Stylesheets: - _typography.scss e05ca517 , 70f53d27c - _colors.scss 11798945a , 3853a726c , 0052024b4b9 , 984d53292 , 2180a72e4 UI examination was carried by Anika and myself and we found nothing of concern. Already existing unit, regression and integration tests passed, giving assurance that the aforementioned commit/changes have had no (interfering) affect to the transit route or flow or other routes within check uk visa or other smart answers.
This commit bumps the front-end toolkit from 4.8.0 to 4.14.0 as a result of the recent update to the front-end toolkit. * alphagov/govuk_frontend_toolkit#296 * alphagov/govuk_frontend_toolkit#294 * alphagov/govuk_frontend_toolkit#292 SmartAnswers makes use of the following files from the frontend_toolkit: Stylesheets: - _conditionals.scss - _typography.scss - _colours.scss - _shims.scss - design-patterns/_buttons.scss - _grid_layout.scss Javascript: - govuk/multivariate-test Having reviewed the diff between 4.8.0 and 4.14.0 alphagov/govuk_frontend_toolkit@0cfc216...37d23c1. This diff composes of 82 commits, 34 Merge Commits and 48 concrete commits. I gone through each tracing the possible areas that this might affect smart answers. To the best of my knowledge I have found only cosmetic changes to the following files. Stylesheets: - _typography.scss e05ca517 , 70f53d27c - _colors.scss 11798945a , 3853a726c , 0052024b4b9 , 984d53292 , 2180a72e4 UI examination was carried by Anika and myself and we found nothing of concern. Already existing unit, regression and integration tests passed, giving assurance that the aforementioned commit/changes have had no (interfering) affect to the transit route or flow or other routes within check uk visa or other smart answers.
This commit bumps the front-end toolkit from 4.8.0 to 4.14.0 as a result of the recent update to the front-end toolkit. * alphagov/govuk_frontend_toolkit#296 * alphagov/govuk_frontend_toolkit#294 * alphagov/govuk_frontend_toolkit#292 SmartAnswers makes use of the following files from the frontend_toolkit: Stylesheets: - _conditionals.scss - _typography.scss - _colours.scss - _shims.scss - design-patterns/_buttons.scss - _grid_layout.scss Javascript: - govuk/multivariate-test Having reviewed the diff between 4.8.0 and 4.14.0 alphagov/govuk_frontend_toolkit@0cfc216...37d23c1. This diff composes of 82 commits, 34 Merge Commits and 48 concrete commits. I gone through each tracing the possible areas that this might affect smart answers. To the best of my knowledge I have found only cosmetic changes to the following files. Stylesheets: - _typography.scss e05ca517 , 70f53d27c - _colors.scss 11798945a , 3853a726c , 0052024b4b9 , 984d53292 , 2180a72e4 UI examination was carried by Anika and myself and we found nothing of concern. Already existing unit, regression and integration tests passed, giving assurance that the aforementioned commit/changes have had no (interfering) affect to the transit route or flow or other routes within check uk visa or other smart answers.
Related to this PR alphagov/smart-answers#2655
Here, I have added a new option (cookieDuration), this option help to set the cookie expiration duration in days.
When this option isn't supplied the the default cookie duration is set to expire in 30 days.
Motivation
The motivation here is, by reducing the cookie expiration will allow returning users to be assigned
to different cohorts during A/B tests.
According to the performance analyst, A/B tests performance is measured using sessions and with
a default 30 day expiration, they will only see the one variant for that period. Being able to
reduce this increases the likelihood of users seeing different variants across their sessions.
Sample code
Sample outcome