Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configurable duration (in days) for AB Test cookie. #292

Merged
merged 1 commit into from
Jul 14, 2016
Merged

Configurable duration (in days) for AB Test cookie. #292

merged 1 commit into from
Jul 14, 2016

Conversation

ikennaokpala
Copy link
Contributor

@ikennaokpala ikennaokpala commented Jul 13, 2016

Related to this PR alphagov/smart-answers#2655

Here, I have added a new option (cookieDuration), this option help to set the cookie expiration duration in days.

When this option isn't supplied the the default cookie duration is set to expire in 30 days.

Motivation

The motivation here is, by reducing the cookie expiration will allow returning users to be assigned
to different cohorts during A/B tests.

According to the performance analyst, A/B tests performance is measured using sessions and with
a default 30 day expiration, they will only see the one variant for that period. Being able to
reduce this increases the likelihood of users seeing different variants across their sessions.

Sample code

  new GOVUK.MultivariateTest({
      el: '.get-started a',
      name: 'startButton_201607',
      customDimensionIndex: <customDimensionIndex>,
      cookieDuration: 14, // set to 14 days
      contentExperimentId: <contentExperimentId>,
      cohorts: {
        original: { callback: function() {}, variantId: 0},
        next: { html: 'Next', variantId: 1 },
        continue: { html: 'Continue', variantId: 2 }
      }
    });

Sample outcome

screen shot 2016-07-13 at 15 33 38

@dsingleton
Copy link
Contributor

Thanks @ikennaokpala, being able to set a duration for the experiment is a really useful feature, and was needed recently in some other work. So 👍 to the feature idea.

Could you add a test for the new behaviour? There's some prior art for checking cookie values, so I think it should be easy.

One small, non-blocking, point: Is days is the best unit for the duration? Recently we needed a duration in minutes, rather than days, which was possible with Maths, but makes me wonder if we should use seconds, would that be more consistent?

If there's no consensus on that, i'd be more than happy to merge with days, but worth discussing until theres tests.

Here, I have added a new option (cookieDuration), this option help to set the cookie expiration duration in days.

When this option isn't supplied the the default cookie duration is set to expire in 30 days.

The motivation here is, by reducing the cookie expiration will allow returning users to be assigned
to different cohorts during A/B tests.

According to the performance analyst, A/B tests performance is measured using sessions and with a default 30 day expiration,
they will only see the one variant for that period.  Being able to reduce this increases the likelihood of users seeing different variants
across their sessions.
@dsingleton dsingleton merged commit 058e05c into alphagov:master Jul 14, 2016
@dsingleton
Copy link
Contributor

👍 i'm fine with the duration being in days, it's a sensible default and if someone needs to set a lower duration then the caller can calculate a fraction value.

@ikennaokpala ikennaokpala deleted the configurable-cookie-expiration branch July 14, 2016 14:56
ikennaokpala added a commit to alphagov/smart-answers that referenced this pull request Jul 22, 2016
This commit bumps the front-end toolkit from 4.8.0 to 4.14.0 as a result
of the recent update to the front-end toolkit.

* alphagov/govuk_frontend_toolkit#296
* alphagov/govuk_frontend_toolkit#294
* alphagov/govuk_frontend_toolkit#292
ikennaokpala added a commit to alphagov/smart-answers that referenced this pull request Jul 26, 2016
This commit bumps the front-end toolkit from 4.8.0 to 4.14.0 as a result
of the recent update to the front-end toolkit.

* alphagov/govuk_frontend_toolkit#296
* alphagov/govuk_frontend_toolkit#294
* alphagov/govuk_frontend_toolkit#292
ikennaokpala added a commit to alphagov/smart-answers that referenced this pull request Jul 26, 2016
This commit bumps the front-end toolkit from 4.8.0 to 4.14.0 as a result
of the recent update to the front-end toolkit.

* alphagov/govuk_frontend_toolkit#296
* alphagov/govuk_frontend_toolkit#294
* alphagov/govuk_frontend_toolkit#292

SmartAnswers makes use of the following files from the frontend_toolkit:
Stylesheets:
- _conditionals.scss
- _typography.scss
- _colours.scss
- _shims.scss
- design-patterns/_buttons.scss
- _grid_layout.scss

Javascript:
- govuk/multivariate-test

Having reviewed the diff between 4.8.0 and 4.14.0 alphagov/govuk_frontend_toolkit@0cfc216...37d23c1.

This diff composes of 82 commits, 34 Merge Commits and 48 concrete commits.

I gone through each tracing the possible areas that this might affect smart answers. To the best of my knowledge I have found only cosmetic changes to the following files.

Stylesheets:
- _typography.scss e05ca517 , 70f53d27c
- _colors.scss 11798945a , 3853a726c , 0052024b4b9 , 984d53292 , 2180a72e4

UI examination was carried by Anika and myself and we found nothing of concern.

Already existing unit, regression and integration tests passed, giving assurance
that the aforementioned commit/changes have had no (interfering) affect to the
transit route or flow or other routes within check uk visa or other smart answers.
ikennaokpala added a commit to alphagov/smart-answers that referenced this pull request Jul 27, 2016
This commit bumps the front-end toolkit from 4.8.0 to 4.14.0 as a result
of the recent update to the front-end toolkit.

* alphagov/govuk_frontend_toolkit#296
* alphagov/govuk_frontend_toolkit#294
* alphagov/govuk_frontend_toolkit#292

SmartAnswers makes use of the following files from the frontend_toolkit:
Stylesheets:
- _conditionals.scss
- _typography.scss
- _colours.scss
- _shims.scss
- design-patterns/_buttons.scss
- _grid_layout.scss

Javascript:
- govuk/multivariate-test

Having reviewed the diff between 4.8.0 and 4.14.0 alphagov/govuk_frontend_toolkit@0cfc216...37d23c1.

This diff composes of 82 commits, 34 Merge Commits and 48 concrete commits.

I gone through each tracing the possible areas that this might affect smart answers. To the best of my knowledge I have found only cosmetic changes to the following files.

Stylesheets:
- _typography.scss e05ca517 , 70f53d27c
- _colors.scss 11798945a , 3853a726c , 0052024b4b9 , 984d53292 , 2180a72e4

UI examination was carried by Anika and myself and we found nothing of concern.

Already existing unit, regression and integration tests passed, giving assurance
that the aforementioned commit/changes have had no (interfering) affect to the
transit route or flow or other routes within check uk visa or other smart answers.
ikennaokpala added a commit to alphagov/smart-answers that referenced this pull request Jul 27, 2016
This commit bumps the front-end toolkit from 4.8.0 to 4.14.0 as a result
of the recent update to the front-end toolkit.

* alphagov/govuk_frontend_toolkit#296
* alphagov/govuk_frontend_toolkit#294
* alphagov/govuk_frontend_toolkit#292

SmartAnswers makes use of the following files from the frontend_toolkit:
Stylesheets:
- _conditionals.scss
- _typography.scss
- _colours.scss
- _shims.scss
- design-patterns/_buttons.scss
- _grid_layout.scss

Javascript:
- govuk/multivariate-test

Having reviewed the diff between 4.8.0 and 4.14.0 alphagov/govuk_frontend_toolkit@0cfc216...37d23c1.

This diff composes of 82 commits, 34 Merge Commits and 48 concrete commits.

I gone through each tracing the possible areas that this might affect smart answers. To the best of my knowledge I have found only cosmetic changes to the following files.

Stylesheets:
- _typography.scss e05ca517 , 70f53d27c
- _colors.scss 11798945a , 3853a726c , 0052024b4b9 , 984d53292 , 2180a72e4

UI examination was carried by Anika and myself and we found nothing of concern.

Already existing unit, regression and integration tests passed, giving assurance
that the aforementioned commit/changes have had no (interfering) affect to the
transit route or flow or other routes within check uk visa or other smart answers.
ikennaokpala added a commit to alphagov/smart-answers that referenced this pull request Jul 27, 2016
This commit bumps the front-end toolkit from 4.8.0 to 4.14.0 as a result
of the recent update to the front-end toolkit.

* alphagov/govuk_frontend_toolkit#296
* alphagov/govuk_frontend_toolkit#294
* alphagov/govuk_frontend_toolkit#292

SmartAnswers makes use of the following files from the frontend_toolkit:
Stylesheets:
- _conditionals.scss
- _typography.scss
- _colours.scss
- _shims.scss
- design-patterns/_buttons.scss
- _grid_layout.scss

Javascript:
- govuk/multivariate-test

Having reviewed the diff between 4.8.0 and 4.14.0 alphagov/govuk_frontend_toolkit@0cfc216...37d23c1.

This diff composes of 82 commits, 34 Merge Commits and 48 concrete commits.

I gone through each tracing the possible areas that this might affect smart answers. To the best of my knowledge I have found only cosmetic changes to the following files.

Stylesheets:
- _typography.scss e05ca517 , 70f53d27c
- _colors.scss 11798945a , 3853a726c , 0052024b4b9 , 984d53292 , 2180a72e4

UI examination was carried by Anika and myself and we found nothing of concern.

Already existing unit, regression and integration tests passed, giving assurance
that the aforementioned commit/changes have had no (interfering) affect to the
transit route or flow or other routes within check uk visa or other smart answers.
ikennaokpala added a commit to alphagov/smart-answers that referenced this pull request Jul 28, 2016
This commit bumps the front-end toolkit from 4.8.0 to 4.14.0 as a result
of the recent update to the front-end toolkit.

* alphagov/govuk_frontend_toolkit#296
* alphagov/govuk_frontend_toolkit#294
* alphagov/govuk_frontend_toolkit#292

SmartAnswers makes use of the following files from the frontend_toolkit:
Stylesheets:
- _conditionals.scss
- _typography.scss
- _colours.scss
- _shims.scss
- design-patterns/_buttons.scss
- _grid_layout.scss

Javascript:
- govuk/multivariate-test

Having reviewed the diff between 4.8.0 and 4.14.0 alphagov/govuk_frontend_toolkit@0cfc216...37d23c1.

This diff composes of 82 commits, 34 Merge Commits and 48 concrete commits.

I gone through each tracing the possible areas that this might affect smart answers. To the best of my knowledge I have found only cosmetic changes to the following files.

Stylesheets:
- _typography.scss e05ca517 , 70f53d27c
- _colors.scss 11798945a , 3853a726c , 0052024b4b9 , 984d53292 , 2180a72e4

UI examination was carried by Anika and myself and we found nothing of concern.

Already existing unit, regression and integration tests passed, giving assurance
that the aforementioned commit/changes have had no (interfering) affect to the
transit route or flow or other routes within check uk visa or other smart answers.
ikennaokpala added a commit to alphagov/smart-answers that referenced this pull request Jul 28, 2016
This commit bumps the front-end toolkit from 4.8.0 to 4.14.0 as a result
of the recent update to the front-end toolkit.

* alphagov/govuk_frontend_toolkit#296
* alphagov/govuk_frontend_toolkit#294
* alphagov/govuk_frontend_toolkit#292

SmartAnswers makes use of the following files from the frontend_toolkit:
Stylesheets:
- _conditionals.scss
- _typography.scss
- _colours.scss
- _shims.scss
- design-patterns/_buttons.scss
- _grid_layout.scss

Javascript:
- govuk/multivariate-test

Having reviewed the diff between 4.8.0 and 4.14.0 alphagov/govuk_frontend_toolkit@0cfc216...37d23c1.

This diff composes of 82 commits, 34 Merge Commits and 48 concrete commits.

I gone through each tracing the possible areas that this might affect smart answers. To the best of my knowledge I have found only cosmetic changes to the following files.

Stylesheets:
- _typography.scss e05ca517 , 70f53d27c
- _colors.scss 11798945a , 3853a726c , 0052024b4b9 , 984d53292 , 2180a72e4

UI examination was carried by Anika and myself and we found nothing of concern.

Already existing unit, regression and integration tests passed, giving assurance
that the aforementioned commit/changes have had no (interfering) affect to the
transit route or flow or other routes within check uk visa or other smart answers.
ikennaokpala added a commit to alphagov/smart-answers that referenced this pull request Jul 28, 2016
This commit bumps the front-end toolkit from 4.8.0 to 4.14.0 as a result
of the recent update to the front-end toolkit.

* alphagov/govuk_frontend_toolkit#296
* alphagov/govuk_frontend_toolkit#294
* alphagov/govuk_frontend_toolkit#292

SmartAnswers makes use of the following files from the frontend_toolkit:
Stylesheets:
- _conditionals.scss
- _typography.scss
- _colours.scss
- _shims.scss
- design-patterns/_buttons.scss
- _grid_layout.scss

Javascript:
- govuk/multivariate-test

Having reviewed the diff between 4.8.0 and 4.14.0 alphagov/govuk_frontend_toolkit@0cfc216...37d23c1.

This diff composes of 82 commits, 34 Merge Commits and 48 concrete commits.

I gone through each tracing the possible areas that this might affect smart answers. To the best of my knowledge I have found only cosmetic changes to the following files.

Stylesheets:
- _typography.scss e05ca517 , 70f53d27c
- _colors.scss 11798945a , 3853a726c , 0052024b4b9 , 984d53292 , 2180a72e4

UI examination was carried by Anika and myself and we found nothing of concern.

Already existing unit, regression and integration tests passed, giving assurance
that the aforementioned commit/changes have had no (interfering) affect to the
transit route or flow or other routes within check uk visa or other smart answers.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants