Campaigns Testing Sequence

Campaigns functional tests (h/t/ @dkoo)

Test 1

Test 1 conditions

• Campaign A: Overlay campaign, sitewide default, contains newsletter prompt
• Campaign B: Inline campaign, displays every page, contains newsletter prompt
• Campaign C: Inline campaign, displays every page, contains newsletter prompt
• Campaign D: Inline campaign, displays every page, does NOT contain newsletter prompt
• Suppress Newsletter campaigns if visitor is coming from email: ENABLED
• Suppress all Newsletter campaigns if at least one Newsletter campaign was permanently dismissed: ENABLED

Test 1 results

• When viewing the site for the first time in a new incognito window: all campaigns are displayed, as expected.
• After refreshing the page, without interacting with any campaigns: all campaigns still show, as expected.
• After dismissing Campaign D (no newsletter prompt), it does not reappear after refreshing the page. Campaigns A-C do appear as expected.
• NOT WORKING: After dismissing Campaign A, B, or C (contains a newsletter prompt): I encounter the utm_suppression notice highlighted below on subsequent page refreshes, and the other newsletter prompt campaigns are not suppressed.
• Loading a page in a new incognito window with the utm_medium=email param: Campaigns A-C are suppressed, while Campaign D is not, as expected.
• After navigating to another page, without the utm_medium param: Campaigns A-C are still not displayed, as expected.

Test 2

Test 2 conditions

• Campaigns A-D: Same as in Test 1
• Suppress Newsletter campaigns if visitor is coming from email: DISABLED
• Suppress all Newsletter campaigns if at least one Newsletter campaign was permanently dismissed: ENABLED

Test 2 results

• Loading a page in a new incognito window with the utm_medium=email param: All campaigns are displayed, as expected.

Test 3

Test 3 conditions

• Campaigns A-D: Same as in Tests 1-2
• Suppress Newsletter campaigns if visitor is coming from email: ENABLED
• Suppress all Newsletter campaigns if at least one Newsletter campaign was permanently dismissed: DISABLED

Test 3 results

• Loading a page in a new incognito window with the utm_medium=email param: Campaigns A-C are not displayed, while Campaign D, as expected.
• After navigating to another page, without the utm_medium param: Campaigns A-C are still not displayed, while Campaign D is, as expected.
• Loading a page in a new incognito window without the utm_medium=email param: All campaigns are displayed, as expected.
• After dismissing Campaign A, B, or C (contains a newsletter prompt): Only the dismissed campaign is suppressed on subsequent page loads, as expected.

Test 4

Test 4 conditions

• Campaigns A-C: Same as in Tests 1-3
• Campaign D: Inline campaign, displays once a day, no newsletter prompt

Test 4 results

• In a new incognito window, all campaigns are displayed, as expected.
• After navigating to a new page: Campaigns A-C are displayed, Campaign D is not, as expected.
• NOT WORKING: If the first page visited is an AMP page and the second page is not (or vice versa), Campaign D is still displayed on the second page (but is not on subsequent pages—seems to be that the suppression is just not remembered across AMP vs. non-AMP sessions).

Test 5

Test 5 conditions

• Campaign A: Overlay campaign, sitewide default, contains newsletter prompt, TEST MODE
• Campaigns B-D: Same as in Tests 1-3

Test 5 results

• In a new incognito window: Campaign A is not displayed, Campaigns B-D are displayed, as expected.
• After logging into WP Admin: Campaign A is displayed, Campaigns B-D are not, as expected.

Test 6

Test 6 conditions:

• Campaigns A-C: Same as in Tests 1-3
• Campaign D: UTM Suppression field set to "suppress this"

Test 6 results:

• In a new incognito window, all campaigns are displayed, as expected.
• When visting the page with the utm_source=suppress+this or utm_source=suppress%20this param, Campaigns A-C are displayed, while Campaign D is not, as expected.
• NOT WORKING (?): After navigating to a new page, without the utm_source param: Campaign D is displayed again. Expected: the suppression should be remembered across pages once it happens the first time. Marked (?) as I'm not sure whether my expectation is correct?