My Site Dashboard: add AB test after site creation and option to change the default in Settings

[leandroalonso]

Part of #17873

• Add AB test setup after site creation flow to enable Home or Site Menu as the default
• Add an option in Settings to change the initial screen to Home or Site Menu

Simulator.Screen.Recording.-.iPhone.13.-.2022-03-11.at.14.13.08.mp4

Additional info

The AB calculation is based on the token and it's the same used for the Empty Stats project.

To test

Do a clean install of the app and don't enable the My Site Dashboard feature flag. Then:

1. Login or create a new account
2. Go to Settings
3. ✅ You shouldn't see an "Initial Screen" option

Enable the My Site Dashboard feature flag:

1. Go to Settings
2. ✅ You should see an "Initial Screen" option
3. Change the value
4. Make sure that an event is fired on the console, like: 🔵 Tracked: app_settings_initial_screen_changed <selected: site_menu>
5. Close and reopen the app
6. ✅ Make sure the option you selected is now the default

For the AB flow, there's no guarantee the Dashboard will be displayed for you. You can create a new account and site and test if it's displayed (or not).

If the app_settings_initial_screen_changed event is fired correctly, please validate it on Tracks.

Regression Notes

1. Potential unintended areas of impact
   n/a

2. What I did to test those areas of impact (or what existing automated tests I relied on)
   n/a

3. What automated tests I added (or what prevented me from doing so)
   n/a

PR submission checklist:

• I have completed the Regression Notes.
• I have considered adding unit tests for my changes.
• I have considered adding accessibility improvements for my changes.
• I have considered if this change warrants user-facing release notes and have added them to RELEASE-NOTES.txt if necessary.

[wpmobilebot]

You can test the Jetpack changes on this Pull Request by downloading it from AppCenter here with build number: pr18125-9297cea. IPA is available here. If you need access to this, you can ask a maintainer to add you.

[wpmobilebot]

You can test the WordPress changes on this Pull Request by downloading it from AppCenter here with build number: pr18125-9297cea. IPA is available here. If you need access to this, you can ask a maintainer to add you.

[hassaanelgarem]

Works as described 🚀

One rookie question though: Why are we not using ExPlat (Abacus) for the AB experiment?

[momo-ozawa]

@hassaanelgarem Good question!

From pbArwn-284-p2:

Initially we wanted to use ExPlat to [run an A/B test where we would make the feature available to 50% of our users], but it wasn’t fully ready for us at the time [...] we eventually decided to go with a simple, yet more reliable, solution where we assign a variation to the user based on their authentication token

I haven't checked the status of A/B tests on ExPlat, but this is why we've been using the auth token.

[momo-ozawa]

Works as described! I validated the app_settings_initial_screen_changed event on Tracks ✅