Bug 1622800 - part 11 to 14: Enable all locales and create hook #6459

JohanLorenzo · 2020-04-24T16:44:24Z

Thank you for submitting a pull request, your contributions are greatly appreciated!

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

Please make sure you've run the test scheme and all the tests pass. If your changes affect existing tests please make sure to update the tests as well.

… jobs

JohanLorenzo · 2020-04-30T18:17:51Z

.cron.yml

+      job:
+          type: decision-task
+          target-tasks-method: l10n_screenshots
+      when: []  # Manual trigger only


I will give the link later after this lands.

Here's the link https://firefox-ci-tc.services.mozilla.com/hooks/project-releng/cron-task-mozilla-mobile-firefox-ios%2Fl10-screenshots.

It doesn't work yet (see #6529). I'll send this link out once I know it's working.

JohanLorenzo · 2020-04-30T18:18:04Z

.taskcluster.yml

@@ -256,7 +256,7 @@ tasks:
                                            ~/.local/bin/taskgraph action-callback
                                        else: >
                                            PIP_IGNORE_INSTALLED=0 pip install --user /builds/worker/checkouts/taskgraph &&
-                                            PIP_IGNORE_INSTALLED=0 pip install --user arrow taskcluster pyyaml &&


These dependencies aren't used.

JohanLorenzo · 2020-04-30T18:18:49Z

l10n-screenshots.sh

@@ -21,7 +31,7 @@ fi

 LOCALES=$*
 if [ $# -eq 0 ]; then
-  LOCALES="af an anp ar ast az bg bn bo br bs ca co cs cy da de dsb el en-CA en-GB en-US eo es-AR es-CL es-MX es eu fa fi fr ga-IE gd gl gu-IN he hi-IN hr hsb hu hy-AM ia id is it ja jv ka kab kk km kn ko lo lt lv ml mr ms my nb-NO ne-NP nl nn-NO oc or pa-IN pl pt-BR pt-PT rm ro ru ses si sk sl sq su sv-SE ta te th tl tr uk ur uz vi zgh zh-CN zh-TW"


This list has been transferred to l10n-screenshots-locales.txt so it can be used by taskgraph.

JohanLorenzo · 2020-04-30T18:19:11Z

taskcluster/ci/build/kind.yml

@@ -13,9 +13,10 @@ transforms:

 jobs:
    screenshots:
+        attributes:
+            chunk_locales: ["en-US"]


This value is used by the bitrise transform down below.

JohanLorenzo · 2020-04-30T18:22:37Z

taskcluster/ci/config.yml

            provisioner: 'mobile-{level}'
            implementation: docker-worker
            os: linux
-            worker-type: 'b-linux'
+            worker-type: 'bitrise'


I created this new worker-type. It's a type defined for taskcluster. It currently maps our capacity in bitrise https://hg.mozilla.org/ci/ci-configuration/rev/39f5c268f49092e33278ee1ddd2bdf0cfb09db73#l1.33. If I kept using b-linux, we would end up with 10 taskcluster workers all delegating work to bitrise as the same time. This means 7 of these workers would be polling Bitrise while knowing Bitrise not doing anything for them.

JohanLorenzo · 2020-04-30T18:29:41Z

taskcluster/ci/generate-screenshots/kind.yml

+not-for-locales:
+    - en-US   # Already built as part of the build task
+
+locales-per-chunk: 6


Here's how I came up with this number.

Our bitrise plan have a 90-minute-limit. Thanks to @isabelrios' great work in #6420, 6 locales takes between 50 to 60 minutes to complete when everything is fine. If some tests fail, xcode will rerun the failed tests, adding more minutes to the counter.

A 7th locale put some of jobs to 70 minutes, which only gives 20 minutes in case of a problem. So I think 6 locales is a sweet spot.

We have 92 locales which yields 16 chunks. Our bitrise plan has 3 concurrent builds. So, the full suite will take about 6 hours to complete. I think it's a great improvement based on the current situation 😃

Yeah, I think so too...the time spent is reduced by more than half of the current time needed...

JohanLorenzo · 2020-04-30T18:30:10Z

taskcluster/ffios_taskgraph/loader/screenshots_locales.py

+from ..screenshots_locales import get_screenshots_locales
+
+
+def loader(kind, path, config, params, loaded_tasks):


This function loads all locales and chunks them into jobs of 6 locales.

JohanLorenzo · 2020-04-30T18:32:42Z

taskcluster/ffios_taskgraph/transforms/bitrise.py


        worker.setdefault("docker-image", {"in-tree": "screenshots"})
-        worker.setdefault("max-run-time", 3600)
+        worker.setdefault("max-run-time", 10800)


I set the max-run-time to 3 hours. Basically, we may end up in situation where a TC worker schedules a job to bitrise, but the latter is overloaded (say because of the smoketests jobs which aren't monitored by Taskcluster). So the TC worker will schedule a job then wait for another to finish. Since a job is max 90 minutes, I set the limit to twice that value.

JohanLorenzo · 2020-04-30T18:33:50Z

taskcluster/scripts/bitrise-schedule.py

@@ -50,7 +50,7 @@ def sync_main(
    parser.add_argument("--branch", required=True, help="the git branch to generate screenshots from")
    parser.add_argument("--commit", required=True, help="the git commit hash to generate screenshots from")
    parser.add_argument("--workflow", required=True, help="the bitrise workflow to schedule")
-    parser.add_argument("--locale", required=True, help="locale to generate the screenshots for")
+    parser.add_argument("--locale", dest="locales", metavar="LOCALE", action="append", required=True, help="locale to generate the screenshots for (can be repeated)")


This change enables the script to be called this way: bitrise-schedule.py --locale en-CA --locale es --locale fr

JohanLorenzo · 2020-04-30T18:34:52Z

taskcluster/scripts/bitrise-schedule.py

@@ -173,9 +175,11 @@ async def download_log(client, build_slug):

    response = await do_http_request_json(client, url)
    download_url = response["expiring_raw_log_url"]
-    if not download_url:
-        raise TaskException("Bitrise has no log to offer for job {}. Please check https://app.bitrise.io/app/{}".format(build_slug, BITRISE_APP_SLUG_ID))


Because of this line, I ended up with a red job, even though Bitrise generated all locales. Their API just failed to provide us a link to their logs.

I also fixed the URL to point to the build, directly.

JohanLorenzo · 2020-04-30T18:36:48Z

This was tested (with a limited number of locales) at https://firefox-ci-tc.services.mozilla.com/tasks/TBzi794DR0yeZ_OVS7HyAQ/runs/2.

The next step to me, is to see if how it works in production 😃

Please let me know if you have any questions, I'm happy to explain anything 🙂

isabelrios

LGTM! amazing work!! :)

isabelrios · 2020-05-04T13:57:12Z

l10n-screenshots.sh

@@ -21,7 +31,7 @@ fi

 LOCALES=$*
 if [ $# -eq 0 ]; then
-  LOCALES="af an anp ar ast az bg bn bo br bs ca co cs cy da de dsb el en-CA en-GB en-US eo es-AR es-CL es-MX es eu fa fi fr ga-IE gd gl gu-IN he hi-IN hr hsb hu hy-AM ia id is it ja jv ka kab kk km kn ko lo lt lv ml mr ms my nb-NO ne-NP nl nn-NO oc or pa-IN pl pt-BR pt-PT rm ro ru ses si sk sl sq su sv-SE ta te th tl tr uk ur uz vi zgh zh-CN zh-TW"
+  LOCALES=$(cat "$PROJECT_DIR/l10n-screenshots-locales.txt")


isabelrios · 2020-05-04T14:01:41Z

taskcluster/ci/generate-screenshots/kind.yml

+not-for-locales:
+    - en-US   # Already built as part of the build task
+
+locales-per-chunk: 6


Yeah, I think so too...the time spent is reduced by more than half of the current time needed...

JohanLorenzo force-pushed the bug-1622800-11 branch 8 times, most recently from ff1a075 to d3b1caf Compare April 30, 2020 13:49

JohanLorenzo added 3 commits April 30, 2020 17:39

Bug 1622800 - part 11: Enable screenshot tests for all locales

a984bed

Bug 1622800 - part 12: Use dedicated bitrise workers

330eacc

Bug 1622800 - part 13: Create a hook to run the production screenshot…

d7de0fa

… jobs

JohanLorenzo force-pushed the bug-1622800-11 branch from d3b1caf to d7de0fa Compare April 30, 2020 15:40

JohanLorenzo changed the title ~~Bug 1622800 - part 11: WIP~~ Bug 1622800 - part 11 to 13: Enable all locales and create hook Apr 30, 2020

Bug 1622800 - part 14: Do not fail job on missing bitrise logs

533b564

JohanLorenzo force-pushed the bug-1622800-11 branch from 58dd85b to 533b564 Compare April 30, 2020 18:17

JohanLorenzo changed the title ~~Bug 1622800 - part 11 to 13: Enable all locales and create hook~~ Bug 1622800 - part 11 to 14: Enable all locales and create hook Apr 30, 2020

JohanLorenzo marked this pull request as ready for review April 30, 2020 18:17

JohanLorenzo commented Apr 30, 2020

View reviewed changes

JohanLorenzo requested review from dnarcese and isabelrios April 30, 2020 18:35

isabelrios approved these changes May 4, 2020

View reviewed changes

dnarcese approved these changes May 4, 2020

View reviewed changes

JohanLorenzo merged commit 2a5ca92 into mozilla-mobile:master May 4, 2020

JohanLorenzo deleted the bug-1622800-11 branch May 4, 2020 15:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bug 1622800 - part 11 to 14: Enable all locales and create hook #6459

Bug 1622800 - part 11 to 14: Enable all locales and create hook #6459

JohanLorenzo commented Apr 24, 2020

JohanLorenzo Apr 30, 2020

JohanLorenzo May 5, 2020

JohanLorenzo Apr 30, 2020

JohanLorenzo Apr 30, 2020

JohanLorenzo Apr 30, 2020

JohanLorenzo Apr 30, 2020

JohanLorenzo Apr 30, 2020

isabelrios May 4, 2020

JohanLorenzo Apr 30, 2020

JohanLorenzo Apr 30, 2020

JohanLorenzo Apr 30, 2020

JohanLorenzo Apr 30, 2020 •

edited

Loading

JohanLorenzo commented Apr 30, 2020 •

edited

Loading

isabelrios left a comment

isabelrios May 4, 2020

isabelrios May 4, 2020

		from ..screenshots_locales import get_screenshots_locales


		def loader(kind, path, config, params, loaded_tasks):

Bug 1622800 - part 11 to 14: Enable all locales and create hook #6459

Bug 1622800 - part 11 to 14: Enable all locales and create hook #6459

Conversation

JohanLorenzo commented Apr 24, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JohanLorenzo Apr 30, 2020 • edited Loading

Choose a reason for hiding this comment

JohanLorenzo commented Apr 30, 2020 • edited Loading

isabelrios left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JohanLorenzo Apr 30, 2020 •

edited

Loading

JohanLorenzo commented Apr 30, 2020 •

edited

Loading