streamline functions currently provided by i18n_tool.py

[cfm]

• Adapt securedrop-client's make extract-strings target
  • Can this be generalized in securedrop-tooling?
  • Does this let us combine the securedrop/{securedrop,desktop} Weblate components? (This might violate the "No side effects" constraint.)
• Enforce in CI (make check-strings)
  • Can this be generalized in securedrop-tooling?
• Per https://github.com/freedomofpress/securedrop-engineering/pull/30#issuecomment-1633171470:
  • Reimplement "Compile .po catalogs → .mo machine objects" as make compile-translations
• make update-docs → https://github.com/freedomofpress/securedrop-docs/blob/main/docs/includes/l10n.txt
  • Counterpoint: streamline functions currently provided by i18n_tool.py #6917 (comment)
  • replace l10n.txt include with link to i18n.rst securedrop-docs#503
• replicate i18n_tool.py list-translators in continuous-localization workflow securedrop-client#1280
  • Counterpoint: streamline functions currently provided by i18n_tool.py #6917 (comment)

[cfm]

The {check,extract}-strings targets are available in 6917-c10s-l10n as of 63be596, which is failing CI in lint as intended. Remaining i18n_tool.py invocations to replace next week:

(.venv) user@sd-dev:~/securedrop$ grep -R "i18n_tool.py" * | grep -Ev "^changelog.md:"
Makefile:	@echo "SUPPORTED_LOCALES = $$(if test -f /opt/venvs/securedrop-app-code/bin/python3; then ./securedrop/i18n_tool.py list-locales --python; else DOCKER_BUILD_VERBOSE=false $(DEVSHELL) ./i18n_tool.py list-locales --python; fi)" | sed 's/\r//' >> securedrop/config.py
Makefile:	@$(DEVSHELL) $(SDROOT)/securedrop/i18n_tool.py list-translators
Makefile:	@$(DEVSHELL) $(SDROOT)/securedrop/i18n_tool.py list-translators --all
securedrop/debian/translations.sh:/tmp/securedrop-app-code-i18n-ve/bin/python3 ./i18n_tool.py --verbose translate-messages --compile
securedrop/bin/translation-test:./i18n_tool.py translate-messages --compile
securedrop/bin/translation-test:SUPPORTED_LOCALES=$(./i18n_tool.py list-locales)
securedrop/bin/generate-docs-screenshots:./i18n_tool.py translate-messages --compile
securedrop/bin/dev-deps:    ./i18n_tool.py translate-messages --compile

[cfm]

#6917 (comment):

• make update-docs → https://github.com/freedomofpress/securedrop-docs/blob/main/docs/includes/l10n.txt

This has always had a bit of smell to me. I think it's the only place outside of packaging where we let a Make target in one repository read or write a file in another. Even though there's a --docs-repo-dir, I've only ever been able to use (e.g.) /home/user/securedrop/securedrop-docs to get Docker to play nicely with permissions and Git roots.

Since our goal here is to automate as much of this process as possible, rather than reimplement this particular step literally, I propose to do one of the following instead, from least to most effort:

1. Replace the include directive in install.rst with a link to i18n.json.
2. Add a Make target that writes i18n.json to either i18n.md or i18n.rst in this repository. Then replace the include directive in install.rst with a link to that file, which GitHub will render. Enforce this in CI, as make check-strings enforces make extract-strings.
3. Add a Make target in securedrop-docs that fetches i18n.json and writes it to the l10n.txt expected by the include directive in install.rst. Use this in CI.

[legoktm]

You have some stuff checked off in the above list, am I correct in assuming that's just a personal list you're using, i.e. those things haven't been fixed in develop yet nor a PR?

[cfm]

Yes, exactly, @legoktm: this checklist tracks the work in progress in i18n-6917-c10s-l10n.

[cfm]

#6917 (comment):

• replicate i18n_tool.py list-translators in continuous-localization workflow securedrop-client#1280

Hot take: we shouldn't do this either. I've proposed this in freedomofpress/securedrop-client#1280 (comment) and presumptively documented it in freedomofpress/securedrop-dev-docs@f91a4d3.

[cfm]

Barring any objections to the reductions of scope I've proposed in #6917 (comment) and #6917 (comment), here's what's left here:

1. securedrop/.circleci/config.yml

   Line 225 in 96ef2ef

   export LOCALES="$(/usr/bin/python3 securedrop/i18n_tool.py list-locales --lines | circleci tests split | tr '\n' ' ')"

   invokes i18n_tool.py and needs to use make supported-locales instead, which means I need to factor out

   securedrop/securedrop/bin/translation-test

   Line 20 in 38655de

   SUPPORTED_LOCALES=$(make --directory .. --quiet supported-locales | jq --raw-output 'join (" ")')

   to take a separator.
2. There are also some tests (outside of the now-gone test_i18n_tool.py) that rely on i18n_tool.py and will need to use either pybabel or precompiled test fixtures instead.
3. As we relearned in Add a GNOME Shell Extension #6712, the securedrop/desktop component is indeed unavoidable, because the Tails environment doesn't have gettext, pybabel, &c. available. Based on my findings just now in verify Weblate-compiled gettext machine objects in CI securedrop-client#1507 (comment), we can have Weblate generate .mo files for this component and verify them in CI, without needing to keep tooling around to update them manually.
4. That will get us to the point of having a draft pull request, which we'll need to test (prior to formal review) in securedrop-i18n-sandbox against weblate-sandbox per adopt continuous localization #6232.