i18n_tool.py: fix Unicode handling; change commit format #4482
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Status
Ready for review
Description of Changes
Use of
six.u
was corrupting translator names. According tohttps://pythonhosted.org/six/index.html#six.u -- "On Python 2, u()
doesn't know what the encoding of the literal is. Each byte is
converted directly to the unicode codepoint of the same value. Because
of this, it's only safe to use u() with strings of ASCII data."
Of course some translators' names were not just ASCII, so I've
replaced usage of
six.u
withu""
, as it should be safe on both Python2 and Python 3 > 3.3, per PEP-0414.
I also changed the commit format to put each contributor's name on its
own line, and tried to clarify the source of the translations. It was
confusing to me, at least, that translations were being "copied from"
a revision that involved a different language.
Finally, we were gathering the list of contributors from commits other
than just the Weblate contributions, so whoever last ran the tool was
getting credited for translations since. I've changed the tool to only
look for contributors in the automated Weblate commits.
Testing
git checkout -b i18npy2 i18n_tool
.i18n_tool.py
under Python 2:PYTHON_VERSION=2 securedrop/bin/dev-shell python2 ./i18n_tool.py --verbose update-from-weblate
i18npy2
branch. There should be a commit for each supported locale, added in alphabetic order, with only a few contributors listed for each. The names of contributors to Swedish, Norwegian, and Icelandic translations should appear correctly, with no line noise.git checkout -b i18npy3 i18n_tool
.i18n_tool.py
under Python 3:PYTHON_VERSION=3 securedrop/bin/dev-shell python3 ./i18n_tool.py --verbose update-from-weblate
i18npy3
branch.Deployment
These changes should only affect development.
Checklist
If you made non-trivial code changes: