Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

i18n_tool.py: fix Unicode handling; change commit format #4482

Merged
merged 2 commits into from
Jun 4, 2019

Conversation

rmol
Copy link
Contributor

@rmol rmol commented May 29, 2019

Status

Ready for review

Description of Changes

Use of six.u was corrupting translator names. According to
https://pythonhosted.org/six/index.html#six.u -- "On Python 2, u()
doesn't know what the encoding of the literal is. Each byte is
converted directly to the unicode codepoint of the same value. Because
of this, it's only safe to use u() with strings of ASCII data."

Of course some translators' names were not just ASCII, so I've
replaced usage of six.u with u"", as it should be safe on both Python
2 and Python 3 > 3.3, per PEP-0414.

I also changed the commit format to put each contributor's name on its
own line, and tried to clarify the source of the translations. It was
confusing to me, at least, that translations were being "copied from"
a revision that involved a different language.

Finally, we were gathering the list of contributors from commits other
than just the Weblate contributions, so whoever last ran the tool was
getting credited for translations since. I've changed the tool to only
look for contributors in the automated Weblate commits.

Testing

  1. Check out this branch.
  2. Create a testing branch from it with git checkout -b i18npy2 i18n_tool.
  3. Run i18n_tool.py under Python 2:
    PYTHON_VERSION=2 securedrop/bin/dev-shell python2 ./i18n_tool.py --verbose update-from-weblate
  4. Check the commits added to your i18npy2 branch. There should be a commit for each supported locale, added in alphabetic order, with only a few contributors listed for each. The names of contributors to Swedish, Norwegian, and Icelandic translations should appear correctly, with no line noise.
  5. Create a testing branch from it with git checkout -b i18npy3 i18n_tool.
  6. Run i18n_tool.py under Python 3:
    PYTHON_VERSION=3 securedrop/bin/dev-shell python3 ./i18n_tool.py --verbose update-from-weblate
  7. Check the commits added to your i18npy3 branch.

Deployment

These changes should only affect development.

Checklist

If you made non-trivial code changes:

  • I have written a test plan and validated it for this PR

rmol added 2 commits May 30, 2019 13:40
Use of six.u was corrupting translator names. According to
https://pythonhosted.org/six/index.html#six.u -- "On Python 2, u()
doesn't know what the encoding of the literal is. Each byte is
converted directly to the unicode codepoint of the same value. Because
of this, it's only safe to use u() with strings of ASCII data."

Of course some translators' names were not just ASCII, so I've
replaced usage of six.u with u"", as it should be safe on both Python
2 and Python 3 > 3.3, per PEP-0414.

I also changed the commit format to put each contributor's name on its
own line, and tried to clarify the source of the translations. It was
confusing to me, at least, that translations were being "copied from"
a revision that involved a different language.

Finally, we were gathering the list of contributors from commits other
than just the Weblate contributions, so whoever last ran the tool was
getting credited for translations since. I've changed the tool to only
look for contributors in the automated Weblate commits.
@kushaldas kushaldas merged commit 2be9c98 into freedomofpress:develop Jun 4, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants