Hotfix/2729 fetch translated exercises part 2

[cpauya]

Hi @aronasorman - this fixes #2729 with these notes from @jesumer

Central Server Requirements

1. Include the .po files from Khan Academy's Crowdin account during "update_language_packs".
2. This is required to get the latest translations but the above should do for our test
   • Issue fix by adding --rebuild make option into update_language_packs command
3. make sure it's the same version as distributed

Distributed Server Requirements

1. make sure it's the same version as central

To test

At Central server:

1. Run python manage.py update_language_packs -l de --no-srts --no-dubbed --no-kalite-trans --rebuild command.
2. Check if this will get the proper updated mo.files from CrowdIn.(we can check this out to centralserver/_crowdin_cache zip file).

At Distributed:

1. If it is successful then go to the distributed browser.(NOTE: Make sure you've done the "setup" already).
2. Run python manage.py languagepackdownload -l de --commandline.
3. Go to the language page and set the language code to de.
4. Go to the exercises and try to test on some of the exercises.
5. Check the translated words.

To test thru the Python shell
This is taken from fle-internal/ka-lite-central#226 (comment)

from django.utils.translation import activate, ugettext as _

activate("de")
s = "What is <code>X</code>?"
_(s)

Which should display this:

>>> from django.utils.translation import activate, ugettext as _
>>>
>>> activate("de")
>>> s = "What is <code>X</code>?"
>>> _(s)
'Was ist <code>X</code>?'
>>>

References:

1. Official Khan Academy German translations on CrowdIn
2. learn.math.algebra-basics.exercises.pot - 43% -- Search for What is <code>X</code>? text as per example above.

[aronasorman]

I took quite some time trying to find a perseus exercise that's at least partially translated in KA.

Exercise: Linear Algebra -> Unit Vectors

Notice the instructions to the right (Fulle die Lucken)

Which is present in the assessment item data:

So I can confirm that this works. Although @cpauya maybe you have a better example for everyone to see.

[aronasorman]

Addressed some of mike's issues. Just waiting for tests to pass, and then I'm merging this.

[cpauya]

Hi @aronasorman - thanks for the fixes.

Will try to find an exercise and post here with screenshots.

We will need these issues merged to central though to get the updated translations:

1. Issue fix by adding --rebuild make option into update_language_packs command
2. Include the .po files from Khan Academy's Crowdin account during "update_language_packs".

[cpauya]

@aronasorman Sadly I can't try the exercise you mentioned above because it's greyed-out on my sidebar topics and clicking the entry loads a "Content was not found..." message.

I'm waiting for bin/kalite manage unpack_assessment_zip http://eslgenie.com/media/assessment_item_resources.zip -f to finish to see if an updated assessmentitems.json will fix it.

[cpauya]

@aronasorman - I was able to reproduce your string above (see screenshot) so executing bin/kalite manage unpack_assessment_zip http://eslgenie.com/media/assessment_item_resources.zip -f helped! Thanks for the tip.

Found the discrepancy between the asssessmentitems.json strings and the translation strings from CrowdIn at django.mo. I used this exercise url: http://127.0.0.1:8008/learn/khan/math/early-math/cc-early-math-counting-topic/cc-early-math-counting/count-to-100/

Sample script

from django.utils.translation import activate, ugettext as _

activate("de")

# 1. Sample string to check that we are indeed translating to German.
s = "What is <code>X</code>?"
_(s)

# 2. String from `assessmentitems.json` API call to http://127.0.0.1:8008/api/assessment_item/x83e816580f82fd5f/
s = "Let's count by tens.\n\n![](/content/khan/0dbaca258ebbe4fc6c6c4fb901b9cbc159d09c2b.png)\n\n**What number did we miss?**\n\n \n\n[[\u2603 radio 1]]"
_(s)

# 3. This is the string from the CrowdIn file learn.math.early-math.exercises-de.po:31062
s = "Let's count by tens.\n\n![](https://ka-perseus-graphie.s3.amazonaws.com/0dbaca258ebbe4fc6c6c4fb901b9cbc159d09c2b.png)\n\n**What number did we miss?**\n\n \n\n[[☃ radio 1]]"
_(s)

And here's the result

>>> from django.utils.translation import activate, ugettext as _
>>>
>>> activate("de")
>>>
>>> # 1. Sample string to check that we are indeed translating to German.
>>> s = "What is <code>X</code>?"
>>> _(s)
'Was ist <code>X</code>?'
>>>
>>> # 2. String from `assessmentitems.json` API call to http://127.0.0.1:8008/api/assessment_item/x83e816580f82fd5f/
>>> s = "Let's count by tens.\n\n![](/content/khan/0dbaca258ebbe4fc6c6c4fb901b9cbc159d09c2b.png)\n\n**What number did we miss?**\n\n \n\n[[\u2603 radio 1]]"
>>> _(s)
u"Let's count by tens.\n\n![](/content/khan/0dbaca258ebbe4fc6c6c4fb901b9cbc159d09c2b.png)\n\n**What number did we miss?**\n\n \n\n[[\\u2603 radio 1]]"
>>>
>>> # 3. This is the string from the CrowdIn file learn.math.early-math.exercises-de.po:31062
>>> s = "Let's count by tens.\n\n![](https://ka-perseus-graphie.s3.amazonaws.com/0dbaca258ebbe4fc6c6c4fb901b9cbc159d09c2b.png)\n\n**What number did we miss?**\n\n \n\n[[☃ radio 1]]"
>>> _(s)
'Lass uns in Zehnerschritten z\xc3\xa4hlen. \n\n ![](https://ka-perseus-graphie.s3.amazonaws.com/0dbaca258ebbe4fc6c6c4fb901b9cbc159d09c2b.png) \n\n **Welche Zahl fehlt?** \n\n\n\n [[\xe2\x98\x83 radio 1]]'

Question

1. Based on comment 2 above - The string from asssessmentitems.json differs from the string that's on the django.mo translation file:

   1.1.The /content/khan/0dbaca258ebbe4fc6c6c4fb901b9cbc159d09c2b.png versus the https://ka-perseus-graphie.s3.amazonaws.com/0dbaca258ebbe4fc6c6c4fb901b9cbc159d09c2b.png and;
   1.2. The \u2603 compared to the ☃ character.

Do we have the script or codes that generated the asssessmentitems.json from the .zip file mentioned above so we can change it and complete this PR?

[aronasorman]

Ah. So essentially we can't get the exact string from the mo file, because it differs slightly from what's in assessmentitems.json due to the url.

So the best solution in here is to do the same amazon.com to /content url conversion we do with the english assessment strings in the mo file, before we insert the mo file into the language pack.

That seems to be a central server issue though. Not sure if we have to change anything here in this PR @cpauya.

[cpauya]

Yes @aronasorman - let's leave this open until fle-internal/ka-lite-central#238 is done and merged then.

[jesumer]

Hi @aronasorman,

As what @cpauya hip chat instruction:

1. run update_language_packs -l de --no-srts --no-dubbed on central.
2. check an *exercise-de.pot contains the latest approved strings.
3. another option is, do an export of an *exercise-de.pot from CrowdIn account of KA.
4. compare that with the downloaded *exercise-de.pot.

I have tried to reproduce the instructions above:

but when I was at step 3 and trying to export I got this:

I guess my issue #3087 is not necessary since we can now get the latest pot files from crowdin.

[aronasorman]

Let's merge this, so I can test it with the dummy language packs.