feat: add experimental atlas to pull_translations #1093

OmarIthawi · 2023-03-30T16:23:30Z

Description

This is an experimental off-by-defualt feature for moving the translation files ouside the repos.

Run OPENEDX_ATLAS_PULL=true make translations to use atlas to pull translations instead of transifex.

Changes

Centralize i18n messages import in src/i18n/index.js
Add new pull_translations implementation via atlas
Use the new i18n-imports.js script from feat: new intl-imports script for atlas src/i18n/index.js generation frontend-platform#463

Reviewers

Get initial feedback (done by Brian)
Get the dependencies bumped to bring up intl-imports.js script:
Upgrade [email protected] in exams library (blocker): feat: upgraded to node v18, added .nvmrc and updated workflows frontend-lib-special-exams#89
Quick QA:
Screenshot
Fix tests

Sample expected outcome

The file tree after running the new pull_translations will look like the following:

All messages will be pulled fresh, without depending on any messages from the MFE or its depedencies.

frontend-app-learning $ tree src/i18n/
src/i18n/
├── index.js
└── messages
    ├── frontend-app-learning
    │   ├── ar.json
    │   ├── de.json
    │   ├── es_419.json
    │   ├── fr_CA.json
    │   ├── index.js
    │   ├── ru.json
    │   ├── uk.json
    │   └── zh_CN.json
    ├── frontend-component-footer
    │   ├── ar.json
    │   ├── de_DE.json
    │   ├── de.json
    │   ├── es_419.json
    │   ├── fr_CA.json
    │   ├── fr.json
    │   ├── hi.json
    │   ├── it_IT.json
    │   ├── it.json
    │   ├── pt.json
    │   ├── pt_PT.json
    │   ├── ru.json
    │   ├── uk.json
    │   └── zh_CN.json
    └── frontend-component-header

Sample package `index.js` file

For each directory/package an index.js file will be generated if and only if there are messages in the files:

// This file is generated by the Open edX generate-mfe-i18n-imports.
// Refer to docs yada yada ...
//

import messages0ar from './ar.json';
import messages1de from './de.json';
import messages2es_419 from './es_419.json';
import messages3fr_CA from './fr_CA.json';
import messages4ru from './ru.json';
import messages5uk from './uk.json';
import messages6zh_CN from './zh_CN.json';
const messages = {
  'ar': messages0ar,
  'zh-cn': messages6zh_CN,
  'uk': messages5uk,
  'ru': messages4ru,
  'fr-ca': messages3fr_CA,
  'es-419': messages2es_419,
  'de': messages1de,
};
export default messages;

Sample MFE `src/i18n/index.js` file

A new index.js is going to be generated containing only downloaded languages.

// This file is generated by the Open edX frontend-platform's i18n-imports commands.
//

import messages1frontendcomponentheader from './messages/frontend-component-header';
// footer is omitted due to not having translations
import messages2frontendapplearning from './messages/frontend-app-learning';

export default [
  messages1frontendcomponentheader,
  messages2frontendapplearning,
];

References

This pull request is part of the FC-0012 project which is sparked by the Translation Infrastructure update OEP-58.

Check the links above for full information about the overall project.

Internalization is being rearchitected in Open edX Python, XBlock, Micro-frontend, and other projects. There are a number of immediately visible changes:

Remove source and language translations from the repositories, hence no .json, .po or .mo files will be committed into the repos.
Add standardized make extract_translations in all repositories
Push user-facing messages strings into openedx/openedx-translations.
Integrate root repositories with openedx/openedx-atlas to pull translations on build/deploy time

Breaking Changes

One of the primary goals of the project is to avoid breaking changes. If you noticed any suspicious code, please raise your concern. But before that, please know the strategy we're following to avoid breaking changes

openedx-webhooks · 2023-03-30T16:23:37Z

Thanks for the pull request, @OmarIthawi! Please note that it may take us up to several weeks or months to complete a review and merge your PR.

Feel free to add as much of the following information to the ticket as you can:

supporting documentation
Open edX discussion forum threads
timeline information ("this must be merged by XX date", and why that is)
partner information ("this is a course on edx.org")
any other information that can help Product understand the context for the PR

All technical communication about the code itself will be done via the GitHub pull request interface. As a reminder, our process documentation is here.

Please let us know once your PR is ready for our review and all tests are green.

codecov · 2023-03-30T16:44:58Z

Codecov Report

Patch and project coverage have no change.

Comparison is base (e0acb50) 87.03% compared to head (d147125) 87.03%.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #1093   +/-   ##
=======================================
  Coverage   87.03%   87.03%           
=======================================
  Files         253      253           
  Lines        4381     4381           
  Branches     1108     1108           
=======================================
  Hits         3813     3813           
  Misses        549      549           
  Partials       19       19

Impacted Files	Coverage Δ
src/index.jsx	`0.00% <ø> (ø)`

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

OmarIthawi · 2023-03-31T16:19:33Z

Hi, @edx/engage-squad I'd love a quick sanity check before I invest more time in shaping up this work, remove temp. references and add documentation.

cc: @mphilbrick211, it turns out I can't tag the right team.

brian-smith-tcril

This is looking really great! Definitely a good pattern when combined with openedx/frontend-platform#463!

Just one little note as a reminder so we don't forget and merge this while pointing to the Zeit Labs fork, but other than that this is looking good to go!

Makefile

mphilbrick211 · 2023-04-05T00:09:45Z

Hi, @edx/engage-squad I'd love a quick sanity check before I invest more time in shaping up this work, remove temp. references and add documentation.

cc: @mphilbrick211, it turns out I can't tag the right team.

@OmarIthawi sorry for the late reply. @openedx/engage-squad - could someone please take a look at this? Thanks!

itsjeyd · 2023-04-13T06:12:37Z

Hey @OmarIthawi, just checking in to see if @brian-smith-tcril's feedback addressed your need for a quick sanity check? Or are you still looking for additional input from the engage-squad before continuing?

OmarIthawi · 2023-04-13T12:03:29Z

Hey @OmarIthawi, just checking in to see if @brian-smith-tcril's feedback addressed your need for a quick sanity check? Or are you still looking for additional input from the engage-squad before continuing?

@itsjeyd I've got the initial review. Thanks for checking.

I'm now blocked by other PRs which brings the required intl-imports.js command:

brian-smith-tcril · 2023-04-13T21:01:42Z

Hey @OmarIthawi, just checking in to see if @brian-smith-tcril's feedback addressed your need for a quick sanity check? Or are you still looking for additional input from the engage-squad before continuing?

@itsjeyd I've got the initial review. Thanks for checking.

I'm now blocked by other PRs which brings the required intl-imports.js command:

chore(deps): update frontend-platform version frontend-component-footer#286

chore(deps): update frontend-platform version frontend-component-header#329

feat: new intl-imports script for atlas src/i18n/index.js generation frontend-platform#463

All the linked PRs are merged now!

OmarIthawi · 2023-04-14T11:41:27Z

Thanks @brian-smith-tcril. We still need to tag new releases on both the header and footer on GitHub so they're published on npm.

brian-smith-tcril · 2023-04-20T13:31:16Z

We still need to tag new releases on both the header and footer on GitHub so they're published on npm.

Releases are out for header and footer now!
https://github.com/openedx/frontend-component-header/releases/tag/v3.7.3
https://github.com/openedx/frontend-component-footer/releases/tag/v11.7.3

OmarIthawi · 2023-04-20T13:35:54Z

We still need to tag new releases on both the header and footer on GitHub so they're published on npm.

Releases are out for header and footer now! https://github.com/openedx/frontend-component-header/releases/tag/v3.7.3 https://github.com/openedx/frontend-component-footer/releases/tag/v11.7.3

Thanks! It turns out we have another blocker. I've updated the PR description: https://github.com/openedx/frontend-lib-special-exams is stuck on [email protected] and needs the following PR to be merged:

feat: upgraded to node v18, added .nvmrc and updated workflows frontend-lib-special-exams#89

OmarIthawi · 2023-05-01T05:36:53Z

@mphilbrick211 this is now ready for review.

mphilbrick211 · 2023-05-01T18:26:53Z

@mphilbrick211 this is now ready for review.

@itsjeyd FYI!

brian-smith-tcril

LGTM! Once conflicts are resolved let's

itsjeyd · 2023-05-03T06:57:06Z

Thanks for the ping @mphilbrick211 :)

This is an experimental off-by-defualt feature for moving the translation files ouside the repos. Run `OPENEDX_ATLAS_PULL=true make translations` to use atlas to pull translations instead of transifex.

OmarIthawi · 2023-05-03T14:50:55Z

Conflicts are fixed again. Thanks all for the reviews and help!!

brian-smith-tcril · 2023-05-04T15:56:26Z

@itsjeyd conflicts are resolved on this one!

itsjeyd · 2023-05-09T08:41:51Z

@OmarIthawi @brian-smith-tcril Thanks for the update! Marking this as ready for merge.

@brian-smith-tcril If you have permission to merge (?) please go ahead, otherwise we can ping the Aurora team; let me know.

openedx-webhooks · 2023-05-09T14:03:48Z

@OmarIthawi 🎉 Your pull request was merged! Please take a moment to answer a two question survey so we can improve your experience in the future.

This is an experimental off-by-defualt feature for moving the translation files ouside the repos. Run `OPENEDX_ATLAS_PULL=true make translations` to use atlas to pull translations instead of transifex. Refs: FC-12 OEP-58

openedx-webhooks added the open-source-contribution PR author is not from Axim or 2U label Mar 30, 2023

OmarIthawi force-pushed the generate-i18n-modules branch from 997bb03 to 66e1084 Compare March 30, 2023 16:28

This was referenced Mar 30, 2023

temp: manually pull_translations with atlas zeit-labs/frontend-app-learning#2

Closed

feat: new intl-imports script for atlas src/i18n/index.js generation openedx/frontend-platform#463

Merged

brian-smith-tcril reviewed Apr 3, 2023

View reviewed changes

Makefile Outdated Show resolved Hide resolved

OmarIthawi force-pushed the generate-i18n-modules branch 2 times, most recently from 5c00ba7 to f5b8924 Compare April 4, 2023 22:33

OmarIthawi force-pushed the generate-i18n-modules branch 4 times, most recently from 589f071 to 468d396 Compare April 11, 2023 15:25

This was referenced Apr 12, 2023

chore(deps): update frontend-platform version openedx/frontend-component-footer#286

Merged

chore(deps): update frontend-platform version openedx/frontend-component-header#329

Merged

OmarIthawi force-pushed the generate-i18n-modules branch from 468d396 to d27b9ce Compare April 12, 2023 14:50

itsjeyd added the waiting on author PR author needs to resolve review requests, answer questions, fix tests, etc. label Apr 13, 2023

OmarIthawi marked this pull request as ready for review April 13, 2023 12:06

OmarIthawi force-pushed the generate-i18n-modules branch 3 times, most recently from 13fbdfd to e252883 Compare April 18, 2023 16:15

OmarIthawi requested a review from brian-smith-tcril April 18, 2023 16:20

OmarIthawi force-pushed the generate-i18n-modules branch 2 times, most recently from 9d0401c to 620d847 Compare April 19, 2023 16:21

OmarIthawi marked this pull request as draft April 20, 2023 13:28

brian-smith-tcril added the blocked by other work PR cannot be finished until other work is complete label Apr 20, 2023

OmarIthawi force-pushed the generate-i18n-modules branch 3 times, most recently from 1403d97 to c0b8ca6 Compare April 27, 2023 08:42

OmarIthawi marked this pull request as ready for review April 27, 2023 08:59

mphilbrick211 removed blocked by other work PR cannot be finished until other work is complete waiting on author PR author needs to resolve review requests, answer questions, fix tests, etc. labels May 1, 2023

brian-smith-tcril approved these changes May 2, 2023

View reviewed changes

itsjeyd added the waiting on author PR author needs to resolve review requests, answer questions, fix tests, etc. label May 3, 2023

feat: add experimental atlas to pull_translations

d147125

This is an experimental off-by-defualt feature for moving the translation files ouside the repos. Run `OPENEDX_ATLAS_PULL=true make translations` to use atlas to pull translations instead of transifex.

OmarIthawi force-pushed the generate-i18n-modules branch from c0b8ca6 to d147125 Compare May 3, 2023 14:50

brian-smith-tcril removed the waiting on author PR author needs to resolve review requests, answer questions, fix tests, etc. label May 3, 2023

brian-smith-tcril merged commit d2de063 into openedx:master May 9, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add experimental atlas to pull_translations #1093

feat: add experimental atlas to pull_translations #1093

OmarIthawi commented Mar 30, 2023 •

edited

Loading

openedx-webhooks commented Mar 30, 2023 •

edited

Loading

codecov bot commented Mar 30, 2023 •

edited

Loading

OmarIthawi commented Mar 31, 2023

brian-smith-tcril left a comment

mphilbrick211 commented Apr 5, 2023

itsjeyd commented Apr 13, 2023

OmarIthawi commented Apr 13, 2023 •

edited

Loading

brian-smith-tcril commented Apr 13, 2023

OmarIthawi commented Apr 14, 2023

brian-smith-tcril commented Apr 20, 2023

OmarIthawi commented Apr 20, 2023 •

edited

Loading

OmarIthawi commented May 1, 2023

mphilbrick211 commented May 1, 2023

brian-smith-tcril left a comment

itsjeyd commented May 3, 2023

OmarIthawi commented May 3, 2023

brian-smith-tcril commented May 4, 2023

itsjeyd commented May 9, 2023

openedx-webhooks commented May 9, 2023

feat: add experimental atlas to pull_translations #1093

feat: add experimental atlas to pull_translations #1093

Conversation

OmarIthawi commented Mar 30, 2023 • edited Loading

Description

Changes

Reviewers

Sample expected outcome

Sample package index.js file

Sample MFE src/i18n/index.js file

References

Breaking Changes

openedx-webhooks commented Mar 30, 2023 • edited Loading

codecov bot commented Mar 30, 2023 • edited Loading

Codecov Report

OmarIthawi commented Mar 31, 2023

brian-smith-tcril left a comment

Choose a reason for hiding this comment

mphilbrick211 commented Apr 5, 2023

itsjeyd commented Apr 13, 2023

OmarIthawi commented Apr 13, 2023 • edited Loading

brian-smith-tcril commented Apr 13, 2023

OmarIthawi commented Apr 14, 2023

brian-smith-tcril commented Apr 20, 2023

OmarIthawi commented Apr 20, 2023 • edited Loading

OmarIthawi commented May 1, 2023

mphilbrick211 commented May 1, 2023

brian-smith-tcril left a comment

Choose a reason for hiding this comment

itsjeyd commented May 3, 2023

OmarIthawi commented May 3, 2023

brian-smith-tcril commented May 4, 2023

itsjeyd commented May 9, 2023

openedx-webhooks commented May 9, 2023

OmarIthawi commented Mar 30, 2023 •

edited

Loading

Sample package `index.js` file

Sample MFE `src/i18n/index.js` file

openedx-webhooks commented Mar 30, 2023 •

edited

Loading

codecov bot commented Mar 30, 2023 •

edited

Loading

OmarIthawi commented Apr 13, 2023 •

edited

Loading

OmarIthawi commented Apr 20, 2023 •

edited

Loading