From 34f9b16a430551ed7c769ed112c4038dd40500fd Mon Sep 17 00:00:00 2001 From: Ben Henning Date: Wed, 7 Sep 2022 23:59:51 -0700 Subject: [PATCH] Fix part of #4562: Add string consistency tool & CI check (#4563) ## Explanation Fixes part of #4562. This PR introduces two new scripts: - One for computing how many strings are not yet translated for Arabic, Brazilian Portuguese, and Swahili (as compared to the base English strings) - One for verifying newline consistency between the English & non-English strings (based on number of lines) The latter was also added as one of the jobs to be run during the static checks CI workflow, and is specifically useful to ensure newlines weren't inadvertently added by translators (see the updated translation strings for an idea on how often this has happened). Note that, as part of fixing these strings, the source string for lets_get_started was updated to no longer include a newline. This seems reasonable given that we should never use newlines for spacing/styling unless it's for logical splits (like paragraphs). However, I can't verify whether actual style changes are needed since this string is used as part of the unreleased app walkthrough feature. I expect that we'll re-audit the UI of that feature when we revisit it. This PR also includes a major performance fix for RepositoryFile that will benefit all checks that use it. Kotlin's built-in file tree walker is built to implicitly follow symlinks on the filesystem. For Bazel builds, this expands the codebase to >1M files (since directory filtering happens *after* the files are known). Since scripts don't actually need to follow symlinks, the utility was updated to specifically not follow them (using Java NIO's file tree walk routine) which led to an observed ~10x performance improvement. Further optimizations could be done by building our own tree-walker and filtering directories during search (which will probably be necessary if we ever _do_ need to follow symlinks, but it probably won't be needed for a long time since the current performance is quite solid). The former script is useful when manually auditing the translation progress using the codebase as the source of truth rather than Translatewiki. For now, it's been helpful in beta MR1 planning but longer-term it may also be useful as a badge on the repo's README. Finally, the Kotlin stdlib dependency was updated to point to jdk8 instead of jdk7 (since a specific feature was needed from the former, and the codebase can rely on higher Java SDK versions than 7 given that the minimum Java version is 1.8). ## Essential Checklist - [x] The PR title and explanation each start with "Fix #bugnum: " (If this PR fixes part of an issue, prefix the title with "Fix part of #bugnum: ...".) - [x] Any changes to [scripts/assets](https://github.com/oppia/oppia-android/tree/develop/scripts/assets) files have their rationale included in the PR explanation. - [x] The PR follows the [style guide](https://github.com/oppia/oppia-android/wiki/Coding-style-guide). - [x] The PR does not contain any unnecessary code changes from Android Studio ([reference](https://github.com/oppia/oppia-android/wiki/Guidance-on-submitting-a-PR#undo-unnecessary-changes)). - [x] The PR is made from a branch that's **not** called "develop" and is up-to-date with "develop". - [x] The PR is **assigned** to the appropriate reviewers ([reference](https://github.com/oppia/oppia-android/wiki/Guidance-on-submitting-a-PR#clarification-regarding-assignees-and-reviewers-section)). ## For UI-specific PRs only N/A -- this is a developer infrastructure-only PR, except for all of the string newline fixes (which just result in a more correct UI for non-English), and the changed English string (which is inaccessible to demonstrate). Screenshots could be uploaded, but the newlines need to be removed regardless so it doesn't seem specifically useful here (plus those languages may not be obvious to verify for reviewers). Commits: * Create dedicated alpha application component. This simplifies application component management significantly and allows individual build flavors to have their own unique module lists. * Add beta & GA update notices. This also introduces dedicated beta & GA build flavors which is a necessary prerequisite. It also introduces an extra beta, alpha, and dev mode labels for the splash screen (the latter 2 were extra) with 2 second minimum wait timers for beta and alpha to ensure they are seen. A 5-second safety timer was added to ensure the splash screen can always be passed even if something goes wrong at the domain level (since there are now quite a few moving pieces to determine the user's current onboarding state). * Add build tests for the new beta & GA flavors. * Fix broken test per earlier changes. * Fix general broken tests & builds. Tests broken due to changes to the app startup experience haven't yet been fixed. * Lint fixes. * First part of adding tests for GA notices. There's a bunch left to do here, this is mainly needed so that I can transfer changes to a different machine. * Update TransformAndroidManifestTest.kt Correct typos. * Fix tests & static checks. This also removes temporary debug code and TODOs, and finishes the tests for SplashActivity. * Post-merge fixes. * Test fixes. * Fix Gradle test. * Add some string resource checks/tools. Also, fixes major performance issue with all file-based CI checks. * Ensure newline consistency in translated strings. Also, fix reporting in new validation check script. * Add tests & fix static checks. * Follow-up adjustments after self-review. --- .github/workflows/static_checks.yml | 5 + app/src/main/res/values-ar/strings.xml | 10 +- app/src/main/res/values-sw/strings.xml | 238 +++++++-------- app/src/main/res/values/strings.xml | 2 +- scripts/BUILD.bazel | 14 + .../oppia/android/scripts/common/BUILD.bazel | 3 + .../android/scripts/common/RepositoryFile.kt | 8 +- .../org/oppia/android/scripts/xml/BUILD.bazel | 32 +- .../xml/StringLanguageTranslationCheck.kt | 45 +++ .../scripts/xml/StringResourceParser.kt | 109 +++++++ .../xml/StringResourceValidationCheck.kt | 76 +++++ .../org/oppia/android/scripts/xml/BUILD.bazel | 35 ++- .../xml/StringLanguageTranslationCheckTest.kt | 270 +++++++++++++++++ .../scripts/xml/StringResourceParserTest.kt | 279 ++++++++++++++++++ .../xml/StringResourceValidationCheckTest.kt | 259 ++++++++++++++++ third_party/maven_install.json | 12 +- third_party/versions.bzl | 2 +- 17 files changed, 1266 insertions(+), 133 deletions(-) create mode 100644 scripts/src/java/org/oppia/android/scripts/xml/StringLanguageTranslationCheck.kt create mode 100644 scripts/src/java/org/oppia/android/scripts/xml/StringResourceParser.kt create mode 100644 scripts/src/java/org/oppia/android/scripts/xml/StringResourceValidationCheck.kt create mode 100644 scripts/src/javatests/org/oppia/android/scripts/xml/StringLanguageTranslationCheckTest.kt create mode 100644 scripts/src/javatests/org/oppia/android/scripts/xml/StringResourceParserTest.kt create mode 100644 scripts/src/javatests/org/oppia/android/scripts/xml/StringResourceValidationCheckTest.kt diff --git a/.github/workflows/static_checks.yml b/.github/workflows/static_checks.yml index ff531ffbebb..f9dfbaa86fb 100644 --- a/.github/workflows/static_checks.yml +++ b/.github/workflows/static_checks.yml @@ -180,6 +180,11 @@ jobs: gh issue list --limit 2000 --repo oppia/oppia-android --json number > $(pwd)/open_issues.json bazel run //scripts:todo_open_check -- $(pwd) scripts/assets/todo_open_exemptions.pb open_issues.json + - name: String Resource Validation Check + if: always() + run: | + bazel run //scripts:string_resource_validation_check -- $(pwd) + # Note that caching is intentionally not enabled for this check since licenses should always be # verified without any potential influence from earlier builds (i.e. always from a clean build to # ensure the results exactly match the current state of the repository). diff --git a/app/src/main/res/values-ar/strings.xml b/app/src/main/res/values-ar/strings.xml index 087958c60f5..fc9d006d5ea 100644 --- a/app/src/main/res/values-ar/strings.xml +++ b/app/src/main/res/values-ar/strings.xml @@ -443,13 +443,13 @@ لماذا لا يتم تشغيل الصوت الخاص بي؟ كيف يمكنني تنزيل موضوع؟ لا أجد سؤالي هنا. ماذا الان؟ - <p>إذا كانت هذه هي المرة الأولى التي تنشئ فيها ملفًا شخصيًا وليس لديك رقم تعريف شخصي: </p> \n<p> 1. من منتقي الملف الشخصي ، اضغط على\n<strong> قم بإعداد ملفات تعريف متعددة</strong>\n</p>\n<p> 2. قم بإنشاء رقم تعريف شخصي و\n<strong>احفظ</strong>\n</p> \n<p> 3. املأ جميع البيانات للملف الشخصي.</p> \n<ol> \n<li>(اختياري) قم بتحميل صورة.</li> \n<li>إدخال اسم.</li> \n<li>(اختياري) قم بتعيين رقم تعريف شخصي مكون من 3 أرقام.</li> \n</ol> \n<p> 4. اضغط\n<strong>إنشاء</strong> . تمت إضافة هذا الملف الشخصي إلى منتقي ملف التعريف الخاص بك!\n<br/> \n<br/> إذا قمت بإنشاء ملف تعريف من قبل ولديك رقم تعريف شخصي:\n</p>\n<p> 1. من منتقي الملف الشخصي ، اضغط على\n<strong>إضافة الملف الشخصي</strong>\n</p> \n<p> 2. أدخل رقم التعريف الشخصي الخاص بك وانقر فوق\n<strong>إرسال</strong>\n</p> \n<p>3. املأ جميع الحقول للملف الشخصي.</p> \n<ol> \n<li>(اختياري) قم بتحميل صورة.</li> \n<li>إدخال اسم.</li> \n<li>(اختياري) قم بتعيين رقم تعريف شخصي مكون من 3 أرقام.</li> \n</ol> \n<p> 4. اضغط\n<strong>إنشاء</strong> . تمت إضافة هذا الملف الشخصي إلى منتقي ملف التعريف الخاص بك!\n<br/><br/> ملاحظة: فقط ال\n<u>مدير</u> قادر على إدارة الملفات الشخصية.\n</p> - <p>بمجرد حذف ملف التعريف:</p>\n<p><br></p> \n<p>\n<li>لا يمكن استعادة ملف التعريف.</li>\n</p>\n<p><li>سيتم حذف معلومات الملف الشخصي مثل الاسم والصور والتقدم بشكل دائم.</li></p>\n<p><br></p>\n<p>لحذف ملف تعريف (باستثناء<u>المسؤول</u></p>\n<p>1. من الصفحة الرئيسية للمسؤول ، اضغط على زر القائمة أعلى اليسار.</p>\n<p>2. اضغط على<strong>ضوابط المسؤول</strong></p>\n<p>3. اضغط على<strong>تحرير ملفات التعريف</strong></p>\n<p>4. اضغط على الملف الشخصي الذي ترغب في حذفه.</p>\n<p>5. في الجزء السفلي من الشاشة ، انقر فوق<strong>حذف الملف الشخصي</strong></p>\n<p>6. اضغط<strong>حذف</strong>لتأكيد الحذف.</p>\n<p><br></p>\n<p>ملاحظة:<u>المسؤول</u>فقط هو القادر على إدارة الملفات الشخصية.</p> + <p>إذا كانت هذه هي المرة الأولى التي تنشئ فيها ملفًا شخصيًا وليس لديك رقم تعريف شخصي: </p> <p> 1. من منتقي الملف الشخصي ، اضغط على<strong> قم بإعداد ملفات تعريف متعددة</strong></p><p> 2. قم بإنشاء رقم تعريف شخصي و<strong>احفظ</strong></p> <p> 3. املأ جميع البيانات للملف الشخصي.</p> <ol> <li>(اختياري) قم بتحميل صورة.</li> <li>إدخال اسم.</li> <li>(اختياري) قم بتعيين رقم تعريف شخصي مكون من 3 أرقام.</li> </ol> <p> 4. اضغط<strong>إنشاء</strong> . تمت إضافة هذا الملف الشخصي إلى منتقي ملف التعريف الخاص بك!<br/> <br/> إذا قمت بإنشاء ملف تعريف من قبل ولديك رقم تعريف شخصي:</p><p> 1. من منتقي الملف الشخصي ، اضغط على<strong>إضافة الملف الشخصي</strong></p> <p> 2. أدخل رقم التعريف الشخصي الخاص بك وانقر فوق<strong>إرسال</strong></p> <p>3. املأ جميع الحقول للملف الشخصي.</p> <ol> <li>(اختياري) قم بتحميل صورة.</li> <li>إدخال اسم.</li> <li>(اختياري) قم بتعيين رقم تعريف شخصي مكون من 3 أرقام.</li> </ol> <p> 4. اضغط<strong>إنشاء</strong> . تمت إضافة هذا الملف الشخصي إلى منتقي ملف التعريف الخاص بك!<br/><br/> ملاحظة: فقط ال<u>مدير</u> قادر على إدارة الملفات الشخصية.</p> + <p>بمجرد حذف ملف التعريف:</p><p><br></p> <p><li>لا يمكن استعادة ملف التعريف.</li></p><p><li>سيتم حذف معلومات الملف الشخصي مثل الاسم والصور والتقدم بشكل دائم.</li></p><p><br></p><p>لحذف ملف تعريف (باستثناء<u>المسؤول</u></p><p>1. من الصفحة الرئيسية للمسؤول ، اضغط على زر القائمة أعلى اليسار.</p><p>2. اضغط على<strong>ضوابط المسؤول</strong></p><p>3. اضغط على<strong>تحرير ملفات التعريف</strong></p><p>4. اضغط على الملف الشخصي الذي ترغب في حذفه.</p><p>5. في الجزء السفلي من الشاشة ، انقر فوق<strong>حذف الملف الشخصي</strong></p><p>6. اضغط<strong>حذف</strong>لتأكيد الحذف.</p><p><br></p><p>ملاحظة:<u>المسؤول</u>فقط هو القادر على إدارة الملفات الشخصية.</p> <p>لتغيير بريدك الإلكتروني / رقم هاتفك:</p> <p>1. من الصفحة الرئيسية للمشرف ، اضغط على زر القائمة أعلى اليسار.</p> <p>2. اضغط على <strong> عناصر تحكم المسؤول </ strong>.</p> <p>3. اضغط على <strong> تعديل الحساب </ strong>.</p> <p><br></p> <p>إذا كنت تريد تغيير بريدك الإلكتروني:</p> <p>4. أدخل بريدك الإلكتروني الجديد وانقر على <strong> حفظ </ strong>.</p> <p>5. يتم إرسال رابط التأكيد لتأكيد بريدك الإلكتروني الجديد. ستنتهي صلاحية الرابط بعد 24 ساعة ويجب النقر عليه لربطه بحسابك.</p> <p><br></p> <p>في حالة تغيير رقم هاتفك: </ p> <p> 4. أدخل رقم هاتفك الجديد وانقر على <strong> تحقق </ strong>.</p> <p>5. يتم إرسال رمز لتأكيد رقمك الجديد. ستنتهي صلاحية الرمز بعد 5 دقائق ويجب إدخاله في الشاشة الجديدة لربطه بحسابك.</p> - <p>%1$s \n<i>\"أو-بي-يا\"</i>(فنلندية) - \"للتعلم\"</p>\n<p><br></p><p>%1$sمهمتنا هي مساعدة أي شخص على تعلم أي شيء يريده بطريقة فعالة وممتعة.</p><p><br></p><p>من خلال إنشاء مجموعة من الدروس المجانية عالية الجودة والفعالة بشكل واضح بمساعدة معلمين من جميع أنحاء العالم ، تهدف %1$s إلى تزويد الطلاب بتعليم جيد - بغض النظر عن مكان وجودهم أو الموارد التقليدية التي يمكنهم الوصول إليها.</p><p><br></p><p>كطالب ، يمكنك أن تبدأ مغامرتك التعليمية من خلال تصفح الموضوعات المدرجة في الصفحة الرئيسية!</p> + <p>%1$s <i>\"أو-بي-يا\"</i>(فنلندية) - \"للتعلم\"</p><p><br></p><p>%1$sمهمتنا هي مساعدة أي شخص على تعلم أي شيء يريده بطريقة فعالة وممتعة.</p><p><br></p><p>من خلال إنشاء مجموعة من الدروس المجانية عالية الجودة والفعالة بشكل واضح بمساعدة معلمين من جميع أنحاء العالم ، تهدف %1$s إلى تزويد الطلاب بتعليم جيد - بغض النظر عن مكان وجودهم أو الموارد التقليدية التي يمكنهم الوصول إليها.</p><p><br></p><p>كطالب ، يمكنك أن تبدأ مغامرتك التعليمية من خلال تصفح الموضوعات المدرجة في الصفحة الرئيسية!</p> <p>المشرف هو المستخدم الرئيسي الذي يدير ملفات التعريف والإعدادات لكل ملف تعريف على حسابه. هم على الأرجح والدك أو معلمك أو وصي عليك الذي أنشأ هذا الملف الشخصي لك.</p><p><br></p><p>يمكن للمسؤولين إدارة الملفات الشخصية وتعيين أرقام التعريف الشخصية وتغيير الإعدادات الأخرى ضمن حساباتهم. بناءً على ملف التعريف الخاص بك ، قد تكون أذونات المسؤول مطلوبة لبعض الميزات مثل تنزيل الموضوعات وتغيير رقم التعريف الشخصي وغير ذلك.</p><p><br></p><p>لمعرفة من هو المسؤول لديك ، انتقل إلى منتقي الملف الشخصي. الملف الشخصي الأول المدرج ولديه \"المسؤول\" مكتوب باسمه هو المسؤول.</p> <p>إذا لم يتم تحميل مشغل الاستكشاف</p><p><br></p><p>تحقق لمعرفة ما إذا كان التطبيق محدثًا أم لا:</p><p> <ol> <li> انتقل إلى متجر Play وتأكد من تحديث التطبيق إلى أحدث إصدار </li> </ol> </p><p><br></p><p>تحقق من اتصالك بالإنترنت:</p><p> <li> إذا كان اتصالك بالإنترنت بطيئًا ، فحاول إعادة الاتصال بشبكة Wi-Fi أو الاتصال بشبكة أخرى. </li> </p><p><br></p><p>اطلب من المشرف التحقق من أجهزتهم واتصال الإنترنت:</p><p> <li> اطلب من المشرف استكشاف الأخطاء وإصلاحها باستخدام الخطوات المذكورة أعلاه </li> </p><p><br></p><p>أخبرنا إذا كنت لا تزال تواجه مشكلات في التحميل:</p><p> <li> أبلغ عن مشكلة عن طريق الاتصال بنا على admin@oppia.org. </li> </p> - <p>إذا لم يتم تشغيل الصوت الخاص بك</p><p><br></p>\n<p>تحقق لمعرفة ما إذا كان التطبيق محدثًا أم لا:</p>\n<p> <li>انتقل إلى متجر Play وتأكد من تحديث التطبيق إلى أحدث إصدار</li> </p><p><br></p>\n<p>تحقق من اتصالك بالإنترنت:</p><p> <li>إذا كان اتصالك بالإنترنت بطيئًا ، فحاول إعادة الاتصال بشبكة Wi-Fi أو الاتصال بشبكة أخرى. قد يتسبب الإنترنت البطيء في تحميل الصوت بشكل غير منتظم ، مما يجعل من الصعب تشغيله.</li> </p><p><br></p>\n<p>اطلب من المسؤول التحقق من أجهزتهم واتصال الإنترنت:</p><p> \n<li>اطلب من المسؤول استكشاف الأخطاء وإصلاحها باستخدام الخطوات المذكورة أعلاه</li> </p><p><br></p>\n<p>أخبرنا إذا كنت لا تزال تواجه مشكلات في التحميل:</p><p> <li>أبلغ عن مشكلة عن طريق الاتصال بنا على admin@oppia.org.</li></p> - <p>لتنزيل استكشاف:</p>\n<p>1. من الصفحة الرئيسية ، انقر فوق موضوع أو استكشاف.</p>\n<p>2. من صفحة الموضوع هذه ، انقر على علامة التبويب <strong> معلومات </ strong>.</p>\n<p>3. اضغط على <strong> تنزيل الموضوع </ strong>.</p>\n<p>4. اعتمادًا على إعدادات التطبيق ، قد تحتاج إلى موافقة المسؤول أو اتصال Wifi ثابتًا لإكمال التنزيل. إذا لزم الأمر ، بمجرد استيفاء هذه المتطلبات ، يتم تنزيل الموضوع على الجهاز ويمكن استخدامه في وضع عدم الاتصال بواسطة جميع ملفات التعريف. <p> + <p>إذا لم يتم تشغيل الصوت الخاص بك</p><p><br></p><p>تحقق لمعرفة ما إذا كان التطبيق محدثًا أم لا:</p><p> <li>انتقل إلى متجر Play وتأكد من تحديث التطبيق إلى أحدث إصدار</li> </p><p><br></p><p>تحقق من اتصالك بالإنترنت:</p><p> <li>إذا كان اتصالك بالإنترنت بطيئًا ، فحاول إعادة الاتصال بشبكة Wi-Fi أو الاتصال بشبكة أخرى. قد يتسبب الإنترنت البطيء في تحميل الصوت بشكل غير منتظم ، مما يجعل من الصعب تشغيله.</li> </p><p><br></p><p>اطلب من المسؤول التحقق من أجهزتهم واتصال الإنترنت:</p><p> <li>اطلب من المسؤول استكشاف الأخطاء وإصلاحها باستخدام الخطوات المذكورة أعلاه</li> </p><p><br></p><p>أخبرنا إذا كنت لا تزال تواجه مشكلات في التحميل:</p><p> <li>أبلغ عن مشكلة عن طريق الاتصال بنا على admin@oppia.org.</li></p> + <p>لتنزيل استكشاف:</p><p>1. من الصفحة الرئيسية ، انقر فوق موضوع أو استكشاف.</p><p>2. من صفحة الموضوع هذه ، انقر على علامة التبويب <strong> معلومات </ strong>.</p><p>3. اضغط على <strong> تنزيل الموضوع </ strong>.</p><p>4. اعتمادًا على إعدادات التطبيق ، قد تحتاج إلى موافقة المسؤول أو اتصال Wifi ثابتًا لإكمال التنزيل. إذا لزم الأمر ، بمجرد استيفاء هذه المتطلبات ، يتم تنزيل الموضوع على الجهاز ويمكن استخدامه في وضع عدم الاتصال بواسطة جميع ملفات التعريف. <p> <p> إذا لم تتمكن من العثور على سؤالك أو كنت ترغب في الإبلاغ عن خطأ ، فاتصل بنا على admin@oppia.org. </p> diff --git a/app/src/main/res/values-sw/strings.xml b/app/src/main/res/values-sw/strings.xml index e258e430b69..140af141b6b 100644 --- a/app/src/main/res/values-sw/strings.xml +++ b/app/src/main/res/values-sw/strings.xml @@ -109,15 +109,15 @@ Samahani,vipeo vingi havitumiki na programu.Tafadhali rekebisha jibu lako. Samahani, nguvu za juu zaidi ya 5 haiungwi mkono na programu. Tafadhali rekebisha jibu lako. Samahani, nguvu zinazorudiwa/vielezo hazihimiliwi na programu. Tafadhali punguza jibu lako kwa nguvu moja. - \nPembejeo haipo kwa mzizi wa mraba. - \nKugawanya kwa sufuri si halali. Tafadhali rekebisha jibu lako. + Pembejeo haipo kwa mzizi wa mraba. + Kugawanya kwa sufuri si halali. Tafadhali rekebisha jibu lako. Inaonekana umeingiza baadhi ya vigezo. Tafadhali hakikisha kuwa jibu lako lina nambari pekee na uondoe vigezo vyovyote kutoka kwa jibu lako. - \nTafadhali tumia vigezo vilivyobainishwa katika swali na si %s. + Tafadhali tumia vigezo vilivyobainishwa katika swali na si %s. Mlinganyo yako inakosa ishara \'=\'. - \nMlinganyo wako una ishara nyingi sana \'=\'. Inapaswa kuwa na moja tu. - \nMoja ya pande za \'=\' katika mlinganyo wako ni tupu. - \nChaguo la kukokotoa \'%s\' haitumiki. Tafadhali rekebisha jibu lako. - \nUlimaanisha sqrt? Ikiwa sivyo, tafadhali tenganisha vigeu kwa kutumia alama za kuzidisha. + Mlinganyo wako una ishara nyingi sana \'=\'. Inapaswa kuwa na moja tu. + Moja ya pande za \'=\' katika mlinganyo wako ni tupu. + Chaguo la kukokotoa \'%s\' haitumiki. Tafadhali rekebisha jibu lako. + Ulimaanisha sqrt? Ikiwa sivyo, tafadhali tenganisha vigeu kwa kutumia alama za kuzidisha. Samahani, hatukuweza kuelewa jibu lako. Tafadhali iangalie ili kuhakikisha kuwa hakuna hitilafu zozote. Washa sauti kwa somo hili. Hadithi Zilizochezwa hivi karibuni @@ -157,21 +157,21 @@ Jibu lako lina koloni mbili (:) karibu na kila moja. Idadi ya masharti si sawa na masharti yanayohitajika. Uwiano hauwezi kuwa na 0 kama kipengele. - \nUkubwa usiojulikana - \n Baiti %s + Ukubwa usiojulikana + Baiti %s %s KB %s MB %s GB - \nSahihi! - \nMada: %s - \n%1$s katika %2$s + Sahihi! + Mada: %s + %1$s katika %2$s Sura 1\n - \n\n Sura %s \n + \n Sura %s \n Hadithi 1\n - \n Hadithi %s\n + Hadithi %s\n %s kati ya Sura %s Imekamilika @@ -179,104 +179,104 @@ Somo 1\n - \n Masomo %s \n + Masomo %s \n - \nUkurasa wa kuchagua wasifu - \nMsimamizi - \nChagua wasifu wako - \nOngeza Wasifu + Ukurasa wa kuchagua wasifu + Msimamizi + Chagua wasifu wako + Ongeza Wasifu Weka Wasifu Nyingi - \nOngeza hadi watumiaji 10 kwenye akaunti yako. Nzuri sana kwa familia na madarasa. - \nUdhibiti vya Msimamizi + Ongeza hadi watumiaji 10 kwenye akaunti yako. Nzuri sana kwa familia na madarasa. + Udhibiti vya Msimamizi Lugha Vidhibiti vya Msimamizi - \nIdhinisha kuongeza wasifu + Idhinisha kuongeza wasifu Idhinisha kufikia Vidhibiti vya Msimamizi - \nIdhini ya Msimamizi Inahitajika + Idhini ya Msimamizi Inahitajika Weka Nambari ya Siri ya Msimamizi ili kuunda akaunti mpya. Weka Nambari ya Siri ya Msimamizi ili kufikia Vidhibiti vya Msimamizi. - \nNambari ya Siri ya Msimamizi + Nambari ya Siri ya Msimamizi Nambari ya Siri ya Msimamizi si Sahihi. Tafadhali jaribu tena. - \nTafadhali weka Nambari ya Siri ya Msimamizi. - \nWasilisha + Tafadhali weka Nambari ya Siri ya Msimamizi. + Wasilisha Funga - \nKabla ya kuongeza wasifu, tunahitaji kulinda akaunti yako kwa kuunda Nambari ya Siri. Hii hukupa uwezo wa kuidhinisha upakuaji na kudhibiti wasifu kwenye kifaa. - \nTumia Nambari yako ya siri uliyoweka kwa akaunti za kibinafsi kama vile benki au usalama wa kijamii. - \nNambari mpya ya Siri ya tarakimu 5 + Kabla ya kuongeza wasifu, tunahitaji kulinda akaunti yako kwa kuunda Nambari ya Siri. Hii hukupa uwezo wa kuidhinisha upakuaji na kudhibiti wasifu kwenye kifaa. + Tumia Nambari yako ya siri uliyoweka kwa akaunti za kibinafsi kama vile benki au usalama wa kijamii. + Nambari mpya ya Siri ya tarakimu 5 Thibitisha Nambari ya Siri ya tarakimu 5 - \nNambari yako ya Siri inapaswa kuwa na urefu wa tarakimu 5. - \nTafadhali hakikisha kwamba Nambari za Siri zote mbili zinalingana. + Nambari yako ya Siri inapaswa kuwa na urefu wa tarakimu 5. + Tafadhali hakikisha kwamba Nambari za Siri zote mbili zinalingana. Hifadhi - \nIdhinisha kuongeza wasifu - \nOngeza Wasifu - \nOngeza Wasifu + Idhinisha kuongeza wasifu + Ongeza Wasifu + Ongeza Wasifu Jina* - \nNambari ya Siri ya tarakimu 3* + Nambari ya Siri ya tarakimu 3* Thibitisha Nambari ya Siri ya tarakimu 3* - \nRuhusu Upakuaji wa Ufikiaji - \nMtumiaji anaweza kupakua na kufuta maudhui bila Nambari ya Siri ya Msimamizi. + Ruhusu Upakuaji wa Ufikiaji + Mtumiaji anaweza kupakua na kufuta maudhui bila Nambari ya Siri ya Msimamizi. Anzisha Funga - \nUkiwa na Nambari ya Siri, hakuna mtu mwingine anayeweza kufikia wasifu kando na mtumiaji huyu aliyekabidhiwa. - \nTumeshindwa kuhifadhi picha yako. Tafadhali jaribu tena. + Ukiwa na Nambari ya Siri, hakuna mtu mwingine anayeweza kufikia wasifu kando na mtumiaji huyu aliyekabidhiwa. + Tumeshindwa kuhifadhi picha yako. Tafadhali jaribu tena. Jina hili tayari linatumiwa na wasifu mwingine. Tafadhali weka jina la wasifu huu. - \nMajina yanaweza kuwa na herufi pekee. Jaribu jina lingine? - \nNambari yako ya Siri inapaswa kuwa na urefu wa tarakimu 3. - \nTafadhali hakikisha kwamba Nambari za Siri zote mbili zinalingana. - \nMaelezo zaidi kuhusu Nambari za Siri zenye tarakimu 3. - \nSehemu zilizo na alama ya * zinahitajika. + Majina yanaweza kuwa na herufi pekee. Jaribu jina lingine? + Nambari yako ya Siri inapaswa kuwa na urefu wa tarakimu 3. + Tafadhali hakikisha kwamba Nambari za Siri zote mbili zinalingana. + Maelezo zaidi kuhusu Nambari za Siri zenye tarakimu 3. + Sehemu zilizo na alama ya * zinahitajika. Picha ya sasa ya wasifu - \nHariri picha ya wasifu - \nKaribu kwa %s! + Hariri picha ya wasifu + Karibu kwa %s! Jifunze chochote unachotaka kwa njia bora na ya kufurahisha. Ongeza watumiaji kwenye akaunti yako. Elezea uzoefu na uunde hadi wasifu 10. - \nPakua ya nje ya mtandao. + Pakua ya nje ya mtandao. Endelea kujifunza masomo yako bila muunganisho wa mtandao. - \nFurahia! + Furahia! Furahia matukio yako ya kujifunza kwa masomo yetu ya bure na ya ufanisi. Ruka Inayofuata Anza - \nSlaidi %s ya %s - \nHabari, %s! + Slaidi %s ya %s + Habari, %s! Tafadhali weka Nambari yako ya Siri ya Msimamizi. - \nTafadhali weka Nambari yako ya Siri. - \nNambari ya Siri ya Tarakimu 5 ya Msimamizi. - \nNambari ya Siri ya Tarakimu 3 ya Mtumiaji. + Tafadhali weka Nambari yako ya Siri. + Nambari ya Siri ya Tarakimu 5 ya Msimamizi. + Nambari ya Siri ya Tarakimu 3 ya Mtumiaji. Nilisahau nambari yangu. Nambari ya Siri si sahihi. Onyesha ficha Funga - \nMabadiliko ya Nambari ya Siri yamefaulu + Mabadiliko ya Nambari ya Siri yamefaulu Je, umesahau Nambari ya Siri? Ili kuweka upya Nambari yako ya Siri, tafadhali ondoa %s kisha uisakinishe upya.\n\nKumbuka kwamba ikiwa kifaa hakijakuwa mtandaoni, unaweza kupoteza maendeleo ya mtumiaji kwenye akaunti nyingi. Nenda kwenye hifadhi ya Google play. - \nOnyesha/Ficha ishara ya nenosiri - \nIshara ya nenosiri iliyoonyeshwa - \nIshara ya nenosiri iliyofichwa - \nWeka Nambari yako ya Siri - \nWeka Nambari ya Siri + Onyesha/Ficha ishara ya nenosiri + Ishara ya nenosiri iliyoonyeshwa + Ishara ya nenosiri iliyofichwa + Weka Nambari yako ya Siri + Weka Nambari ya Siri Nambari ya Siri ya Msimamizi Ufikiaji wa mipangilio ya msimamizi - \nNambari ya Siri ya Msimamizi inahitajika ili kubadilisha Nambari ya Siri ya mtumiaji + Nambari ya Siri ya Msimamizi inahitajika ili kubadilisha Nambari ya Siri ya mtumiaji Ghairi Wasilisha Nambari ya Siri ya msimamizi si Sahihi. Tafadhali jaribu tena. - \nNambari ya Siri mpya ya %1$s. - \nWeka Nambari mpya ya Siri - \nVipakuliwa Vyangu - \nVipakuliwa + Nambari ya Siri mpya ya %1$s. + Weka Nambari mpya ya Siri + Vipakuliwa Vyangu + Vipakuliwa Sasisho (2) Je, ungependa kuondoka kwenye wasifu wako? Ghairi Toka Mwanzo - \nWasifu + Wasifu Ilitengenezwa kwa %s - \nMara ya mwisho kutumika + Mara ya mwisho kutumika Badilisha jina Weka upya Nambari ya Siri Ufutaji wa Wasifu @@ -284,8 +284,8 @@ Maendeleo yote yatafutwa na hayawezi kurejeshwa. Futa Ghairi - \nRuhusu Ufikiaji wa Upakuaji - \nMtumiaji anaweza kupakua na kufuta maudhui bila Nambari ya Siri ya msimamizi. + Ruhusu Ufikiaji wa Upakuaji + Mtumiaji anaweza kupakua na kufuta maudhui bila Nambari ya Siri ya msimamizi. Picha ya Wasifu Picha ya Wasifu Ghairi @@ -296,51 +296,51 @@ Hifadhi Weka upya Nambari ya Siri Weka Nambari mpya ya Siri ili mtumiaji aiweke anapofikia wasifu wake. - \nNambari ya Siri ya tarakimu 3* - \nNambari ya Siri ya tarakimu 5* + Nambari ya Siri ya tarakimu 3* + Nambari ya Siri ya tarakimu 5* Thibitisha Nambari ya Siri ya tarakimu 3* Thibitisha Nambari ya Siri ya tarakimu 5 - \nNambari yako ya Siri inapaswa kuwa na urefu wa tarakimu 3. - \nNambari yako ya Siri inapaswa kuwa na urefu wa tarakimu 5. + Nambari yako ya Siri inapaswa kuwa na urefu wa tarakimu 3. + Nambari yako ya Siri inapaswa kuwa na urefu wa tarakimu 5. Unda Nambari ya Siri ya tarakimu 3 - \nInahitajika - \nKitufe cha Nyuma + Inahitajika + Kitufe cha Nyuma Inayofuata Jumla - \nHariri akaunti - \nUsimamizi wa Wasifu + Hariri akaunti + Usimamizi wa Wasifu Hariri wasifu - \nRuhusa ya kupakua + Ruhusa ya kupakua Pakua na usasishe kwenye Wi-fi pekee Mada zitapakuliwa na kusasishwa kwenye Wi-fi pekee. Vipakuliwa au masasisho yoyote ya data ya mtandao wa simu yatawekwa kwenye foleni. Sasisha mada moja kwa moja - \nMada zilizopakuliwa ambazo zina maudhui mapya zinazopatikana zitasasishwa moja kwa moja. + Mada zilizopakuliwa ambazo zina maudhui mapya zinazopatikana zitasasishwa moja kwa moja. Maelezo ya programu. Toleo la Programu - \nVitendo vya akaunti + Vitendo vya akaunti Toka Ghairi Sawa Je,una uhakika unataka kutoka kwenye wasifu wako? - \nToleo la Programu %s + Toleo la Programu %s Sasisho la mwisho lilisakinishwa kwenye %s. Tumia nambari ya toleo iliyo hapo juu kutuma maoni kuhusu hitilafu. Toleo la Programu Lugha ya Programu Lugha chaguomsingi ya Sauti Ukubwa wa Maandishi ya Kusoma Ukubwa wa Maandishi ya Kusoma - \nNakala ya hadithi itaonekana hivi. + Nakala ya hadithi itaonekana hivi. A - \nSauti Chaguomsingi + Sauti Chaguomsingi Lugha ya Programu Ukubwa wa Maandishi ya Kusoma Ndogo Kati Kubwa - \nKubwa Zaidi - \nTelezesha upau wa utafutaji ili kudhibiti ukubwa wa maandishi. - \nWasifu - \nHadithi 2 + Kubwa Zaidi + Telezesha upau wa utafutaji ili kudhibiti ukubwa wa maandishi. + Wasifu + Hadithi 2 Mada zinazoendelea Mada zinazoendelea Hadithi Zimekamilika @@ -348,39 +348,39 @@ Chaguzi Hadithi Zimekamilika Mwongozo wa programu - \nJifunze ujuzi mpya wa hesabu katika hadithi zinazokuonyesha jinsi ya kuzitumia katika maisha yako ya kila siku - \n\"Karibu %s!\" + Jifunze ujuzi mpya wa hesabu katika hadithi zinazokuonyesha jinsi ya kuzitumia katika maisha yako ya kila siku + \"Karibu %s!\" Unataka kujifunza nini? - \nMakuu - Hebu \ntuanze. + Makuu + Hebu tuanze. Ndiyo - \nHapana… - \nChagua \nmada tofauti. + Hapana… + Chagua \nmada tofauti. Je, unavutiwa na:\n%s? Kidokezo kipya kinapatikana - \nOnyesha vidokezo na suluhisho + Onyesha vidokezo na suluhisho Nenda juu Vidokezo Fichua Suluhisho - \nFichua Kidokezo + Fichua Kidokezo Onyesha/Ficha orodha ya vidokezo ya %s - \nOnyesha/Ficha suluhisho + Onyesha/Ficha suluhisho Suluhisho pekee ni: Hii itafichua suluhisho. Una uhakika? Fichua sasa hivi - \nhivi karibuni + hivi karibuni %s iliyopita - \njana + jana Rudi kwenye mada Ufafanuzi: - \nIkiwa vitu viwili ni sawa, viunganishe. + Ikiwa vitu viwili ni sawa, viunganishe. Unganisha na kipengee %s Tenganisha vipengee katika %s Hamisha kipengee chini hadi %s - \nHamisha kipengee juu hadi %s + Hamisha kipengee juu hadi %s Juu - \nChini + Chini %s %s dakika @@ -395,16 +395,16 @@ siku %s mada_marudio_mtazamo wa kuchakata tena_tag - \nInaendelea_kuchakata tena_mtazamo_tag - \nTafadhali chagua angalau chaguo moja. - \nToleo la programu lisilotumika + Inaendelea_kuchakata tena_mtazamo_tag + Tafadhali chagua angalau chaguo moja. + Toleo la programu lisilotumika Toleo hili la programu halitumiki tena. Tafadhali isasishe kupitia hifadhi ya michezo. Funga programu - \nkwa - \nWeka uwiano katika fomu x:y. + kwa + Weka uwiano katika fomu x:y. Maandishi madogo zaidi Maandishi kubwa zaidi - \nInakuja Hivi Karibuni + Inakuja Hivi Karibuni Hadithi Zinazopendekezwa Hadithi Kwa Ajili Yako Hali ya Mazoezi @@ -416,28 +416,28 @@ Jibu sahihi lililowasilishwa Jibu sahihi lililowasilishwa: %s Jibu lililowasilishwa lisilo sahihi - \nJibu lililowasilishwa lisilo sahihi: %s + Jibu lililowasilishwa lisilo sahihi: %s Tegemeo ya mhusika wa tatu toleo %s Leseni za Hakimiliki - \nMtazamo wa Leseni ya Hakimiliki - \nNenda nyuma hadi %s - \norodha ya tegemezi ya mhusika wa tatu + Mtazamo wa Leseni ya Hakimiliki + Nenda nyuma hadi %s + orodha ya tegemezi ya mhusika wa tatu orodha ya leseni za hakimiliki - \nRejesha Somo + Rejesha Somo Endelea - \nAnza tena - \nHabari ya asubuhi, - \nHabari ya mchana, + Anza tena + Habari ya asubuhi, + Habari ya mchana, Habari ya jioni, - \nNinawezaje kuunda wasifu mpya? - \nNinawezaje kufuta wasifu? + Ninawezaje kuunda wasifu mpya? + Ninawezaje kufuta wasifu? Nitabadilisha aje barua pepe/nambari yangu ya simu? - \n%s ni nini? + %s ni nini? Msimamizi ni nani? - \nKwa nini kicheza Uchunguzi hakipakii? + Kwa nini kicheza Uchunguzi hakipakii? Kwa nini sauti yangu haichezwi? - \nNitapakua aje Mada? + Nitapakua aje Mada? Sijapata swali langu hapa. Nini sasa? <p>Ikiwa ni mara yako ya kwanza kuunda wasifu na huna Nambari ya Siri: </p> <p> 1. Kutoka kwa Kichagua Wasifu, gusa <strong>Weka Wasifu Nyingi</strong>. </p> <p> 2. Unda Nambari ya Siri na <strong>Hifadhi</strong>. </p> <p> 3. Jaza sehemu zote za wasifu. </p> <ol> <li> (Si lazima) Pakia picha. </li> <li> Weka jina. </li> <li> (Si lazima) Weka Nambari ya Siri yenye tarakimu 3. </li> </ol> <p> 4. Gusa <strong>Unda</strong>. Wasifu huu umeongezwa kwa Kichagua Wasifu wako! <br/> <br/> Ikiwa umeunda wasifu hapo awali na una Nambari ya Siri: </p> <p> 1. Kutoka kwa Kichagua Wasifu, gusa <strong>Ongeza Wasifu</strong>. </p> <p> 2. Weka Nambari yako ya Siri na uguse <strong>Wasilisha</strong>. </p> <p> 3. Jaza sehemu zote za wasifu. </p> <ol> <li> (Si lazima) Pakia picha. </li> <li> Weka jina. </li> <li> (Si lazima) Weka Nambari ya Siri yenye tarakimu 3. </li> </ol> <p> 4. Gusa <strong>Unda</strong>. Wasifu huu umeongezwa kwa Kichagua Wasifu wako! <br/> <br/> Kumbuka: <u>Msimamizi pekee</u> ndiye anayeweza kudhibiti wasifu.</p> <p>Wasifu unapofutwa:</p> <p><br></p> <p> <li> Wasifu hauwezi kurejeshwa. </li> </p> <p> <li> Taarifa ya wasifu kama vile jina, picha na maendeleo yatafutwa kabisa. </li> </p> <p><br></p> <p>Ili kufuta wasifu (bila kujumuisha <u>Msimamizi</u>):</p> <p>1. Kutoka kwa Ukurasa wa Mwanzo wa Msimamizi, gusa kitufe cha menyu kilicho upande wa juu kushoto.</p> <p>2. Gusa <strong>Vidhibiti vya Msimamizi</strong>.</p> <p>3. Gusa <strong>Hariri Wasifu</strong>.</p> <p>4. Gonga Wasifu ambao ungependa kufuta.</p> <p>5. Katika sehemu ya chini ya skrini, gusa <strong>Ufutaji wa Wasifu</strong>.</p> <p>6. Gusa <strong>Futa</strong> ili kuthibitisha kufuta.</p><p><br></p><p>Kumbuka: <u>Msimamizi</u> pekee ndiye anayeweza kudhibiti wasifu.</ p> diff --git a/app/src/main/res/values/strings.xml b/app/src/main/res/values/strings.xml index 145c532470f..fcc62d45eb4 100755 --- a/app/src/main/res/values/strings.xml +++ b/app/src/main/res/values/strings.xml @@ -426,7 +426,7 @@ What do you want to learn? Great - Let’s get \nstarted. + Let’s get started. Yes No… Pick a \ndifferent topic. diff --git a/scripts/BUILD.bazel b/scripts/BUILD.bazel index 2fcfb81eba5..4e8fca5d41e 100644 --- a/scripts/BUILD.bazel +++ b/scripts/BUILD.bazel @@ -92,6 +92,20 @@ kt_jvm_binary( runtime_deps = ["//scripts/src/java/org/oppia/android/scripts/xml:xml_syntax_check_lib"], ) +kt_jvm_binary( + name = "string_language_translation_check", + testonly = True, + main_class = "org.oppia.android.scripts.xml.StringLanguageTranslationCheckKt", + runtime_deps = ["//scripts/src/java/org/oppia/android/scripts/xml:string_language_translation_check_lib"], +) + +kt_jvm_binary( + name = "string_resource_validation_check", + testonly = True, + main_class = "org.oppia.android.scripts.xml.StringResourceValidationCheckKt", + runtime_deps = ["//scripts/src/java/org/oppia/android/scripts/xml:string_resource_validation_check_lib"], +) + TEST_FILE_EXEMPTION_ASSETS = generate_test_file_assets_list_from_text_protos( name = "test_file_exemption_assets", test_file_exemptions_name = ["test_file_exemptions"], diff --git a/scripts/src/java/org/oppia/android/scripts/common/BUILD.bazel b/scripts/src/java/org/oppia/android/scripts/common/BUILD.bazel index 1b3b76ec17b..3a94254e9fc 100644 --- a/scripts/src/java/org/oppia/android/scripts/common/BUILD.bazel +++ b/scripts/src/java/org/oppia/android/scripts/common/BUILD.bazel @@ -53,4 +53,7 @@ kt_jvm_library( testonly = True, srcs = ["RepositoryFile.kt"], visibility = ["//scripts:oppia_script_library_visibility"], + deps = [ + "//third_party:org_jetbrains_kotlin_kotlin-stdlib-jdk8_jar", + ], ) diff --git a/scripts/src/java/org/oppia/android/scripts/common/RepositoryFile.kt b/scripts/src/java/org/oppia/android/scripts/common/RepositoryFile.kt index 1e0565e1dbd..14962f3ee0b 100644 --- a/scripts/src/java/org/oppia/android/scripts/common/RepositoryFile.kt +++ b/scripts/src/java/org/oppia/android/scripts/common/RepositoryFile.kt @@ -1,6 +1,9 @@ package org.oppia.android.scripts.common import java.io.File +import java.nio.file.Files +import java.nio.file.Path +import kotlin.streams.asSequence /** Helper class for managing & accessing files within the project repository. */ class RepositoryFile() { @@ -41,7 +44,10 @@ class RepositoryFile() { expectedExtension: String = "", exemptionsList: List = listOf() ): List { - return File(repoPath).walk().filter { file -> + // Note that Files.walk() is used instead of Kotlin's walk() function since the latter follows + // symbolic links which is almost 10x slower than not following them (due to very deep Bazel + // build directories), and it's not necessary to follow the symlinks. + return Files.walk(File(repoPath).toPath()).asSequence().map(Path::toFile).filter { file -> val isProhibited = checkIfProhibitedFile(retrieveRelativeFilePath(file, repoPath)) !isProhibited && file.isFile && diff --git a/scripts/src/java/org/oppia/android/scripts/xml/BUILD.bazel b/scripts/src/java/org/oppia/android/scripts/xml/BUILD.bazel index 56dc1789827..e31ea659e02 100644 --- a/scripts/src/java/org/oppia/android/scripts/xml/BUILD.bazel +++ b/scripts/src/java/org/oppia/android/scripts/xml/BUILD.bazel @@ -1,10 +1,40 @@ """ Libraries corresponding to XML syntax based check to ensure that all the XML files in the codebase -are syntactically correct. +are syntactically correct and consistent. """ load("@io_bazel_rules_kotlin//kotlin:kotlin.bzl", "kt_jvm_library") +kt_jvm_library( + name = "string_language_translation_check_lib", + testonly = True, + srcs = ["StringLanguageTranslationCheck.kt"], + visibility = ["//scripts:oppia_script_binary_visibility"], + deps = [ + ":string_resource_parser", + ], +) + +kt_jvm_library( + name = "string_resource_parser", + testonly = True, + srcs = ["StringResourceParser.kt"], + visibility = ["//scripts:oppia_script_test_visibility"], + deps = [ + "//scripts/src/java/org/oppia/android/scripts/common:repository_file", + ], +) + +kt_jvm_library( + name = "string_resource_validation_check_lib", + testonly = True, + srcs = ["StringResourceValidationCheck.kt"], + visibility = ["//scripts:oppia_script_binary_visibility"], + deps = [ + ":string_resource_parser", + ], +) + kt_jvm_library( name = "xml_syntax_error_handler", testonly = True, diff --git a/scripts/src/java/org/oppia/android/scripts/xml/StringLanguageTranslationCheck.kt b/scripts/src/java/org/oppia/android/scripts/xml/StringLanguageTranslationCheck.kt new file mode 100644 index 00000000000..35f7e41514b --- /dev/null +++ b/scripts/src/java/org/oppia/android/scripts/xml/StringLanguageTranslationCheck.kt @@ -0,0 +1,45 @@ +package org.oppia.android.scripts.xml + +import java.io.File + +/** + * Script for checking if all strings have been translated across all supported languages. + * + * Usage: + * bazel run //scripts:string_language_translation_check -- + * + * Arguments: + * - path_to_directory_root: directory path to the root of the Oppia Android repository. + * + * Example: + * bazel run //scripts:string_language_translation_check -- $(pwd) + */ +fun main(vararg args: String) { + require(args.isNotEmpty()) { + "Expected: bazel run //scripts:string_language_translation_check -- " + } + + // Path of the repo to be analyzed. + val repoPath = "${args[0]}/" + + val parser = StringResourceParser(File(repoPath)) + val baseTranslations = parser.retrieveBaseStringNames() + val missingTranslations = parser.retrieveAllNonEnglishTranslations().mapValues { (_, xlations) -> + baseTranslations - xlations.strings.keys + } + val missingTranslationCount = missingTranslations.values.sumOf { it.size } + println("$missingTranslationCount translation(s) were found missing.") + if (missingTranslationCount > 0) { + println() + println("Missing translations:") + missingTranslations.forEach { (language, translations) -> + if (translations.isNotEmpty()) { + println("${language.name} (${translations.size}/$missingTranslationCount):") + translations.forEach { translation -> + println("- $translation") + } + println() + } + } + } +} diff --git a/scripts/src/java/org/oppia/android/scripts/xml/StringResourceParser.kt b/scripts/src/java/org/oppia/android/scripts/xml/StringResourceParser.kt new file mode 100644 index 00000000000..955138b337b --- /dev/null +++ b/scripts/src/java/org/oppia/android/scripts/xml/StringResourceParser.kt @@ -0,0 +1,109 @@ +package org.oppia.android.scripts.xml + +import org.oppia.android.scripts.common.RepositoryFile +import org.w3c.dom.Node +import org.w3c.dom.NodeList +import java.io.File +import javax.xml.parsers.DocumentBuilderFactory + +/** + * Parser and processor for all UI-facing string resources, for use in validation and analysis + * scripts. + * + * @property repoRoot the root of the Oppia Android repository being processed + */ +class StringResourceParser(private val repoRoot: File) { + private val translations by lazy { parseTranslations() } + private val documentBuilderFactory by lazy { DocumentBuilderFactory.newInstance() } + + /** Returns the [StringFile] corresponding to the base (i.e. untranslated English) strings. */ + fun retrieveBaseStringFile(): StringFile = translations.getValue(TranslationLanguage.ENGLISH) + + /** Returns the [Set] of all string keys contained within the base strings file. */ + fun retrieveBaseStringNames(): Set = retrieveBaseStringFile().strings.keys + + /** + * Returns a map of all [StringFile]s (keyed by their [StringFile.language]) which represent + * actual translations (i.e. all non-base files--see [retrieveBaseStringFile] for the base + * strings). + */ + fun retrieveAllNonEnglishTranslations(): Map = + translations.filter { (language, _) -> language != TranslationLanguage.ENGLISH } + + private fun parseTranslations(): Map { + // A list of all XML files in the repo to be analyzed. + val stringFiles = RepositoryFile.collectSearchFiles( + repoPath = repoRoot.absolutePath, + expectedExtension = ".xml" + ).filter { + it.toRelativeString(repoRoot).startsWith("app/") && it.nameWithoutExtension == "strings" + }.associateBy { + checkNotNull(it.parentFile?.name?.let(::findTranslationLanguage)) { + "Strings file '${it.toRelativeString(repoRoot)}' does not correspond to a known language:" + + " ${it.parentFile?.name}" + } + }.toSortedMap() // Sorted for consistent output. + val expectedLanguages = TranslationLanguage.values().toSet() + check(expectedLanguages == stringFiles.keys) { + "Missing translation strings for language(s):" + + " ${(expectedLanguages - stringFiles.keys).joinToString() }" + } + return stringFiles.map { (language, file) -> + language to StringFile(language, file, file.parseStrings()) + }.toMap() + } + + private fun File.parseStrings(): Map { + val manifestDocument = documentBuilderFactory.parseXmlFile(this) + val stringsElem = manifestDocument.getChildSequence().single { it.nodeName == "resources" } + val stringElems = stringsElem.getChildSequence().filter { it.nodeName == "string" } + return stringElems.associate { + checkNotNull(it.attributes.getNamedItem("name")?.nodeValue) to checkNotNull(it.textContent) + } + } + + /** + * The language given strings have been translated to/are being represented in. + * + * @property valuesDirectoryName the name of the resource values directory that is expected to + * contain a strings.xml file for strings related to this language + */ + enum class TranslationLanguage(val valuesDirectoryName: String) { + /** Corresponds to Arabic (ar) translations. */ + ARABIC(valuesDirectoryName = "values-ar"), + + /** Corresponds to Brazilian Portuguese (pt-rBR) translations. */ + BRAZILIAN_PORTUGUESE(valuesDirectoryName = "values-pt-rBR"), + + /** Corresponds to English (en) translations. */ + ENGLISH(valuesDirectoryName = "values"), + + /** Corresponds to Swahili (sw) translations. */ + SWAHILI(valuesDirectoryName = "values-sw"); + } + + /** + * A record of a specific set of translations corresponding to one language. + * + * @property language the language of this string file + * @property file the direct [File] to the strings.xml containing the translations + * @property strings a map with keys of string names and values of the actual strings retrieved + * from the strings.xml file + */ + data class StringFile( + val language: TranslationLanguage, + val file: File, + val strings: Map + ) + + private companion object { + private fun DocumentBuilderFactory.parseXmlFile(file: File) = newDocumentBuilder().parse(file) + + private fun Node.getChildSequence() = childNodes.asSequence() + + private fun NodeList.asSequence() = (0 until length).asSequence().map(this::item) + + private fun findTranslationLanguage(valuesDirectoryName: String) = + TranslationLanguage.values().find { it.valuesDirectoryName == valuesDirectoryName } + } +} diff --git a/scripts/src/java/org/oppia/android/scripts/xml/StringResourceValidationCheck.kt b/scripts/src/java/org/oppia/android/scripts/xml/StringResourceValidationCheck.kt new file mode 100644 index 00000000000..5619474b23a --- /dev/null +++ b/scripts/src/java/org/oppia/android/scripts/xml/StringResourceValidationCheck.kt @@ -0,0 +1,76 @@ +package org.oppia.android.scripts.xml + +import org.oppia.android.scripts.xml.StringResourceParser.StringFile +import org.oppia.android.scripts.xml.StringResourceParser.TranslationLanguage +import java.io.File + +/** + * Script for validating consistency between translated and base string resources. + * + * Usage: + * bazel run //scripts:string_resource_validation_check -- + * + * Arguments: + * - path_to_directory_root: directory path to the root of the Oppia Android repository. + * + * Example: + * bazel run //scripts:string_resource_validation_check -- $(pwd) + */ +fun main(vararg args: String) { + require(args.isNotEmpty()) { + "Expected: bazel run //scripts:string_resource_validation_check -- " + } + + // Path of the repo to be analyzed. + val repoPath = "${args[0]}/" + val repoRoot = File(repoPath) + + data class Finding(val language: TranslationLanguage, val file: File, val errorLine: String) + val parser = StringResourceParser(repoRoot) + val baseFile = parser.retrieveBaseStringFile() + val otherTranslations = parser.retrieveAllNonEnglishTranslations() + val inconsistencies = otherTranslations.entries.fold(listOf()) { errors, entry -> + val (_, translatedFile) = entry + errors + computeInconsistenciesBetween(baseFile, translatedFile).map { line -> + Finding(translatedFile.language, translatedFile.file, line) + } + }.groupBy(keySelector = { it.language to it.file }, valueTransform = { it.errorLine }) + + if (inconsistencies.isNotEmpty()) { + println("${inconsistencies.size} language(s) were found with string consistency errors.") + println() + + inconsistencies.forEach { (context, errorLines) -> + val (language, file) = context + println( + "${errorLines.size} consistency error(s) were found for ${language.name} strings (file:" + + " ${file.toRelativeString(repoRoot)}):" + ) + errorLines.forEach { println("- $it") } + println() + } + throw Exception("STRING RESOURCE VALIDATION CHECKS FAILED") + } else println("STRING RESOURCE VALIDATION CHECKS PASSED") +} + +private fun computeInconsistenciesBetween( + baseFile: StringFile, + translatedFile: StringFile +): List { + val commonTranslations = baseFile.strings.intersectWith(translatedFile.strings) + + // Check for inconsistent newlines post-translation. + return commonTranslations.mapNotNull { (stringName, stringPair) -> + val (baseString, translatedString) = stringPair + val baseLines = baseString.split("\\n") + val translatedLines = translatedString.split("\\n") + return@mapNotNull if (baseLines.size != translatedLines.size) { + "string $stringName: original translation uses ${baseLines.size} line(s) but translation" + + " uses ${translatedLines.size} line(s). Please remove any extra lines or add any that are" + + " missing." + } else null // The number of lines match. + } +} + +private fun Map.intersectWith(other: Map) = + keys.intersect(other.keys).associateWith { getValue(it) to other.getValue(it) } diff --git a/scripts/src/javatests/org/oppia/android/scripts/xml/BUILD.bazel b/scripts/src/javatests/org/oppia/android/scripts/xml/BUILD.bazel index 445e7b6d76d..d1170300f87 100644 --- a/scripts/src/javatests/org/oppia/android/scripts/xml/BUILD.bazel +++ b/scripts/src/javatests/org/oppia/android/scripts/xml/BUILD.bazel @@ -1,10 +1,43 @@ """ Tests corresponding to XML syntax based check to ensure that all the XML files in the codebase are -syntactically correct. +syntactically correct and consistent. """ load("@io_bazel_rules_kotlin//kotlin:kotlin.bzl", "kt_jvm_test") +kt_jvm_test( + name = "StringLanguageTranslationCheckTest", + srcs = ["StringLanguageTranslationCheckTest.kt"], + deps = [ + "//scripts/src/java/org/oppia/android/scripts/xml:string_language_translation_check_lib", + "//testing:assertion_helpers", + "//third_party:com_google_truth_truth", + "//third_party:org_jetbrains_kotlin_kotlin-test-junit", + ], +) + +kt_jvm_test( + name = "StringResourceParserTest", + srcs = ["StringResourceParserTest.kt"], + deps = [ + "//scripts/src/java/org/oppia/android/scripts/xml:string_resource_parser", + "//testing:assertion_helpers", + "//third_party:com_google_truth_truth", + "//third_party:org_jetbrains_kotlin_kotlin-test-junit", + ], +) + +kt_jvm_test( + name = "StringResourceValidationCheckTest", + srcs = ["StringResourceValidationCheckTest.kt"], + deps = [ + "//scripts/src/java/org/oppia/android/scripts/xml:string_resource_validation_check_lib", + "//testing:assertion_helpers", + "//third_party:com_google_truth_truth", + "//third_party:org_jetbrains_kotlin_kotlin-test-junit", + ], +) + kt_jvm_test( name = "XmlSyntaxCheckTest", srcs = ["XmlSyntaxCheckTest.kt"], diff --git a/scripts/src/javatests/org/oppia/android/scripts/xml/StringLanguageTranslationCheckTest.kt b/scripts/src/javatests/org/oppia/android/scripts/xml/StringLanguageTranslationCheckTest.kt new file mode 100644 index 00000000000..ebf36a61a72 --- /dev/null +++ b/scripts/src/javatests/org/oppia/android/scripts/xml/StringLanguageTranslationCheckTest.kt @@ -0,0 +1,270 @@ +package org.oppia.android.scripts.xml + +import com.google.common.truth.Truth.assertThat +import org.junit.After +import org.junit.Before +import org.junit.Rule +import org.junit.Test +import org.junit.rules.TemporaryFolder +import org.oppia.android.testing.assertThrows +import org.w3c.dom.Document +import java.io.ByteArrayOutputStream +import java.io.File +import java.io.PrintStream +import java.io.StringWriter +import java.lang.IllegalArgumentException +import javax.xml.parsers.DocumentBuilderFactory +import javax.xml.transform.TransformerFactory +import javax.xml.transform.dom.DOMSource +import javax.xml.transform.stream.StreamResult + +/** Tests for the string_language_translation_check test. */ +// FunctionName: test names are conventionally named with underscores. +@Suppress("FunctionName") +class StringLanguageTranslationCheckTest { + private companion object { + private val ARABIC_STRINGS_SHARED = mapOf("shared_string" to "مشغل رحلة الاستكشاف") + private val ARABIC_STRINGS_EXTRAS = mapOf("arabic_only_string" to "خيارات") + + private val BRAZILIAN_PORTUGUESE_STRINGS_SHARED = mapOf("shared_string" to "Meus Downloads") + private val BRAZILIAN_PORTUGUESE_STRINGS_EXTRAS = mapOf( + "brazilian_portuguese_only_string" to "Reprodutor de Exploração" + ) + + private val ENGLISH_STRINGS_SHARED = mapOf("shared_string" to "Exploration Player") + private val ENGLISH_STRINGS_EXTRAS = mapOf("english_only_string" to "Help") + + private val SWAHILI_STRINGS_SHARED = mapOf("shared_string" to "Kicheza Ugunduzi") + private val SWAHILI_STRINGS_EXTRAS = mapOf("swahili_only_string" to "Badili Wasifu") + } + + @field:[Rule JvmField] var tempFolder = TemporaryFolder() + + private val originalOut: PrintStream = System.out + private val documentBuilderFactory by lazy { DocumentBuilderFactory.newInstance() } + private val transformerFactory by lazy { TransformerFactory.newInstance() } + + private lateinit var outContent: ByteArrayOutputStream + private lateinit var appResources: File + + @Before + fun setUp() { + outContent = ByteArrayOutputStream() + appResources = tempFolder.newFolder("app", "src", "main", "res") + System.setOut(PrintStream(outContent)) + } + + @After + fun restoreStreams() { + System.setOut(originalOut) + } + + @Test + fun testScript_missingPath_throwsException() { + val exception = assertThrows(IllegalArgumentException::class) { runScript(/* With no path. */) } + + assertThat(exception) + .hasMessageThat() + .contains("Expected: bazel run //scripts:string_language_translation_check -- ") + } + + @Test + fun testScript_validPath_noStringFiles_throwsException() { + val exception = assertThrows(IllegalStateException::class) { + runScript(tempFolder.root.absolutePath) + } + + assertThat(exception).hasMessageThat().contains("Missing translation strings for language(s)") + } + + @Test + fun testScript_presentTranslations_allMatch_outputsNoneFoundMissing() { + populateArabicTranslations(ARABIC_STRINGS_SHARED) + populateBrazilianPortugueseTranslations(BRAZILIAN_PORTUGUESE_STRINGS_SHARED) + populateEnglishTranslations(ENGLISH_STRINGS_SHARED) + populateSwahiliTranslations(SWAHILI_STRINGS_SHARED) + + runScript(tempFolder.root.absolutePath) + + assertThat(outContent.asString()).contains("0 translation(s) were found missing") + } + + @Test + fun testScript_presentTranslations_missingSomeArabic_outputsMissingTranslations() { + populateArabicTranslations(ARABIC_STRINGS_EXTRAS) + populateBrazilianPortugueseTranslations(BRAZILIAN_PORTUGUESE_STRINGS_SHARED) + populateEnglishTranslations(ENGLISH_STRINGS_SHARED) + populateSwahiliTranslations(SWAHILI_STRINGS_SHARED) + + runScript(tempFolder.root.absolutePath) + + assertThat(outContent.asString().trim()).isEqualTo( + """ + 1 translation(s) were found missing. + + Missing translations: + ARABIC (1/1): + - shared_string + """.trimIndent().trim() + ) + } + + @Test + fun testScript_presentTranslations_missingSomeBrazilianPortuguese_outputsMissingTranslations() { + populateArabicTranslations(ARABIC_STRINGS_SHARED) + populateBrazilianPortugueseTranslations(BRAZILIAN_PORTUGUESE_STRINGS_EXTRAS) + populateEnglishTranslations(ENGLISH_STRINGS_SHARED) + populateSwahiliTranslations(SWAHILI_STRINGS_SHARED) + + runScript(tempFolder.root.absolutePath) + + assertThat(outContent.asString().trim()).isEqualTo( + """ + 1 translation(s) were found missing. + + Missing translations: + BRAZILIAN_PORTUGUESE (1/1): + - shared_string + """.trimIndent().trim() + ) + } + + @Test + fun testScript_presentTranslations_missingSomeSwahili_outputsMissingTranslations() { + populateArabicTranslations(ARABIC_STRINGS_SHARED) + populateBrazilianPortugueseTranslations(BRAZILIAN_PORTUGUESE_STRINGS_SHARED) + populateEnglishTranslations(ENGLISH_STRINGS_SHARED) + populateSwahiliTranslations(SWAHILI_STRINGS_EXTRAS) + + runScript(tempFolder.root.absolutePath) + + assertThat(outContent.asString().trim()).isEqualTo( + """ + 1 translation(s) were found missing. + + Missing translations: + SWAHILI (1/1): + - shared_string + """.trimIndent().trim() + ) + } + + @Test + fun testScript_presentTranslations_missingMultiple_outputsMissingTranslationsWithTotalCount() { + populateArabicTranslations(ARABIC_STRINGS_EXTRAS) + populateBrazilianPortugueseTranslations(BRAZILIAN_PORTUGUESE_STRINGS_EXTRAS) + populateEnglishTranslations(ENGLISH_STRINGS_SHARED + ENGLISH_STRINGS_EXTRAS) + populateSwahiliTranslations(SWAHILI_STRINGS_EXTRAS) + + runScript(tempFolder.root.absolutePath) + + assertThat(outContent.asString().trim()).isEqualTo( + """ + 6 translation(s) were found missing. + + Missing translations: + ARABIC (2/6): + - shared_string + - english_only_string + + BRAZILIAN_PORTUGUESE (2/6): + - shared_string + - english_only_string + + SWAHILI (2/6): + - shared_string + - english_only_string + """.trimIndent().trim() + ) + } + + @Test + fun testScript_presentTranslations_missingMultiple_someShared_outputsMissingXlationsWithCount() { + populateArabicTranslations(ARABIC_STRINGS_SHARED + ARABIC_STRINGS_EXTRAS) + populateBrazilianPortugueseTranslations( + BRAZILIAN_PORTUGUESE_STRINGS_SHARED + BRAZILIAN_PORTUGUESE_STRINGS_EXTRAS + ) + populateEnglishTranslations(ENGLISH_STRINGS_SHARED + ENGLISH_STRINGS_EXTRAS) + populateSwahiliTranslations(SWAHILI_STRINGS_SHARED + SWAHILI_STRINGS_EXTRAS) + + runScript(tempFolder.root.absolutePath) + + assertThat(outContent.asString().trim()).isEqualTo( + """ + 3 translation(s) were found missing. + + Missing translations: + ARABIC (1/3): + - english_only_string + + BRAZILIAN_PORTUGUESE (1/3): + - english_only_string + + SWAHILI (1/3): + - english_only_string + """.trimIndent().trim() + ) + } + + @Test + fun testScript_missingEnglishTranslations_outputsNoneFoundMissing() { + populateArabicTranslations(ARABIC_STRINGS_SHARED) + populateBrazilianPortugueseTranslations(BRAZILIAN_PORTUGUESE_STRINGS_SHARED) + populateEnglishTranslations(mapOf()) + populateSwahiliTranslations(SWAHILI_STRINGS_SHARED) + + runScript(tempFolder.root.absolutePath) + + // No translations should be found missing if the string is everywhere except the base file. + assertThat(outContent.asString()).contains("0 translation(s) were found missing") + } + + private fun runScript(vararg args: String) = main(*args) + + private fun populateArabicTranslations(strings: Map) { + populateTranslations(appResources, "values-ar", strings) + } + + private fun populateBrazilianPortugueseTranslations(strings: Map) { + populateTranslations(appResources, "values-pt-rBR", strings) + } + + private fun populateEnglishTranslations(strings: Map) { + populateTranslations(appResources, "values", strings) + } + + private fun populateSwahiliTranslations(strings: Map) { + populateTranslations(appResources, "values-sw", strings) + } + + private fun populateTranslations( + resourceDir: File, + valuesDirName: String, + translations: Map + ) { + val document = documentBuilderFactory.newDocumentBuilder().newDocument() + val resourcesRoot = document.createElement("resources").also { document.appendChild(it) } + translations.map { (name, value) -> + document.createElement("string").also { + it.setAttribute("name", name) + it.textContent = value + } + }.forEach(resourcesRoot::appendChild) + writeTranslationsFile(resourceDir, valuesDirName, document.toSource()) + } + + private fun writeTranslationsFile(resourceDir: File, valuesDirName: String, contents: String) { + val valuesDir = File(resourceDir, valuesDirName).also { check(it.mkdir()) } + File(valuesDir, "strings.xml").writeText(contents) + } + + private fun Document.toSource(): String { + // Reference: https://stackoverflow.com/a/5456836. + val transformer = transformerFactory.newTransformer() + return StringWriter().apply { + transformer.transform(DOMSource(this@toSource), StreamResult(this@apply)) + }.toString() + } + + private fun ByteArrayOutputStream.asString() = toString(Charsets.UTF_8.name()) +} diff --git a/scripts/src/javatests/org/oppia/android/scripts/xml/StringResourceParserTest.kt b/scripts/src/javatests/org/oppia/android/scripts/xml/StringResourceParserTest.kt new file mode 100644 index 00000000000..474b0ceba83 --- /dev/null +++ b/scripts/src/javatests/org/oppia/android/scripts/xml/StringResourceParserTest.kt @@ -0,0 +1,279 @@ +package org.oppia.android.scripts.xml + +import com.google.common.truth.Truth.assertThat +import org.junit.Before +import org.junit.Rule +import org.junit.Test +import org.junit.rules.TemporaryFolder +import org.oppia.android.scripts.xml.StringResourceParser.TranslationLanguage.ARABIC +import org.oppia.android.scripts.xml.StringResourceParser.TranslationLanguage.BRAZILIAN_PORTUGUESE +import org.oppia.android.scripts.xml.StringResourceParser.TranslationLanguage.ENGLISH +import org.oppia.android.scripts.xml.StringResourceParser.TranslationLanguage.SWAHILI +import org.oppia.android.testing.assertThrows +import org.w3c.dom.Document +import org.xml.sax.SAXParseException +import java.io.File +import java.io.StringWriter +import javax.xml.parsers.DocumentBuilderFactory +import javax.xml.transform.TransformerFactory +import javax.xml.transform.dom.DOMSource +import javax.xml.transform.stream.StreamResult + +/** Tests for [StringResourceParser]. */ +// FunctionName: test names are conventionally named with underscores. +@Suppress("FunctionName") +class StringResourceParserTest { + @field:[Rule JvmField] val tempFolder = TemporaryFolder() + + private companion object { + private val ARABIC_STRINGS = mapOf( + "shared_string" to "مشغل رحلة الاستكشاف", + "arabic_only_string" to "خيارات" + ) + + private val BRAZILIAN_PORTUGUESE_STRINGS = mapOf( + "shared_string" to "Meus Downloads", + "brazilian_portuguese_only_string" to "Reprodutor de Exploração" + ) + + private val ENGLISH_STRINGS = mapOf( + "shared_string" to "Exploration Player", + "english_only_string" to "Help" + ) + + private val SWAHILI_STRINGS = mapOf( + "shared_string" to "Kicheza Ugunduzi", + "swahili_only_string" to "Badili Wasifu" + ) + } + + private val documentBuilderFactory by lazy { DocumentBuilderFactory.newInstance() } + private val transformerFactory by lazy { TransformerFactory.newInstance() } + + private lateinit var appResources: File + private lateinit var utilityResources: File + + @Before + fun setUp() { + // Ensure there are directories for string resources. + appResources = tempFolder.newFolder("app", "src", "main", "res") + utilityResources = tempFolder.newFolder("utility", "src", "main", "res") + } + + @Test + fun testRetrieveBaseStringFile_noStrings_throwsException() { + val parser = StringResourceParser(tempFolder.root) + + val exception = assertThrows(IllegalStateException::class) { parser.retrieveBaseStringFile() } + + assertThat(exception) + .hasMessageThat() + .contains( + "Missing translation strings for language(s): ARABIC, BRAZILIAN_PORTUGUESE, ENGLISH," + + " SWAHILI" + ) + } + + @Test + fun testRetrieveBaseStringFile_noBaseEnglishStrings_throwsException() { + populateArabicTranslations() + populateBrazilianPortugueseTranslations() + populateSwahiliTranslations() + val parser = StringResourceParser(tempFolder.root) + + val exception = assertThrows(IllegalStateException::class) { parser.retrieveBaseStringFile() } + + assertThat(exception) + .hasMessageThat() + .contains("Missing translation strings for language(s): ENGLISH") + } + + @Test + fun testRetrieveBaseStringFile_noArabicStrings_throwsException() { + populateBrazilianPortugueseTranslations() + populateEnglishTranslations() + populateSwahiliTranslations() + val parser = StringResourceParser(tempFolder.root) + + val exception = assertThrows(IllegalStateException::class) { parser.retrieveBaseStringFile() } + + assertThat(exception) + .hasMessageThat() + .contains("Missing translation strings for language(s): ARABIC") + } + + @Test + fun testRetrieveBaseStringFile_noBrazilianPortugueseStrings_throwsException() { + populateArabicTranslations() + populateEnglishTranslations() + populateSwahiliTranslations() + val parser = StringResourceParser(tempFolder.root) + + val exception = assertThrows(IllegalStateException::class) { parser.retrieveBaseStringFile() } + + assertThat(exception) + .hasMessageThat() + .contains("Missing translation strings for language(s): BRAZILIAN_PORTUGUESE") + } + + @Test + fun testRetrieveBaseStringFile_noSwahiliStrings_throwsException() { + populateArabicTranslations() + populateBrazilianPortugueseTranslations() + populateEnglishTranslations() + val parser = StringResourceParser(tempFolder.root) + + val exception = assertThrows(IllegalStateException::class) { parser.retrieveBaseStringFile() } + + assertThat(exception) + .hasMessageThat() + .contains("Missing translation strings for language(s): SWAHILI") + } + + @Test + fun testRetrieveBaseStringFile_extraStringsDirectory_throwsException() { + populateAllAppTranslations() + populateTranslations(appResources, "values-fake", mapOf()) + val parser = StringResourceParser(tempFolder.root) + + val exception = assertThrows(IllegalStateException::class) { parser.retrieveBaseStringFile() } + + assertThat(exception) + .hasMessageThat() + .contains( + "Strings file 'app/src/main/res/values-fake/strings.xml' does not correspond to a known" + + " language: values-fake" + ) + } + + @Test + fun testRetrieveBaseStringFile_stringsOutsideAppDirectory_areIgnored() { + populateArabicTranslations() + populateBrazilianPortugueseTranslations() + populateSwahiliTranslations() + populateTranslations(utilityResources, "values", mapOf()) + val parser = StringResourceParser(tempFolder.root) + + val exception = assertThrows(IllegalStateException::class) { parser.retrieveBaseStringFile() } + + // An exception is still thrown since resources outside the app directory are ignored. + assertThat(exception) + .hasMessageThat() + .contains("Missing translation strings for language(s): ENGLISH") + } + + @Test + fun testRetrieveBaseStringFile_allStringsPresent_baseStringsInvalidXml_throwsException() { + populateArabicTranslations() + populateBrazilianPortugueseTranslations() + populateSwahiliTranslations() + writeTranslationsFile(appResources, "values", "") + val parser = StringResourceParser(tempFolder.root) + + assertThrows(SAXParseException::class) { parser.retrieveBaseStringFile() } + } + + @Test + fun testRetrieveBaseStringFile_allStringsPresentAndValid_returnsBaseStringFile() { + populateAllAppTranslations() + val parser = StringResourceParser(tempFolder.root) + + val stringFile = parser.retrieveBaseStringFile() + + assertThat(stringFile.language).isEqualTo(ENGLISH) + assertThat(stringFile.file.toRelativeString(tempFolder.root)) + .isEqualTo("app/src/main/res/values/strings.xml") + assertThat(stringFile.strings).containsExactlyEntriesIn(ENGLISH_STRINGS) + } + + @Test + fun testRetrieveBaseStringNames_allStringsPresentAndValid_returnsBaseStringNames() { + populateAllAppTranslations() + val parser = StringResourceParser(tempFolder.root) + + val stringNames = parser.retrieveBaseStringNames() + + assertThat(stringNames).containsExactly("shared_string", "english_only_string") + } + + @Test + fun retrieveAllNonEnglishTranslations_allStringsPresentAndValid_returnsNonEnglishStringFiles() { + populateAllAppTranslations() + val parser = StringResourceParser(tempFolder.root) + + val nonEnglishTranslations = parser.retrieveAllNonEnglishTranslations() + + assertThat(nonEnglishTranslations).hasSize(3) + assertThat(nonEnglishTranslations).containsKey(ARABIC) + assertThat(nonEnglishTranslations).containsKey(BRAZILIAN_PORTUGUESE) + assertThat(nonEnglishTranslations).containsKey(SWAHILI) + assertThat(nonEnglishTranslations).doesNotContainKey(ENGLISH) // Only non-English are included. + val arFile = nonEnglishTranslations[ARABIC] + assertThat(arFile?.language).isEqualTo(ARABIC) + assertThat(arFile?.file?.toRelativeString(tempFolder.root)) + .isEqualTo("app/src/main/res/values-ar/strings.xml") + assertThat(arFile?.strings).containsExactlyEntriesIn(ARABIC_STRINGS) + val ptBrFile = nonEnglishTranslations[BRAZILIAN_PORTUGUESE] + assertThat(ptBrFile?.language).isEqualTo(BRAZILIAN_PORTUGUESE) + assertThat(ptBrFile?.file?.toRelativeString(tempFolder.root)) + .isEqualTo("app/src/main/res/values-pt-rBR/strings.xml") + assertThat(ptBrFile?.strings).containsExactlyEntriesIn(BRAZILIAN_PORTUGUESE_STRINGS) + val swFile = nonEnglishTranslations[SWAHILI] + assertThat(swFile?.language).isEqualTo(SWAHILI) + assertThat(swFile?.file?.toRelativeString(tempFolder.root)) + .isEqualTo("app/src/main/res/values-sw/strings.xml") + assertThat(swFile?.strings).containsExactlyEntriesIn(SWAHILI_STRINGS) + } + + private fun populateAllAppTranslations() { + populateArabicTranslations() + populateBrazilianPortugueseTranslations() + populateEnglishTranslations() + populateSwahiliTranslations() + } + + private fun populateArabicTranslations() { + populateTranslations(appResources, "values-ar", ARABIC_STRINGS) + } + + private fun populateBrazilianPortugueseTranslations() { + populateTranslations(appResources, "values-pt-rBR", BRAZILIAN_PORTUGUESE_STRINGS) + } + + private fun populateEnglishTranslations() { + populateTranslations(appResources, "values", ENGLISH_STRINGS) + } + + private fun populateSwahiliTranslations() { + populateTranslations(appResources, "values-sw", SWAHILI_STRINGS) + } + + private fun populateTranslations( + resourceDir: File, + valuesDirName: String, + translations: Map + ) { + val document = documentBuilderFactory.newDocumentBuilder().newDocument() + val resourcesRoot = document.createElement("resources").also { document.appendChild(it) } + translations.map { (name, value) -> + document.createElement("string").also { + it.setAttribute("name", name) + it.textContent = value + } + }.forEach(resourcesRoot::appendChild) + writeTranslationsFile(resourceDir, valuesDirName, document.toSource()) + } + + private fun writeTranslationsFile(resourceDir: File, valuesDirName: String, contents: String) { + val valuesDir = File(resourceDir, valuesDirName).also { check(it.mkdir()) } + File(valuesDir, "strings.xml").writeText(contents) + } + + private fun Document.toSource(): String { + // Reference: https://stackoverflow.com/a/5456836. + val transformer = transformerFactory.newTransformer() + return StringWriter().apply { + transformer.transform(DOMSource(this@toSource), StreamResult(this@apply)) + }.toString() + } +} diff --git a/scripts/src/javatests/org/oppia/android/scripts/xml/StringResourceValidationCheckTest.kt b/scripts/src/javatests/org/oppia/android/scripts/xml/StringResourceValidationCheckTest.kt new file mode 100644 index 00000000000..c7fc526f687 --- /dev/null +++ b/scripts/src/javatests/org/oppia/android/scripts/xml/StringResourceValidationCheckTest.kt @@ -0,0 +1,259 @@ +package org.oppia.android.scripts.xml + +import com.google.common.truth.Truth.assertThat +import org.junit.After +import org.junit.Before +import org.junit.Rule +import org.junit.Test +import org.junit.rules.TemporaryFolder +import org.oppia.android.testing.assertThrows +import org.w3c.dom.Document +import java.io.ByteArrayOutputStream +import java.io.File +import java.io.PrintStream +import java.io.StringWriter +import javax.xml.parsers.DocumentBuilderFactory +import javax.xml.transform.TransformerFactory +import javax.xml.transform.dom.DOMSource +import javax.xml.transform.stream.StreamResult + +/** Tests for the string_resource_validation_check test. */ +// FunctionName: test names are conventionally named with underscores. +@Suppress("FunctionName") +class StringResourceValidationCheckTest { + private companion object { + private const val AR_STRING_NO_NEWLINES = "مساعدة" + private const val AR_STRING_ONE_NEWLINE = "مساعدة\\n" + private const val AR_STRING_TWO_NEWLINES = "\\nمساعدة\\n" + + private const val PT_BR_STRING_NO_NEWLINES = "Ajuda" + private const val PT_BR_STRING_ONE_NEWLINE = "\\nAjuda" + private const val PT_BR_STRING_TWO_NEWLINES = "\\nAjuda\\n" + + private const val EN_STRING_ONE_NEWLINE = "\\nHelp" + + private const val SW_STRING_NO_NEWLINES = "Msaada" + private const val SW_STRING_ONE_NEWLINE = "\\nMsaada" + private const val SW_STRING_TWO_NEWLINES = "\\nMsaada\\n" + } + + @field:[Rule JvmField] var tempFolder = TemporaryFolder() + + private val originalOut: PrintStream = System.out + private val documentBuilderFactory by lazy { DocumentBuilderFactory.newInstance() } + private val transformerFactory by lazy { TransformerFactory.newInstance() } + + private lateinit var outContent: ByteArrayOutputStream + private lateinit var appResources: File + + @Before + fun setUp() { + outContent = ByteArrayOutputStream() + appResources = tempFolder.newFolder("app", "src", "main", "res") + System.setOut(PrintStream(outContent)) + } + + @After + fun restoreStreams() { + System.setOut(originalOut) + } + + @Test + fun testScript_missingPath_throwsException() { + val exception = assertThrows(IllegalArgumentException::class) { runScript(/* With no path. */) } + + assertThat(exception) + .hasMessageThat() + .contains("Expected: bazel run //scripts:string_resource_validation_check -- ") + } + + @Test + fun testScript_validPath_noStringFiles_fails() { + val exception = assertThrows(IllegalStateException::class) { + runScript(tempFolder.root.absolutePath) + } + + assertThat(exception).hasMessageThat().contains("Missing translation strings for language(s)") + } + + @Test + fun testScript_allMatch_succeeds() { + populateArabicTranslations(mapOf("str1" to AR_STRING_ONE_NEWLINE)) + populateBrazilianPortugueseTranslations(mapOf("str1" to PT_BR_STRING_ONE_NEWLINE)) + populateEnglishTranslations(mapOf("str1" to EN_STRING_ONE_NEWLINE)) + populateSwahiliTranslations(mapOf("str1" to SW_STRING_ONE_NEWLINE)) + + runScript(tempFolder.root.absolutePath) + + assertThat(outContent.asString()).contains("STRING RESOURCE VALIDATION CHECKS PASSED") + } + + @Test + fun testScript_inconsistentLines_arabic_failsWithFindings() { + populateArabicTranslations( + mapOf("str1" to AR_STRING_NO_NEWLINES, "str2" to AR_STRING_TWO_NEWLINES) + ) + populateBrazilianPortugueseTranslations(mapOf("str1" to PT_BR_STRING_ONE_NEWLINE)) + populateEnglishTranslations( + mapOf("str1" to EN_STRING_ONE_NEWLINE, "str2" to EN_STRING_ONE_NEWLINE) + ) + populateSwahiliTranslations(mapOf("str1" to SW_STRING_ONE_NEWLINE)) + + val exception = assertThrows(Exception::class) { runScript(tempFolder.root.absolutePath) } + + // This output check also inadvertently verifies that the script doesn't care about missing + // strings in translated string files. + assertThat(exception).hasMessageThat().contains("STRING RESOURCE VALIDATION CHECKS FAILED") + assertThat(outContent.asString().trim()).isEqualTo( + """ + 1 language(s) were found with string consistency errors. + + 2 consistency error(s) were found for ARABIC strings (file: app/src/main/res/values-ar/strings.xml): + - string str1: original translation uses 2 line(s) but translation uses 1 line(s). Please remove any extra lines or add any that are missing. + - string str2: original translation uses 2 line(s) but translation uses 3 line(s). Please remove any extra lines or add any that are missing. + """.trimIndent().trim() + ) + } + + @Test + fun testScript_inconsistentLines_brazilianPortuguese_failsWithFindings() { + populateArabicTranslations(mapOf("str1" to AR_STRING_ONE_NEWLINE)) + populateBrazilianPortugueseTranslations( + mapOf("str1" to PT_BR_STRING_NO_NEWLINES, "str2" to PT_BR_STRING_TWO_NEWLINES) + ) + populateEnglishTranslations( + mapOf("str1" to EN_STRING_ONE_NEWLINE, "str2" to EN_STRING_ONE_NEWLINE) + ) + populateSwahiliTranslations(mapOf("str1" to SW_STRING_ONE_NEWLINE)) + + val exception = assertThrows(Exception::class) { runScript(tempFolder.root.absolutePath) } + + // This output check also inadvertently verifies that the script doesn't care about missing + // strings in translated string files. + assertThat(exception).hasMessageThat().contains("STRING RESOURCE VALIDATION CHECKS FAILED") + assertThat(outContent.asString().trim()).isEqualTo( + """ + 1 language(s) were found with string consistency errors. + + 2 consistency error(s) were found for BRAZILIAN_PORTUGUESE strings (file: app/src/main/res/values-pt-rBR/strings.xml): + - string str1: original translation uses 2 line(s) but translation uses 1 line(s). Please remove any extra lines or add any that are missing. + - string str2: original translation uses 2 line(s) but translation uses 3 line(s). Please remove any extra lines or add any that are missing. + """.trimIndent().trim() + ) + } + + @Test + fun testScript_inconsistentLines_swahili_failsWithFindings() { + populateArabicTranslations(mapOf("str1" to AR_STRING_ONE_NEWLINE)) + populateBrazilianPortugueseTranslations(mapOf("str1" to PT_BR_STRING_ONE_NEWLINE)) + populateEnglishTranslations( + mapOf("str1" to EN_STRING_ONE_NEWLINE, "str2" to EN_STRING_ONE_NEWLINE) + ) + populateSwahiliTranslations( + mapOf("str1" to SW_STRING_NO_NEWLINES, "str2" to SW_STRING_TWO_NEWLINES) + ) + + val exception = assertThrows(Exception::class) { runScript(tempFolder.root.absolutePath) } + + // This output check also inadvertently verifies that the script doesn't care about missing + // strings in translated string files. + assertThat(exception).hasMessageThat().contains("STRING RESOURCE VALIDATION CHECKS FAILED") + assertThat(outContent.asString().trim()).isEqualTo( + """ + 1 language(s) were found with string consistency errors. + + 2 consistency error(s) were found for SWAHILI strings (file: app/src/main/res/values-sw/strings.xml): + - string str1: original translation uses 2 line(s) but translation uses 1 line(s). Please remove any extra lines or add any that are missing. + - string str2: original translation uses 2 line(s) but translation uses 3 line(s). Please remove any extra lines or add any that are missing. + """.trimIndent().trim() + ) + } + + @Test + fun testScript_inconsistentLines_allLanguages_failsWithFindings() { + populateArabicTranslations( + mapOf("str1" to AR_STRING_NO_NEWLINES, "str2" to AR_STRING_TWO_NEWLINES) + ) + populateBrazilianPortugueseTranslations( + mapOf("str1" to PT_BR_STRING_NO_NEWLINES, "str2" to PT_BR_STRING_TWO_NEWLINES) + ) + populateEnglishTranslations( + mapOf("str1" to EN_STRING_ONE_NEWLINE, "str2" to EN_STRING_ONE_NEWLINE) + ) + populateSwahiliTranslations( + mapOf("str1" to SW_STRING_NO_NEWLINES, "str2" to SW_STRING_TWO_NEWLINES) + ) + + val exception = assertThrows(Exception::class) { runScript(tempFolder.root.absolutePath) } + + // This output check also inadvertently verifies that the script doesn't care about missing + // strings in translated string files. + assertThat(exception).hasMessageThat().contains("STRING RESOURCE VALIDATION CHECKS FAILED") + assertThat(outContent.asString().trim()).isEqualTo( + """ + 3 language(s) were found with string consistency errors. + + 2 consistency error(s) were found for ARABIC strings (file: app/src/main/res/values-ar/strings.xml): + - string str1: original translation uses 2 line(s) but translation uses 1 line(s). Please remove any extra lines or add any that are missing. + - string str2: original translation uses 2 line(s) but translation uses 3 line(s). Please remove any extra lines or add any that are missing. + + 2 consistency error(s) were found for BRAZILIAN_PORTUGUESE strings (file: app/src/main/res/values-pt-rBR/strings.xml): + - string str1: original translation uses 2 line(s) but translation uses 1 line(s). Please remove any extra lines or add any that are missing. + - string str2: original translation uses 2 line(s) but translation uses 3 line(s). Please remove any extra lines or add any that are missing. + + 2 consistency error(s) were found for SWAHILI strings (file: app/src/main/res/values-sw/strings.xml): + - string str1: original translation uses 2 line(s) but translation uses 1 line(s). Please remove any extra lines or add any that are missing. + - string str2: original translation uses 2 line(s) but translation uses 3 line(s). Please remove any extra lines or add any that are missing. + """.trimIndent().trim() + ) + } + + private fun runScript(vararg args: String) = main(*args) + + private fun populateArabicTranslations(strings: Map) { + populateTranslations(appResources, "values-ar", strings) + } + + private fun populateBrazilianPortugueseTranslations(strings: Map) { + populateTranslations(appResources, "values-pt-rBR", strings) + } + + private fun populateEnglishTranslations(strings: Map) { + populateTranslations(appResources, "values", strings) + } + + private fun populateSwahiliTranslations(strings: Map) { + populateTranslations(appResources, "values-sw", strings) + } + + private fun populateTranslations( + resourceDir: File, + valuesDirName: String, + translations: Map + ) { + val document = documentBuilderFactory.newDocumentBuilder().newDocument() + val resourcesRoot = document.createElement("resources").also { document.appendChild(it) } + translations.map { (name, value) -> + document.createElement("string").also { + it.setAttribute("name", name) + it.textContent = value + } + }.forEach(resourcesRoot::appendChild) + writeTranslationsFile(resourceDir, valuesDirName, document.toSource()) + } + + private fun writeTranslationsFile(resourceDir: File, valuesDirName: String, contents: String) { + val valuesDir = File(resourceDir, valuesDirName).also { check(it.mkdir()) } + File(valuesDir, "strings.xml").writeText(contents) + } + + private fun Document.toSource(): String { + // Reference: https://stackoverflow.com/a/5456836. + val transformer = transformerFactory.newTransformer() + return StringWriter().apply { + transformer.transform(DOMSource(this@toSource), StreamResult(this@apply)) + }.toString() + } + + private fun ByteArrayOutputStream.asString() = toString(Charsets.UTF_8.name()) +} diff --git a/third_party/maven_install.json b/third_party/maven_install.json index a25cf4753fa..d743a3bb8ee 100644 --- a/third_party/maven_install.json +++ b/third_party/maven_install.json @@ -1,8 +1,8 @@ { "dependency_tree": { "__AUTOGENERATED_FILE_DO_NOT_MODIFY_THIS_FILE_MANUALLY": "THERE_IS_NO_DATA_ONLY_ZUUL", - "__INPUT_ARTIFACTS_HASH": 134596076, - "__RESOLVED_ARTIFACTS_HASH": -1383535075, + "__INPUT_ARTIFACTS_HASH": -2010577555, + "__RESOLVED_ARTIFACTS_HASH": 6219109, "conflict_resolution": { "androidx.constraintlayout:constraintlayout:1.1.3": "androidx.constraintlayout:constraintlayout:2.0.1", "androidx.core:core:1.0.1": "androidx.core:core:1.3.1", @@ -12,7 +12,7 @@ "com.google.truth:truth:0.43": "com.google.truth:truth:1.1.3", "junit:junit:4.12": "junit:junit:4.13.2", "org.jetbrains.kotlin:kotlin-reflect:1.3.41": "org.jetbrains.kotlin:kotlin-reflect:1.5.0", - "org.jetbrains.kotlin:kotlin-stdlib-jdk7:1.3.72": "org.jetbrains.kotlin:kotlin-stdlib-jdk7:1.4.10", + "org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.3.72": "org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.4.10", "org.mockito:mockito-core:2.19.0": "org.mockito:mockito-core:3.9.0" }, "dependencies": [ @@ -7157,8 +7157,10 @@ { "coord": "com.squareup:kotlinpoet:1.6.0", "dependencies": [ + "org.jetbrains.kotlin:kotlin-stdlib-jdk7:1.4.10", "org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.4.10", - "org.jetbrains.kotlin:kotlin-reflect:1.5.0" + "org.jetbrains.kotlin:kotlin-reflect:1.5.0", + "org.jetbrains.kotlin:kotlin-stdlib:1.5.0" ], "directDependencies": [ "org.jetbrains.kotlin:kotlin-reflect:1.5.0", @@ -7179,6 +7181,8 @@ "coord": "com.squareup:kotlinpoet:jar:sources:1.6.0", "dependencies": [ "org.jetbrains.kotlin:kotlin-stdlib-jdk8:jar:sources:1.4.10", + "org.jetbrains.kotlin:kotlin-stdlib:jar:sources:1.5.0", + "org.jetbrains.kotlin:kotlin-stdlib-jdk7:jar:sources:1.4.10", "org.jetbrains.kotlin:kotlin-reflect:jar:sources:1.5.0" ], "directDependencies": [ diff --git a/third_party/versions.bzl b/third_party/versions.bzl index 289c58ad5a8..245d14f20b9 100644 --- a/third_party/versions.bzl +++ b/third_party/versions.bzl @@ -69,7 +69,7 @@ MAVEN_PRODUCTION_DEPENDENCY_VERSIONS = { "javax.annotation:javax.annotation-api:jar": "1.3.2", "javax.inject:javax.inject": "1", "nl.dionsegijn:konfetti": "1.2.5", - "org.jetbrains.kotlin:kotlin-stdlib-jdk7:jar": "1.3.72", + "org.jetbrains.kotlin:kotlin-stdlib-jdk8:jar": "1.3.72", "org.jetbrains.kotlinx:kotlinx-coroutines-android": "1.4.1", "org.jetbrains.kotlinx:kotlinx-coroutines-core": "1.4.1", "org.jetbrains:annotations:jar": "13.0",