Add FXIOS-9487 - Remote settings config auto fetch script for password rules #21023

nbhasin2 · 2024-07-15T20:12:27Z

📜 Tickets

💡 Description

This script is to fetch remote settings on Tuesdays and Thursdays automatically and create necessary PRs if there are any changes.

Added github actions for auto fetch
Added python script to fetch remote settings
Added initial remote settings record for password rules

Ref: https://github.com/issammani/firefox-ios/blob/main/.github/workflows/firefox-ios-update_credential_provider_script.yml

📝 Checklist

You have to check all boxes before merging

Filled in the above information (tickets numbers and description of your work)
Updated the PR name to follow our PR naming guidelines
Wrote unit tests and/or ensured the tests suite is passing
When working on UI, I checked and implemented accessibility (minimum Dynamic Text and VoiceOver)
If needed, I updated documentation / comments for complex code and public methods
If needed, added a backport comment (example @Mergifyio backport release/v120)

nbhasin2 · 2024-07-15T20:13:26Z

@issammani I am tagging you as you did some work around fetching data using python script from HG for autofill
And also tagging @razvanlitianu as he is involved in the autofill area in general

issammani

This looks great. Thanks @nbhasin2.

Tried locally and it works as expected when the json file changed / not changed. I left some minor nits.

firefox-ios/Client/Assets/RemoteSettingsData/Update_Remote_Settings.py

issammani · 2024-07-19T13:15:35Z

firefox-ios/Client/Assets/RemoteSettingsData/Update_Remote_Settings.py

+
+    if not changes_detected:
+        print("No changes detected in any rules.")
+


It won't break the code since we check if the directory exists before creating it, but maybe we should remove the directory after we are done ? Something like shutil.rmtree(GITHUB_ACTIONS_TMP_PATH)

true, I left it here as I wasn't sure what to do here 🤷🏼
One question that I had was since its run on github actions, does it spin a new VM everytime an action is run or all share the same VM

Good question. So looking at the docs, github actions doesn't cache that directory between runs. And since the files in there won't be committed and not in the final PR it doesn't really matter if we remove it or not 😄

We can leave it be then for any logs during that run!

nbhasin2 · 2024-07-19T20:00:28Z

firefox-ios/Client/Assets/RemoteSettingsData/RemoteSettingsFetchConfig.json

+      "name": "Password Rules",
+      "url": "https://firefox.settings.services.mozilla.com/v1/buckets/main/collections/password-rules/records",
+      "file": "./firefox-ios/Client/Assets/RemoteSettingsData/RemotePasswordRules.json"
+    }


Making it ready for review, once this gets merged I will start the process of adding more files for upcoming remote settings work. So think of this as the initial step

nbhasin2 · 2024-07-22T19:17:45Z

Once @isabelrios gives the r+ I will merge this in 🤞🏼

isabelrios · 2024-08-01T15:15:59Z

.github/workflows/firefox-ios-update_remote_settings_data_script.yml

+      - name: Commit and push changes
+        run: |
+          git diff
+          git diff --quiet || (git add . && git commit -m "Update remote settings with latest rules" && git push)


If I got it right you are creating a temp file when comparing the changes.. git git add . it may be added... I always try to be specific about the files that I would expect to be changed and commited in the PR to save time not having to fix the automatic PR

Oh you mean, make it explicit?

Yes, if that makes sense.. something like: https://github.com/nbhasin2/firefox-ios-1/blob/0ac1fe4aa1ac3663fc1a98ee09f8853066e5ff99/.github/workflows/check-rust-component-dependency.yml#L72

isabelrios · 2024-08-01T15:24:20Z

.github/workflows/firefox-ios-update_remote_settings_data_script.yml

+      - name: Send Slack notification if GitHub Action fails
+        if: '!cancelled()'
+        id: slack
+        uses: slackapi/slack


I think this needs to be like: slackapi/[email protected], I see an error in your for for this github action

Also, not sure if just with that there will be a notification, usually there are some fields to be filled in in this step.. we can try to trigger the action from your fork and see

Ah since I was basing it off
https://github.com/mozilla-mobile/firefox-ios/blob/main/.github/workflows/firefox-ios-update_credential_provider_script.yml

let me add https://github.com/mozilla-mobile/firefox-ios/blob/main/test-fixtures/ci/slack-notification-payload-autofill.json

and tag you again for another review :)

Good catch @isabelrios

firefox-ios/Client/Assets/RemoteSettingsData/Update_Remote_Settings.py

github-actions · 2024-08-17T00:24:14Z

This PR has been automatically marked as stale. Please leave any comment to keep this PR opened. It will be closed automatically if no further update occurs in the next 7 days. Thank you for your contributions!

…d rules

nbhasin2 · 2024-09-03T22:32:03Z

.github/workflows/firefox-ios-update_remote_settings_data_script.yml

+        env:
+          JOB_STATUS: ${{ job.status == 'success' && ':white_check_mark:' || job.status == 'failure' && ':x:' }}
+          JOB_STATUS_COLOR: ${{ job.status == 'success' && '#36a64f' || job.status == 'failure' && '#FF0000' }}
+          SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
+          SLACK_WEBHOOK_TYPE: INCOMING_WEBHOOK
+        with:
+          payload-file-path: "./test-fixtures/ci/slack-notification-payload-remote-settings-fetch.json"


@isabelrios added payload, could you review please 🙏🏼

The slack notification is correctly received now:
Failing:

Working:

This is received in the Mobile Test eng alerts sandbox, where we test notifications, we monitor that, if you want to move that to firefox-ios let me know so that we change the slack token

Thanks Isabel, we can leave it there for now. Maybe once I add more items we can look into sending it to its own area.

nbhasin2 · 2024-09-03T22:32:36Z

.github/workflows/firefox-ios-update_remote_settings_data_script.yml

+      - name: Commit and push changes
+        run: |
+          git diff
+          git diff --quiet || (git add firefox-ios/Client/Assets/RemoteSettingsData/RemotePasswordRules.json && git commit -m "Update remote settings with latest rules" && git push)


@isabelrios also added specific json file location, tagging you to double check

You would need to remove the latest part: && git commit -m "Update remote settings with latest rules" && git push
The github action fails to do the push directly, I don't think we want that push to happen without review..

If you remove that part, there is the next step to create the PR with the changes to review and then merge

@isabelrios let me DM you as I am wondering why do I even need the commit part considering we are doing the create pull request

mobiletest-ci-bot · 2024-09-03T22:42:06Z

	Messages
📖	Edited 1 files
📖	Created 5 files

Generated by 🚫 Danger Swift against 04ed227

isabelrios · 2024-09-04T12:48:55Z

Once @isabelrios gives the r+ I will merge this in 🤞🏼

Sorry, there is one more change to make or the github action will consistently fail. Please see my comment to remove part of the git diff statement. Thanks!

nbhasin2 requested review from issammani and razvanlitianu July 15, 2024 20:12

issammani reviewed Jul 19, 2024

View reviewed changes

nbhasin2 marked this pull request as ready for review July 19, 2024 19:59

nbhasin2 requested review from a team as code owners July 19, 2024 19:59

nbhasin2 requested a review from isabelrios July 19, 2024 19:59

nbhasin2 commented Jul 19, 2024

View reviewed changes

issammani self-requested a review July 22, 2024 04:35

issammani approved these changes Jul 22, 2024

View reviewed changes

nbhasin2 requested review from ih-codes, dataports and DanielDervishi July 23, 2024 17:43

isabelrios reviewed Aug 1, 2024

View reviewed changes

ih-codes reviewed Aug 1, 2024

View reviewed changes

firefox-ios/Client/Assets/RemoteSettingsData/Update_Remote_Settings.py Show resolved Hide resolved

ih-codes approved these changes Aug 1, 2024

View reviewed changes

razvanlitianu approved these changes Aug 2, 2024

View reviewed changes

github-actions bot added the stale Stalebot use this label to stale issues and PRs label Aug 17, 2024

github-actions bot closed this Aug 24, 2024

nbhasin2 reopened this Sep 3, 2024

nbhasin2 added 4 commits September 3, 2024 18:03

Add FXIOS-9487 - Remote settings config auto fetch script for passwor…

10e185d

…d rules

fixed file location

dd29534

minor changes to the makedirs portion

3fdfb6d

minor file name change

1d7cdca

nbhasin2 force-pushed the nb/remote_setting_config_password_rule branch from 0ac1fe4 to 1d7cdca Compare September 3, 2024 22:04

Added missing slack payload file and updated script

7988674

nbhasin2 removed the stale Stalebot use this label to stale issues and PRs label Sep 3, 2024

nbhasin2 commented Sep 3, 2024

View reviewed changes

Fixes

04ed227

isabelrios approved these changes Sep 4, 2024

View reviewed changes

nbhasin2 merged commit 858d432 into mozilla-mobile:main Sep 4, 2024
13 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add FXIOS-9487 - Remote settings config auto fetch script for password rules #21023

Add FXIOS-9487 - Remote settings config auto fetch script for password rules #21023

nbhasin2 commented Jul 15, 2024

nbhasin2 commented Jul 15, 2024

issammani left a comment

issammani Jul 19, 2024

nbhasin2 Jul 19, 2024

issammani Jul 22, 2024

nbhasin2 Jul 22, 2024

nbhasin2 Jul 19, 2024

nbhasin2 commented Jul 22, 2024

isabelrios Aug 1, 2024

nbhasin2 Aug 1, 2024

isabelrios Aug 1, 2024

isabelrios Aug 1, 2024

isabelrios Aug 1, 2024

nbhasin2 Aug 1, 2024

github-actions bot commented Aug 17, 2024

nbhasin2 Sep 3, 2024

isabelrios Sep 4, 2024

nbhasin2 Sep 4, 2024

nbhasin2 Sep 3, 2024

isabelrios Sep 4, 2024 •

edited

Loading

nbhasin2 Sep 4, 2024

mobiletest-ci-bot commented Sep 3, 2024 •

edited

Loading

isabelrios commented Sep 4, 2024 •

edited

Loading


		if not changes_detected:
		print("No changes detected in any rules.")

Add FXIOS-9487 - Remote settings config auto fetch script for password rules #21023

Add FXIOS-9487 - Remote settings config auto fetch script for password rules #21023

Conversation

nbhasin2 commented Jul 15, 2024

📜 Tickets

💡 Description

📝 Checklist

nbhasin2 commented Jul 15, 2024

issammani left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nbhasin2 commented Jul 22, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Aug 17, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

isabelrios Sep 4, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mobiletest-ci-bot commented Sep 3, 2024 • edited Loading

isabelrios commented Sep 4, 2024 • edited Loading

isabelrios Sep 4, 2024 •

edited

Loading

mobiletest-ci-bot commented Sep 3, 2024 •

edited

Loading

isabelrios commented Sep 4, 2024 •

edited

Loading