Add a deploy confirmation step on web

[roryabraham]

Problem

Coming from here, we went a few days without noticing production deploys were broken.

Solution

We've addressed the root cause and I created rhysd/actionlint#440 to try and prevent any similar type of error in the future.

However, let's also add an automated check that just confirms that the deploy completed as we expect, at least on web. In order to do this, we can just download the version.json file hosted on staging and/or production and verify that it matches what we expect after the deploy is done.

[roryabraham]

Thinking about this some more, even if we had this in place it would not have caught the error we had. Because the URL we were using was wrong for the upload, there's no reason to think it would've been right on the verification step. So I'm going to just close this actually.

[roryabraham]

if we wanted to do this, I think we'd want it to be a manual check. Not sure how we'd fit that in as a deployer responsibility though

[roryabraham]

we decided to add this after all

[melvin-bot]

The solution for this issue has been 🚀 deployed to production 🚀 in version 9.0.13-4 and is now subject to a 7-day regression period 📆. Here is the list of pull requests that resolve this issue:

• [No QA] Add verification for web deploy #46212

If no regressions arise, payment will be issued on 2024-08-05. 🎊

[melvin-bot]

Skipping the payment summary for this issue since all the assignees are employees or vendors. If this is incorrect, please manually add the payment summary SO.