Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

4.14.0 Release checklist #3981

Closed
53 of 65 tasks
compulim opened this issue Jul 8, 2021 · 6 comments
Closed
53 of 65 tasks

4.14.0 Release checklist #3981

compulim opened this issue Jul 8, 2021 · 6 comments
Assignees
@compulim
Labels
release Release checklist

Comments

@compulim
Copy link
Contributor

compulim commented Jul 8, 2021 •
edited
Loading

Checklist

Build

  1. Bump MockBot to Bot Framework SDK release 4.14.0
  2. Bump botframework-directlinejs to 0.14.1
    • No newer version
  3. Bump to 4.14.0
    • Update CHANGELOG.md to mark specific changes in 4.14.0
    • Run npm version --no-git-tag-version 4.14.0
    • Merged into master, the PR number is Bump to 4.14.0 #3984
    • Commit is af70aaa
    • Do not merge any other unrelated changes after this PR. Any other PR merged, will need to be re-tested
  4. Run daily pipeline manually, set "generate release version number" to true
    • (This will not push to NPM or CDN)
    • Pipeline name is BotFramework-WebChat-daily
    • The build number is 257964 and commit is af70aaa
  5. Wait for WebChat-release-testing pipeline to complete
    • Pipeline name is Push-Release-Testing-to-GitHub-Pages
    • The release ID is 2751
  6. Add manual tests to WebChat-release-testing as needed

Test

The test should run against the build artifacts from Azure Pipelines.

  1. Manual testing on major browsers using webchat-release-testing
    • Before starting testing, update all the browser version to latest
    • Chrome 91.0.4472.124
    • Edge 91.0.864.64
    • Firefox 89.0.2
    • IE11 (IE Mode under Windows 11 22000.51)
    • macOS Safari 14.1.1 (16611.2.7.1.4)
    • iOS Safari 14.6
    • Android Chrome 91.0.4472.120
  2. Test specific fixes related to 4.14.0 and previous releases
    • Using a new engine to strip Markdown (PR #3917)

Release

  1. Verify on WebChat-release-testing
  2. Make sure you are on master or qfe branch, run git status to check
  3. git pull
  4. Verify /package.json, /package-lock.json, and CHANGELOG.md has a version of 4.14.0
  5. git log
    • Verify the latest commit is af70aaa
  6. git tag v4.14.0
  7. git push -u upstream v4.14.0
    • You do not need to kick off a build again, use the previous build
  8. Create a new GitHub release, copy entries from CHANGELOG.md
    • Subresource Integrity can be generated by
      • From CDN: curl -H 'Accept-Encoding: gzip' https://cdn.botframework.com/botframework-webchat/4.14.0/webchat.js | gunzip - | openssl dgst -sha384 -binary | openssl base64 -A
      • From local: cat webchat.js | openssl dgst -sha384 -binary | openssl base64 -A
    • Attach assets including 3 JS files, stats.json and 5 tarballs
      • You can copy the artifacts from webchat-release-testing/drops
      • Tarballs download from npmjs 
        curl -L -o botframework-directlinespeech-sdk-4.14.0.tgz https://registry.npmjs.org/botframework-directlinespeech-sdk/-/botframework-directlinespeech-sdk-4.14.0.tgz
curl -L -o botframework-webchat-4.14.0.tgz https://registry.npmjs.org/botframework-webchat/-/botframework-webchat-4.14.0.tgz
curl -L -o botframework-webchat-core-4.14.0.tgz https://registry.npmjs.org/botframework-webchat-core/-/botframework-webchat-core-4.14.0.tgz
curl -L -o botframework-webchat-api-4.14.0.tgz https://registry.npmjs.org/botframework-webchat-api/-/botframework-webchat-api-4.14.0.tgz
curl -L -o botframework-webchat-component-4.14.0.tgz https://registry.npmjs.org/botframework-webchat-component/-/botframework-webchat-component-4.14.0.tgz
  9. Kick off release to NPM
    • Release name is [[PROD]]Push-WebChat-to-npmjs
    • The build number is 257964 release number is 33 and commit is af70aaa
    • Verify package content then click Resume
    • Retain the build indefinitely
  10. Kick off release to CDN (CDN CUTOFF TIME IS 2PM PST)
    1. Prepare the email for approval
    • Release name is [[PROD]]Push-WebChat-to-Prod-CDN-with-approval
    • The build number is 257964, release number is 39 and commit is af70aaa
    • Script build number is 131156 (this is fixed)
    1. Send reminder email to approvers
    • Retain the build indefinitely

Post-release verification - complete within 30mins / 1hr of the release

Notification to interested parties

Post-release checklist

These are chores that we should do before starting the cycle to reduce ripple effects if we do it in mid-cycle.

Tips:

  • Clean your repo before start
  • Remove node_modules from all folder
    • git clean -fdx
  • Never delete package-lock.json
  • If you mess it up, tableflip and redo
  • In component/package.json
    • Remove reference to botframework-webchat-core by hand-modifying package.json
    • Then, npm install (symlinks will be broken afterward)
    • Then, add those references back by hand-modifying package.json
    • This also applies for other packages with similar dependencies/symlinks
    • To build afterward, do tableflip to rebuild those symlinks

Applies to all releases

This list should be copied to versions in the future.

The PR is https://github.com/microsoft/BotFramework-WebChat/pull/XXX

Applies to major/minor releases

Bump all dependencies to latest version

On each package.json under root and /packages/:

  • All NPM packages (@babel/*, jest, lerna, typescript, webpack, etc) in PR #XXX
    • Don't bump the following packages
      • react, react-dom, redux, redux-saga, @types/react
      • sanitize-html
        • When bumping sanitize-html, make sure bundle/postinstall scripts continue to works as we only transpile a single index.js file and assume the sanitize-html package will keep it that way
      • serve
        • Because newer version could be buggy on Windows
      • rxjs@5 on playground
      • webpack@4
        • Starting from version 5, they dropped Node.js polyfills for browsers
        • Keep the following dependencies because their latest version requires webpack@5
      • microsoft-cognitiveservices-speech-sdk
        • We will bump it in a separate PR
      • adaptivecards
        • We will bump it in a separate PR
      • expect@25 under test/harness
        • The newer version requires Node.js (graceful-fs)
  • Run npm audit fix to make sure everything is fixed
  • Test under IE11 to make sure all dependencies are working
  • List steps to verify bumping microsoft-cognitiveservices-speech-sdk

Obstacles to bump npm

As of writing, [email protected] (bundled by node@16) has issue on running postinstall script.

Also, [email protected] is peer-depends on @angular/common, which npm@7 will automatically install missing peer dependencies. Track the issue here.

We need at least both issues to be fixed before we can jump to npm@7.

Bump Docker image

The Docker image can be found at root docker-compose.yml and Dockerfile*.

  • Docker container for headless Chrome (#XXX)
    • They recently moved from 3.14.159-xxx tag scheme to a more sensible 87.0 tag scheme
    • Tags can be found at https://hub.docker.com/r/selenium/node-chrome/tags
    • Preferably in separate PR because screenshots change can be large occasionally
    • Run tests locally, as the screenshots can be slightly different
@compulim compulim added the release Release checklist label Jul 8, 2021
@compulim compulim self-assigned this Jul 8, 2021
@christopherjbaker
Copy link

I would like to request that you update sanitize-html whether or not in works with IE11. I'm sure you can figure out a polyfil that makes those things work, but even without that. Versions prior to 2.0, as well as some of the 2+ ones, have major security issues, and holding those back for a browser that nobody uses (see image) seems like a very, very bad idea.
image

@compulim
Copy link
Contributor Author

compulim commented Jul 15, 2021 •
edited
Loading

Will see what I can do with sanitize-html. I am still working on bumping dependencies.

Security issues and platform support are two different things. We will do both.

In my experience, issues due to IE11 are much easier/shorter to resolve than one of the green bars in your chart.

Put it this way... compare the time I spend last year fixing issues on IE11 vs. another browser, IE11 didn't take me as much time.

@christopherjbaker
Copy link

Ok, thanks! I'll check back in later.

I did do some quick looking over there at what isn't IE11 compatible. They direct people to this section of their readme. They changed the paradigm of their project, and no longer provide a browser-ready build, instead telling dependants to do the build themselves if needed. I imagine most projects using you on the FE will do so as part of a build, and it looks like your project also has a build (I didn't do the botframework integration, so I don't know much about this project), so as long as those builds add IE11 pollyfils when they need compatibility, there shouldn't be anything to do to upgrade (from a quick look, it doesn't look like their API changed between 1 and 2, just that paradigm). Hopefully this helps.

@compulim
Copy link
Contributor Author

Thanks for the tips!

Yap, polyfill + transpile sanitize-html should work for IE11. That's what we are doing from time to time too. And that's why I say fixing IE11 issues are, most of the time, easier than fixing another browser.

But sometimes, transpiling could be tricky. Let me share all my pain points and experiences to you.

Web Chat have 2 deliverables:

  • Bundle (*.js), it's transpiled through Webpack, we can configure it easily
  • NPM packages, it will be up to web devs on how to transpile it
    • If they are using create-react-app (or anything that transpile /node modules/), they should be fine
    • If they build their own pipeline, it may not easy for them to configure the pipeline
      • The team which want to integrate our component, may not be the same team which own the build pipeline. Thus, they may not be authorized to configure it

Sometimes, architecting web components could be more complex than web apps. We need to think about the customer's team structure, how they are going to write/build/architect their apps, how they are integrating our component.

For transpiling, one trick is to use postinstall script to transpile packages on-the-fly. Then a few issues along this route:

  • Recently npm@7 has a bug on postinstall and it still repro on 7.19.1 as of yesterday
    • NPM team just released 7.20.0 hours ago, I believe it is not fixed yet, but I should give it a try
  • If custom NPM registry is configured with CI/CD pipelines in Azure DevOps, npx during postinstall will fail 😭
    • Workaround: we need to install babel as a production dependency, instead of relying on npx or as dev dependencies... not great, but not too bad

Anyway... I will find ways to transpile sanitize-html on-the-fly. Many options in my head, need to find the leanest way to do the job.

BTW, thanks for choosing our component.

@compulim
Copy link
Contributor Author

FYI, I tested transpiling [email protected] and it works on IE11. 😉

@compulim compulim closed this as completed Sep 2, 2021
@compulim
Copy link
Contributor Author

compulim commented Sep 8, 2021 •
edited
Loading

@christopherjbaker 4.14.1 is out. sanitize-html is bumped to 2.4.0. Now with zero production-hitting vulnerabilities (as of now). Also works with IE11. Everyone should be happy. 😉

image

FYI, [email protected] was released just an hour ago.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@compulim compulim

Labels
release Release checklist
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@christopherjbaker @compulim