Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

🚀 Release v1.53.0 #4236

Closed
23 of 24 tasks
mrnicegyu11 opened this issue May 15, 2023 · 6 comments
Closed
23 of 24 tasks

🚀 Release v1.53.0 #4236

mrnicegyu11 opened this issue May 15, 2023 · 6 comments
Assignees
@Surfict @matusdrobuliak66
Labels
release Preparation for pre-release/release t:maintenance Some planned maintenance work
Milestone
Pastel de Nata

Comments

@mrnicegyu11
Copy link
Member

mrnicegyu11 commented May 15, 2023 •
edited by matusdrobuliak66
Loading

Release version

1.53.0

Commit SHA

2ab6150f23ae2765f7373d3c49019eea79e8faaa

Previous pre-release

https://github.com/ITISFoundation/osparc-simcore/releases/tag/staging_PastelDeNata4

Did the commit CI suceeded?

  • The commit CI succeeded.

Motivation

lorem ipsum

Changes

Staging JellyBeans3

Release Issue: #4156

Staging PastelDeNata1

Release Issue: #4200

Staging PastelDeNata2

Release Issue: #4224

Staging PastelDeNata3

Release Issue: #4240

Staging PastelDeNata4

Release Issue: #4261

No response

Devops check 👷

Tests assessment: e2e testing check 🧪

No response

Test assessment: targeted-testing 🔍️

No response

Test assessment: user-testing 🧐

No response

Summary 📝

  • Prepare release link
make release-prod version=1.53.0 git_sha=2ab6150f23ae2765f7373d3c49019eea79e8faaa
  • Draft release changelog
  • Announce maintenance ( ** ANNOUNCE AT LEAST 24 HOURS BEFORE ** )
  • redis {"start": "2023-05-31T13:00:00.000Z", "end": "2023-05-31T14:30:00.000Z", "reason": "Release 1.53.0"}
    • aws
    • dalco
    • tip
  • status page (https://manage.statuspage.io/)
    • osparc
    • s4l
  • mattermost channels
    • maintenance
    • power users

Releasing 🚀

  • Maintenance page up.
  • Release by publishing draft
  • Check release CI
  • Check deployed
    • aws deploy
    • dalco deploy
    • tip deploy
  • Delete announcement
  • Check e2e runs
  • Announce
:tada: https://github.com/ITISFoundation/osparc-simcore/releases/tag/v1.53.0
@mrnicegyu11 mrnicegyu11 added t:maintenance Some planned maintenance work release Preparation for pre-release/release labels May 15, 2023
@mrnicegyu11
Copy link
Member Author

Caveat: Deployment agent in aws-prod is paused for

@GitHK
Copy link
Contributor

GitHK commented May 16, 2023

As asked by @odeimaiz I have added the country code UK for osparc.io
Since this restarts the director-v2 we need to schedule a maintenance in order to release the changes for the env vars changed on the webserver.

Relative commit

https://git.speag.com/oSparc/osparc-ops-deployment-configuration/-/tree/a427c0599462e366406b125e2b6e37722c1e5316

@matusdrobuliak66 matusdrobuliak66 added this to the Pastel de Nata milestone May 16, 2023
@GitHK
Copy link
Contributor

GitHK commented May 16, 2023

@mrnicegyu11 @matusdrobuliak66 @odeimaiz
The env var were update on prod AWS since the auto deploy agent is paused and this old required a webserver restart.
As of now consider this to be deployed on production. There is no more urgency to release this before the 1st of June

@matusdrobuliak66
Copy link
Contributor

env variable LOGIN_2FA_CODE_EXPIRATION_SEC=960 needed to be added to the osparc-ops-deployment-configuration

@mrnicegyu11
Copy link
Member Author

There is something very weird going on with node dalco06. We absolutely cannot ssh into it anymore, but it is still part of the docker swarm and runs containers. Let's use the user logout and maintenance to remove this machine from the swarm for further investigation. During the maintenance when users are logged out, please run docker node update --availability drain osparc-dalco-06, potentially making sure that also from dalco staging no user is working on this machine. Thanks

@matusdrobuliak66
Copy link
Contributor

Release after notes:

  • A lot of hanging sidecars were on Dalco, and also a few on AWS, it took us a long time to close them (most of them had s3 upload issues)
  • in-house network stopped working for 0.5-1 hour
  • few ENV vars were missing in the config file after the first deployment (DYNAMIC_SIDECAR_ENABLE_VOLUME_LIMITS & LOG_FORMAT_LOCAL_DEV_ENABLED) so we added them and retag the ops repos

Observations:

  • the company network was not working and in aws we limit who can access it with ips from the company, therefore we were not able to access the AWS monitoring stack
  • In the Dalco production when the maintenance page was up and the Redis maintenance key setup, we were not able to use testing.* endpoint to bypass these restrictions

@pcrespov pcrespov mentioned this issue Jun 7, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Surfict Surfict

@matusdrobuliak66 matusdrobuliak66

Labels
release Preparation for pre-release/release t:maintenance Some planned maintenance work
Projects
None yet
Milestone
Pastel de Nata
Development

No branches or pull requests
4 participants
@Surfict @GitHK @mrnicegyu11 @matusdrobuliak66