Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐛Allow using S3 in different region than us-east-1 and S3 settings are not following correct definitions (⚠️ devops) #5667

Merged
merged 23 commits into from
Apr 15, 2024

Conversation

sanderegg
Copy link
Member

@sanderegg sanderegg commented Apr 11, 2024

What do these changes do?

The S3 settings module is very old and that is an understatement.

Since we are deploying into Oregon, we now face the problem that it is impossible to switch the S3 settings to another region because the S3_ENDPOINT cannot be nullified due to a validator.

This PR removes that validator and removes the default values of S3_REGION which makes changing the region very cumbersome.

Related issue/s

How to test

Dev-ops checklist

@sanderegg sanderegg added this to the Enchanted Odyssey milestone Apr 11, 2024
@sanderegg sanderegg self-assigned this Apr 11, 2024
@sanderegg sanderegg force-pushed the bugfix/old-s3-settings-crap branch from 1d8421d to f4d9a18 Compare April 11, 2024 16:19
Copy link

codecov bot commented Apr 11, 2024

Codecov Report

Attention: Patch coverage is 85.29412% with 5 lines in your changes are missing coverage. Please review.

Project coverage is 87.5%. Comparing base (cafbf96) to head (5acc4c2).
Report is 109 commits behind head on master.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff            @@
##           master   #5667      +/-   ##
=========================================
+ Coverage    84.5%   87.5%    +2.9%     
=========================================
  Files          10    1362    +1352     
  Lines         214   56354   +56140     
  Branches       25    1255    +1230     
=========================================
+ Hits          181   49321   +49140     
- Misses         23    6761    +6738     
- Partials       10     272     +262     
Flag Coverage Δ
integrationtests 65.0% <50.0%> (?)
unittests 85.4% <85.2%> (+0.8%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
packages/aws-library/src/aws_library/s3/client.py 100.0% <ø> (ø)
...ttings-library/src/settings_library/basic_types.py 100.0% <100.0%> (ø)
...ckages/settings-library/src/settings_library/s3.py 100.0% <100.0%> (ø)
...s/agent/src/simcore_service_agent/core/settings.py 100.0% <100.0%> (ø)
...mcore_service_agent/modules/volumes_cleanup/_s3.py 98.3% <100.0%> (ø)
...es/dynamic_sidecar/docker_service_specs/sidecar.py 90.1% <100.0%> (ø)
...car/src/simcore_service_dask_sidecar/file_utils.py 95.6% <88.8%> (ø)
...ice_director_v2/modules/dynamic_sidecar/volumes.py 66.0% <0.0%> (ø)

... and 1335 files with indirect coverage changes

@sanderegg sanderegg changed the title 🐛S3 settings are not following correct definitions 🐛Allow using S3 in different region than us-east-1 and S3 settings are not following correct definitions Apr 11, 2024
@sanderegg sanderegg added a:storage issue related to storage service a:director-v2 issue related with the director-v2 service a:dynamic-sidecar dynamic-sidecar service a:dask-service Any of the dask services: dask-scheduler/sidecar or worker a:models-library a:agent agent service a:aws-library labels Apr 12, 2024
@sanderegg sanderegg force-pushed the bugfix/old-s3-settings-crap branch from 32b335b to d772a75 Compare April 12, 2024 12:54
@sanderegg sanderegg marked this pull request as ready for review April 12, 2024 12:55
Copy link
Member

@pcrespov pcrespov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thx

Copy link
Contributor

@GitHK GitHK left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all good, except te missing validator

@sanderegg sanderegg force-pushed the bugfix/old-s3-settings-crap branch from 925bfe9 to 5acc4c2 Compare April 14, 2024 20:51
Copy link

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud

@sanderegg sanderegg merged commit 5884e48 into ITISFoundation:master Apr 15, 2024
56 checks passed
@sanderegg sanderegg deleted the bugfix/old-s3-settings-crap branch April 15, 2024 06:18
@sanderegg sanderegg changed the title 🐛Allow using S3 in different region than us-east-1 and S3 settings are not following correct definitions 🐛Allow using S3 in different region than us-east-1 and S3 settings are not following correct definitions (⚠️ devops) Apr 16, 2024
pcrespov pushed a commit to pcrespov/osparc-simcore that referenced this pull request Apr 16, 2024
@matusdrobuliak66 matusdrobuliak66 mentioned this pull request Apr 26, 2024
28 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a:agent agent service a:aws-library a:dask-service Any of the dask services: dask-scheduler/sidecar or worker a:director-v2 issue related with the director-v2 service a:dynamic-sidecar dynamic-sidecar service a:models-library a:storage issue related to storage service
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants