Bootstrap automatically #19
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
orange-kao
force-pushed
the
orange-improve-bootstrap
branch
2 times, most recently
from
3bc0e68 to
4770cf2
Compare
facetoe
reviewed
Oct 28, 2021
rpm_s3_mirror/mirror.py
Outdated
| mirror_start = time.monotonic() | ||
| update_time = now() | ||
| upstream_metadata = upstream_repository.parse_metadata() | ||
|
|
||
| bootstrap = False if self.s3.exists(urlparse(upstream_repository.base_url).path[1:] + "repodata/repomd.xml") else True |
There was a problem hiding this comment.
It would be better to add a method to RPMRepository "exists()" and you can then instantiate mirror_repository a bit earlier and do something like:
bootstrap = False if mirror_repository.exists() else TrueThere was a problem hiding this comment.
Fixed. Thank you.
orange-kao
force-pushed
the
orange-improve-bootstrap
branch
from
4770cf2 to
4598602
Compare
facetoe
reviewed
Nov 1, 2021
rpm_s3_mirror/repository.py
Outdated
| def exists(self): | ||
| try: | ||
| self._req(self.session.get, "repodata/repomd.xml") | ||
| except HTTPError: |
There was a problem hiding this comment.
Should probably check for 404 for not present, and 200 for present and raise on anything else.
There was a problem hiding this comment.
Fixed, thank you. S3 will respond with HTTP 403 (Access Denied) if the object does not exist. Also check HTTP 404 for normal HTTP server behaviour.
Bootstrap automatically if repomd.xml does not exist in S3 bucket
S3 can tolerate consecutive slash in object key, such as "path//object" but S3 will treat it different to "path/object". User misconfiguration may result in excessive use of S3 storage space. So detect and throw exception if consecutive slashes is detected in URL path.
orange-kao
force-pushed
the
orange-improve-bootstrap
branch
from
4598602 to
3689d18
Compare
facetoe
approved these changes
Nov 2, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Remove --bootstrap argument. If
repomd.xmldoes not exist in S3 bucket, bootstrap automatically.