Skip to content

Commit

Permalink
Support zstd-compressed repositories, including streaming compression
Browse files Browse the repository at this point in the history
Sometimes a zstd-compressed repository will have used streaming
compression. The simpler zstd package does not support it, so use the
full zstandard package instead.
  • Loading branch information
nicois committed Jul 18, 2024
1 parent 201825c commit d5a0895
Show file tree
Hide file tree
Showing 3 changed files with 6 additions and 6 deletions.
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ dependencies = [
"python-dateutil>=2.8.1,<3",
"botocore>=1.23.50,<2",
"lxml>=4.6.5,<5",
"zstd>=1.5.5.1",
"zstandard>=0.21.0",
]

[project.optional-dependencies]
Expand Down
2 changes: 1 addition & 1 deletion rpm_s3_mirror.spec
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Requires: python3-requests
Requires: python3-dateutil
Requires: python3-botocore
Requires: python3-lxml
Requires: python3-zstd
Requires: python3-zstandard
Requires: systemd
Requires: zchunk

Expand Down
8 changes: 4 additions & 4 deletions rpm_s3_mirror/repository.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Copyright (c) 2020 Aiven, Helsinki, Finland. https://aiven.io/
import dataclasses
import lzma
import zstd
import zstandard
import re
import subprocess
from abc import abstractmethod
Expand Down Expand Up @@ -215,9 +215,9 @@ def _compress(self, root, open_size, open_checksum):

def decompress(filename: Path | str) -> bytes:
try:
with open(filename, "rb") as f:
return zstd.decompress(f.read())
except zstd.Error:
with zstandard.open(filename) as f:
return f.read()
except zstandard.ZstdError:
with gzip.open(filename) as f:
return f.read()

Expand Down

0 comments on commit d5a0895

Please sign in to comment.