Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Store attestations for PEP740 #16302

Merged
merged 41 commits into from
Aug 21, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
ab7b15a
Store and retrieve attestations
DarkaMaul Jul 18, 2024
1cf746d
Merge branch 'main' into dm/store-attestations
DarkaMaul Jul 18, 2024
cf1359f
Continue to work.
DarkaMaul Jul 18, 2024
ae262d4
Update attestation storage and retrieval
DarkaMaul Jul 18, 2024
e580a73
Update comments
DarkaMaul Jul 18, 2024
fdcd782
Fix import
DarkaMaul Jul 18, 2024
226fd0d
Please linter
DarkaMaul Jul 18, 2024
00ca535
Use the correct event to attach a publisher_url to a `File`
DarkaMaul Jul 18, 2024
600b398
Rename ReleaseFileAttestation to Attestation
DarkaMaul Jul 23, 2024
2bdedb5
Merge branch 'main' into dm/store-attestations
DarkaMaul Jul 23, 2024
4fd0143
Merge branch 'main' into dm/store-attestations
DarkaMaul Jul 25, 2024
9dd8a91
Update names
DarkaMaul Jul 25, 2024
bd042f2
Update table migration
DarkaMaul Jul 26, 2024
1d05571
Update metrics
DarkaMaul Jul 29, 2024
a55a00f
Update attestations storage
DarkaMaul Jul 30, 2024
1e65c5f
Merge branch 'main' into dm/store-attestations
DarkaMaul Aug 7, 2024
d9bd6a8
Update pypi-attestations and sigstore dependencies
DarkaMaul Aug 7, 2024
efefd50
Fix wrong merge.
DarkaMaul Aug 7, 2024
7f29774
Generate Provenance file on upload.
DarkaMaul Aug 8, 2024
2f31749
Generate and store provenance file during upload.
DarkaMaul Aug 9, 2024
fed1c40
Merge branch 'main' into dm/store-attestations
DarkaMaul Aug 12, 2024
0c96751
Improve tests
DarkaMaul Aug 12, 2024
ff338da
Fix test error
DarkaMaul Aug 12, 2024
284c488
Fix test error
DarkaMaul Aug 12, 2024
31f633b
Merge branch 'main' into dm/store-attestations
DarkaMaul Aug 13, 2024
16eb83a
Merge branch 'main' into dm/store-attestations
DarkaMaul Aug 13, 2024
a36bb1a
Merge branch 'main' into dm/store-attestations
DarkaMaul Aug 13, 2024
ba6752d
Fix merge error
DarkaMaul Aug 14, 2024
a26047d
Simplify legacy answer
DarkaMaul Aug 15, 2024
8b99fb7
Introduce AttestationsService
DarkaMaul Aug 15, 2024
342b8bf
Rename AttestationsService to ReleaseVerification service and integra…
DarkaMaul Aug 16, 2024
6e4c91e
Remove useless check
DarkaMaul Aug 16, 2024
80988d0
Integrate generate_and_store_provenance within persist_attestations
DarkaMaul Aug 16, 2024
7a10776
Merge branch 'main' into dm/store-attestations
DarkaMaul Aug 16, 2024
9914be2
Merge branch 'refs/heads/main' into dm/store-attestations
DarkaMaul Aug 16, 2024
eec0b46
Remove file.publisher_url which is no longer used.
DarkaMaul Aug 16, 2024
98d39c4
Rename ReleaseAttestationService to IntegrityService
DarkaMaul Aug 20, 2024
0a013a8
Merge branch 'main' into dm/store-attestations
di Aug 21, 2024
08a4d9d
Merge branch 'main' into dm/store-attestations
di Aug 21, 2024
8d00c87
Linting
di Aug 21, 2024
2ca1918
requirements: bump sigstore, pypi-attestations
woodruffw Aug 21, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions requirements/main.in
Original file line number Diff line number Diff line change
Expand Up @@ -62,8 +62,8 @@ redis>=2.8.0,<6.0.0
rfc3986
sentry-sdk
setuptools
sigstore~=3.0.0
pypi-attestations==0.0.9
sigstore~=3.2.0
pypi-attestations==0.0.11
sqlalchemy[asyncio]>=2.0,<3.0
stdlib-list
stripe
Expand Down
12 changes: 6 additions & 6 deletions requirements/main.txt
Original file line number Diff line number Diff line change
Expand Up @@ -1770,9 +1770,9 @@ pyparsing==3.1.2 \
--hash=sha256:a1bac0ce561155ecc3ed78ca94d3c9378656ad4c94c1270de543f621420f94ad \
--hash=sha256:f9db75911801ed778fe61bb643079ff86601aca99fcae6345aa67292038fb742
# via linehaul
pypi-attestations==0.0.9 \
--hash=sha256:3bfc07f64a8db0d6e2646720e70df7c7cb01a2936056c764a2cc3268969332f2 \
--hash=sha256:4b38cce5d221c8145cac255bfafe650ec0028d924d2b3572394df8ba8f07a609
pypi-attestations==0.0.11 \
--hash=sha256:b730e6b23874d94da0f3817b1f9dd3ecb6a80d685f62a18ad96e5b0396149ded \
--hash=sha256:e74329074f049568591e300373e12fcd46a35e21723110856546e33bf2949efa
# via -r requirements/main.in
pyqrcode==1.2.1 \
--hash=sha256:1b2812775fa6ff5c527977c4cd2ccb07051ca7d0bc0aecf937a43864abe5eff6 \
Expand Down Expand Up @@ -2079,9 +2079,9 @@ sentry-sdk==2.13.0 \
--hash=sha256:6beede8fc2ab4043da7f69d95534e320944690680dd9a963178a49de71d726c6 \
--hash=sha256:8d4a576f7a98eb2fdb40e13106e41f330e5c79d72a68be1316e7852cf4995260
# via -r requirements/main.in
sigstore==3.0.0 \
--hash=sha256:6cc7dc92607c2fd481aada0f3c79e710e4c6086e3beab50b07daa9a50a79d109 \
--hash=sha256:a6a9538a648e112a0c3d8092d3f73a351c7598164764f1e73a6b5ba406a3a0bd
sigstore==3.2.0 \
--hash=sha256:25c8a871a3a6adf959c0cde598ea8bef8794f1a29277d067111eb4ded4ba7f65 \
--hash=sha256:d18508f34febb7775065855e92557fa1c2c16580df88f8e8903b9514438bad44
# via
# -r requirements/main.in
# pypi-attestations
Expand Down
28 changes: 28 additions & 0 deletions tests/common/db/attestation.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import hashlib

import factory

from warehouse.attestations.models import Attestation

from .base import WarehouseFactory


class AttestationFactory(WarehouseFactory):
class Meta:
model = Attestation

file = factory.SubFactory("tests.common.db.packaging.FileFactory")
attestation_file_blake2_digest = factory.LazyAttribute(
lambda o: hashlib.blake2b(o.file.filename.encode("utf8")).hexdigest()
)
7 changes: 7 additions & 0 deletions tests/common/db/packaging.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@
from warehouse.utils import readme

from .accounts import UserFactory
from .attestation import AttestationFactory
from .base import WarehouseFactory
from .observations import ObserverFactory

Expand Down Expand Up @@ -140,6 +141,12 @@ class Meta:
)
)

attestations = factory.RelatedFactoryList(
AttestationFactory,
factory_related_name="file",
size=1,
)


class FileEventFactory(WarehouseFactory):
class Meta:
Expand Down
25 changes: 25 additions & 0 deletions tests/unit/api/test_simple.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
from pyramid.testing import DummyRequest

from warehouse.api import simple
from warehouse.attestations import IIntegrityService
from warehouse.packaging.utils import API_VERSION

from ...common.db.accounts import UserFactory
Expand Down Expand Up @@ -87,6 +88,16 @@ def test_selects(self, header, expected):


class TestSimpleIndex:

@pytest.fixture
def db_request(self, db_request):
"""Override db_request to add the Release Verification service"""
db_request.find_service = lambda svc, name=None, context=None: {
IIntegrityService: pretend.stub(),
}.get(svc)

return db_request

@pytest.mark.parametrize(
("content_type", "renderer_override"),
CONTENT_TYPE_PARAMS,
Expand Down Expand Up @@ -185,6 +196,17 @@ def test_quarantined_project_omitted_from_index(self, db_request):


class TestSimpleDetail:
@pytest.fixture
def db_request(self, db_request):
"""Override db_request to add the Release Verification service"""
db_request.find_service = lambda svc, name=None, context=None: {
IIntegrityService: pretend.stub(
get_provenance_digest=lambda *args, **kwargs: None,
),
}.get(svc)

return db_request

def test_redirects(self, pyramid_request):
project = pretend.stub(normalized_name="foo")

Expand Down Expand Up @@ -286,6 +308,7 @@ def test_with_files_no_serial(self, db_request, content_type, renderer_override)
"upload-time": f.upload_time.isoformat() + "Z",
"data-dist-info-metadata": False,
"core-metadata": False,
"provenance": None,
}
for f in files
],
Expand Down Expand Up @@ -334,6 +357,7 @@ def test_with_files_with_serial(self, db_request, content_type, renderer_overrid
"upload-time": f.upload_time.isoformat() + "Z",
"data-dist-info-metadata": False,
"core-metadata": False,
"provenance": None,
}
for f in files
],
Expand Down Expand Up @@ -427,6 +451,7 @@ def test_with_files_with_version_multi_digit(
if f.metadata_file_sha256_digest is not None
else False
),
"provenance": None,
}
for f in files
],
Expand Down
11 changes: 11 additions & 0 deletions tests/unit/attestations/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
Loading