#641: Specify and remember the Signing Services we want to use for each Repo / Release.

[sdherr]

closes #641

[sdherr]

@mdellweg ping, I believe this is ready for merging. We have been using this patch without problem for a month now.

[mdellweg]

Can you squash the commits and maybe the migrations too?

[sdherr]

Sure, done, thanks!

[sdherr]

Hmm, we'll I'll try to figure out what the automation is complaining about, but also it occurs to me that we probably update this to use get_url now that pulp/pulpcore#3469 is merged.

[sdherr]

I was wrong previously, the version of this PR that we have been running had diverged from this code and there were a couple of things I had to patch up to get it working right. It should be better now though.

[sdherr]

I added some basic tests to exersize this functionality. I don't know why all the github checks are failing; it doesn't seem to be related to my changes and everything works locally here.

[sdherr]

An example api workflow setting signing service on a specific release:

SIGNING_SERVICE_HREF="/pulp/api/v3/signing-services/77d0ba0f-8d42-4bd5-8818-2707ddcdb11d/

# create the repo, update it to set a signing service override on a release named "signed"
http :8080/pulp/api/v3/repositories/deb/apt/ name=test-repo-signed-release
TEST_REPO_SIGNED_RELEASE="/pulp/api/v3/repositories/deb/apt/699d6531-18e6-48ff-8029-7f0ca6390e49/"
http PATCH :8080$TEST_REPO_SIGNED_RELEASE signing_service_release_overrides:="{\"signed\": \"$SIGNING_SERVICE_HREF\"}"

# create a couple releases
http :8080/pulp/api/v3/content/deb/releases/ codename=signed suite=signed distribution=signed
SIGNED_RELEASE_HREF="/pulp/api/v3/content/deb/releases/1e5ef6b8-bac8-43b0-852a-4d9a8a5ada12/"
http :8080/pulp/api/v3/content/deb/releases/ codename=unsigned suite=unsigned distribution=unsigned
UNSIGNED_RELEASE_HREF="/pulp/api/v3/content/deb/releases/f5ccdb99-47c6-4eb0-b7d1-749b2382dda7/"

# create a couple components
http :8080/pulp/api/v3/content/deb/release_components/ component=main release="$SIGNED_RELEASE_HREF"
SIGNED_COMPONENT_HREF="/pulp/api/v3/content/deb/release_components/8d5e72be-7037-4587-8d40-3d2b545a6870/"
http :8080/pulp/api/v3/content/deb/release_components/ component=main release="$UNSIGNED_RELEASE_HREF"
UNSIGNED_COMPONENT_HREF="/pulp/api/v3/content/deb/release_components/b36535a8-3917-4646-938a-60edcfc70750/"

# add structure content to repo
http :8080${TEST_REPO_SIGNED_RELEASE}modify/ add_content_units:="[\"$SIGNED_RELEASE_HREF\", \"$UNSIGNED_RELEASE_HREF\", \"$SIGNED_COMPONENT_HREF\", \"$UNSIGNED_COMPONENT_HREF\"]"

# distribute and publish it
http :8080/pulp/api/v3/distributions/deb/apt/ base_path=test_repo_signed_release name=test_repo_signed_release repository="$TEST_REPO_SIGNED_RELEASE"
http :8080/pulp/api/v3/publications/deb/apt/ structured:=true repository="$TEST_REPO_SIGNED_RELEASE"

# observe that the "signed" release contains an InRelease file and the "unsigned" release doesn't

[sdherr]

An example api workflow setting signing service on the whole repo:

SIGNING_SERVICE_HREF="/pulp/api/v3/signing-services/77d0ba0f-8d42-4bd5-8818-2707ddcdb11d/

# create the repo, update it to set a signing service override on a release named "signed"
http :8080/pulp/api/v3/repositories/deb/apt/ name=test-signed-repo
TEST_SIGNED_REPO="/pulp/api/v3/repositories/deb/apt/64de9a9d-9e55-4ffc-b65a-c89a4b0733cc/"
http PATCH :8080$TEST_SIGNED_REPO signing_service="$SIGNING_SERVICE_HREF"

# create a couple releases
http :8080/pulp/api/v3/content/deb/releases/ codename=signed1 suite=signed1 distribution=signed1
RELEASE_ONE_HREF="/pulp/api/v3/content/deb/releases/56ed4769-ee33-459c-b1ce-0f1de4556bb9/"
http :8080/pulp/api/v3/content/deb/releases/ codename=signed2 suite=signed2 distribution=signed2
RELEASE_TWO_HREF="/pulp/api/v3/content/deb/releases/c5d2165e-0132-423c-8588-14c5c5d431e1/"

# create a couple components
http :8080/pulp/api/v3/content/deb/release_components/ component=main release="$RELEASE_ONE_HREF"
COMPONENT_ONE_HREF="/pulp/api/v3/content/deb/release_components/2933c017-fc08-4e77-9089-e20e5f6eca9d/"
http :8080/pulp/api/v3/content/deb/release_components/ component=main release="$RELEASE_TWO_HREF"
COMPONENT_TWO_HREF="/pulp/api/v3/content/deb/release_components/e392bc8e-ca03-496d-a8c4-7ccacb1960f2/"

# add structure content to repo
http :8080${TEST_SIGNED_REPO}modify/ add_content_units:="[\"$RELEASE_ONE_HREF\", \"$RELEASE_TWO_HREF\", \"$COMPONENT_ONE_HREF\", \"$COMPONENT_TWO_HREF\"]"

# distribute and publish it
http :8080/pulp/api/v3/distributions/deb/apt/ base_path=test_signed_repo name=test_signed_repo repository="$TEST_SIGNED_REPO"
http :8080/pulp/api/v3/publications/deb/apt/ structured:=true repository="$TEST_SIGNED_REPO"

# observe that both "signed1" and "signed2" contian InRelease files

[hstct]

LGTM. Will merge this when I get back to work (so probably monday)

[quba42]

I have not done a full review like @hstct, but had a look over the code and performed the following test workflow:
Set up a signing service:

oci-env shell
curl -L https://github.com/pulp/pulp-fixtures/raw/master/common/GPG-PRIVATE-KEY-pulp-qe | gpg --import
echo "6EDF301256480B9B801EBA3D05A5E6DA269D9D98:6:" | gpg --import-ownertrust
SIGNING_SERVICE_NAME='Pulp_QE'
pulpcore-manager add-signing-service --class 'deb:AptReleaseSigningService' "${SIGNING_SERVICE_NAME}" /src/pulp_deb/pulp_deb/tests/functional/sign_deb_release.sh '6EDF301256480B9B801EBA3D05A5E6DA269D9D98'

Main workflow using Pulp CLI and http:

SIGNING_SERVICE_NAME='Pulp_QE'
ENTITIES_NAME='test'
REMOTE_OPTIONS='--url=https://fixtures.pulpproject.org/debian/ --distribution=ragnarok --distribution=ginnungagap --distribution=nosuite --policy=on_demand'
SIGNING_SERVICE_HREF=$(${PULP_CLI_CMD} signing-service show --name=${SIGNING_SERVICE_NAME} | jq -r .pulp_href)
REMOTE_HREF=$(${PULP_CLI_CMD} deb remote create --name=${ENTITIES_NAME} ${REMOTE_OPTIONS} | jq -r .pulp_href)
${HTTP_CMD} post ${PULP_URL}/pulp/api/v3/repositories/deb/apt/ remote=${REMOTE_HREF} name=${ENTITIES_NAME} signing_service_release_overrides:="{\"ragnarok\": \"$SIGNING_SERVICE_HREF\", \"default\": \"$SIGNING_SERVICE_HREF\"}"
${PULP_CLI_CMD} deb repository sync --name=${ENTITIES_NAME}
APT_PUBLICATION_HREF=$(${PULP_CLI_CMD} deb publication create --simple --structured --repository=${ENTITIES_NAME} | jq -r .pulp_href)
${PULP_CLI_CMD} deb distribution create --name=${ENTITIES_NAME} --base-path=${ENTITIES_NAME} --publication=${APT_PUBLICATION_HREF}

This has worked, and I think the PR can be merged as is, I do have a few closing remarks to add:

1. Great job on adding some basic documentation to docs/feature_overview.rst and the strings used for the API docs!
2. It looks like adding a signing_service_release_overrides = {"default": ... along with simple=True on the publication does not interact. I am fine with this, since we want to deprecate usage of simple=True anyway. (This is just something to be aware of).
3. We should create an issue at pulp-cli-deb to add the signing_service_release_overrides and singing_service parameters to the pulp deb repository create command!

[hstct]

We should create an issue at pulp-cli-deb to add the signing_service_release_overrides and singing_service parameters to the pulp deb repository create command!

I created the issue: pulp/pulp-cli-deb#54