Support publishing CI artifacts to S3 #16659

rifelpet · 2024-07-06T01:56:22Z

Ref: #16637 kubernetes/k8s.io#5127 kubernetes/k8s.io#2625

More context in kubernetes/k8s.io#6950 (comment)

k8s-ci-robot · 2024-07-06T01:56:24Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

k8s-ci-robot · 2024-07-06T01:57:35Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from rifelpet. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

rifelpet · 2024-07-06T02:00:13Z

tests/e2e/kubetest2-kops/deployer/build.go

@@ -34,6 +36,7 @@ import (
 const (
 	defaultJobName = "pull-kops-e2e-kubernetes-aws"
 	defaultGCSPath = "gs://k8s-staging-kops/pulls/%v/pull-%v"
+	defaultS3Path  = "s3://k8s-infra-kops-ci-results/pulls/%v/pull-%v"


Created by kubernetes/k8s.io#6950

k8s-ci-robot · 2024-07-06T06:17:38Z

@rifelpet: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-kops-e2e-k8s-aws-calico-k8s-infra	`35c1d03`	link	true	`/test pull-kops-e2e-k8s-aws-calico-k8s-infra`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

upodroid · 2024-07-06T15:49:52Z

We want kubetest2-kops to dynamically create a temporary s3 bucket when the presubmit is running, similar to this code
https://github.com/kubernetes/kops/blob/master/tests/e2e/kubetest2-kops/gce/gcs.go

I made the IAM changes, but it allows presubmits to write to our CI bucket which isn't ideal(existing behaviour we should change).

rifelpet · 2024-07-07T11:36:09Z

Yes I see the migrated job on your PR passed, using k8s-staging-kops:

https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kops/16648/pull-kops-e2e-k8s-aws-calico-k8s-infra/1809653969442574336

I believe this means we could migrate all presubmit jobs to use this, but you're right giving presubmit jobs write permissions to the same bucket used for post-merge artifacts like version markers is not great.

We have the k8s-infra-kops-ci-results bucket created in kubernetes/k8s.io#2678, is that in the correct project and can we use it for either presubmit artifacts or version markers, and then revoke permissions for presubmits to whichever bucket we choose for version markers?

upodroid · 2024-07-07T21:06:56Z

For now, let's retain the existing behaviour as AWS presubmits have been writing to the k8s-staging-kops bucket and let's try to switch to dynamic buckets on AWS.

As for gs://k8s-infra-kops-ci-results, I don't want to use this bucket and we should delete it.

justinsb · 2024-07-19T11:46:13Z

Makefile

@@ -271,6 +274,15 @@ gcs-publish-ci: gsutil version-dist-ci
 	echo "${GCS_URL}/${VERSION}" > ${UPLOAD}/${LATEST_FILE}
 	gsutil -h "Cache-Control:private, max-age=0, no-transform" cp ${UPLOAD}/${LATEST_FILE} ${GCS_LOCATION}

+.PHONY: s3-publish-ci
+s3-publish-ci: VERSION := ${KOPS_CI_VERSION}+${GITSHA}
+s3-publish-ci: version-dist-ci


We do also have hack/upload, which might make this easier?

justinsb · 2024-07-19T11:47:47Z

tests/e2e/kubetest2-kops/builder/build.go

 	cmd.SetDir(b.KopsRoot)
 	exec.InheritOutput(cmd)
 	if err := cmd.Run(); err != nil {
 		return nil, err
 	}

 	// Get the full path (including subdirectory) that we uploaded to
-	// It is written by gcs-publish-ci to .build/upload/latest-ci.txt
+	// It is written by the *-publish-ci make tasks to .build/upload/latest-ci.txt


It might be nice to spell out the two tasks here (assuming we can't combine them), to make it more explicit and to help with searching...

justinsb

This actually LGTM. We're probably holding this PR for now (given that the test-infra one is closed), but I think a good reason to upload artifacts to S3 would be to cut down on bandwidth etc costs for testing. But I guess they aren't that large....

upodroid · 2024-07-19T13:11:35Z

We do need to ship this to fix the AWS presubmits publishing to the k8s-staging-kops bucket. We carried over that practice when the jobs were migrated to the community cluster.

k8s-triage-robot · 2024-10-17T14:10:52Z

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

Mark this PR as fresh with /remove-lifecycle stale
Close this PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot · 2024-11-16T14:16:45Z

The Kubernetes project currently lacks enough active contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

Mark this PR as fresh with /remove-lifecycle rotten
Close this PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-triage-robot · 2024-12-16T14:53:50Z

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

Reopen this PR with /reopen
Mark this PR as fresh with /remove-lifecycle rotten
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

k8s-ci-robot · 2024-12-16T14:53:56Z

@k8s-triage-robot: Closed this PR.

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied

After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied

After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

Reopen this PR with /reopen

Mark this PR as fresh with /remove-lifecycle rotten

Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Add s3-publish-ci make target

ca488d3

k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jul 6, 2024

k8s-ci-robot requested a review from johngmyers July 6, 2024 01:57

k8s-ci-robot requested a review from olemarkus July 6, 2024 01:57

rifelpet mentioned this pull request Jul 6, 2024

Kops - Create S3 bucket for hosting CI results kubernetes/k8s.io#6950

Closed

rifelpet force-pushed the s3-stage-location branch from 8e2ecb8 to 85e8356 Compare July 6, 2024 01:59

rifelpet commented Jul 6, 2024

View reviewed changes

rifelpet changed the title ~~Add s3-publish-ci make target~~ Support publishing CI artifacts to S3 Jul 6, 2024

rifelpet added 2 commits July 5, 2024 21:39

Support publishing CI artifacts to S3 for AWS clusters

3971d9f

Support publishing version markers to S3 buckets

35c1d03

rifelpet force-pushed the s3-stage-location branch from 85e8356 to 35c1d03 Compare July 6, 2024 02:39

rifelpet mentioned this pull request Jul 7, 2024

Delete k8s-infra-kops-ci-results GCS bucket kubernetes/k8s.io#6952

Merged

justinsb reviewed Jul 19, 2024

View reviewed changes

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Oct 17, 2024

k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Nov 16, 2024

k8s-ci-robot closed this Dec 16, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support publishing CI artifacts to S3 #16659

Support publishing CI artifacts to S3 #16659

rifelpet commented Jul 6, 2024 •

edited

Loading

k8s-ci-robot commented Jul 6, 2024

k8s-ci-robot commented Jul 6, 2024

rifelpet Jul 6, 2024

k8s-ci-robot commented Jul 6, 2024

upodroid commented Jul 6, 2024

rifelpet commented Jul 7, 2024

upodroid commented Jul 7, 2024

justinsb Jul 19, 2024

justinsb Jul 19, 2024

justinsb left a comment •

edited

Loading

upodroid commented Jul 19, 2024 •

edited

Loading

k8s-triage-robot commented Oct 17, 2024

k8s-triage-robot commented Nov 16, 2024

k8s-triage-robot commented Dec 16, 2024

k8s-ci-robot commented Dec 16, 2024

Support publishing CI artifacts to S3 #16659

Support publishing CI artifacts to S3 #16659

Conversation

rifelpet commented Jul 6, 2024 • edited Loading

k8s-ci-robot commented Jul 6, 2024

k8s-ci-robot commented Jul 6, 2024

rifelpet Jul 6, 2024

Choose a reason for hiding this comment

k8s-ci-robot commented Jul 6, 2024

upodroid commented Jul 6, 2024

rifelpet commented Jul 7, 2024

upodroid commented Jul 7, 2024

justinsb Jul 19, 2024

Choose a reason for hiding this comment

justinsb Jul 19, 2024

Choose a reason for hiding this comment

justinsb left a comment • edited Loading

Choose a reason for hiding this comment

upodroid commented Jul 19, 2024 • edited Loading

k8s-triage-robot commented Oct 17, 2024

k8s-triage-robot commented Nov 16, 2024

k8s-triage-robot commented Dec 16, 2024

k8s-ci-robot commented Dec 16, 2024

rifelpet commented Jul 6, 2024 •

edited

Loading

justinsb left a comment •

edited

Loading

upodroid commented Jul 19, 2024 •

edited

Loading