Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Federation e2e tests failing: pulling ci tarball from wrong bucket. #240

Closed
colhom opened this issue Jun 29, 2016 · 5 comments
Closed

Federation e2e tests failing: pulling ci tarball from wrong bucket. #240

colhom opened this issue Jun 29, 2016 · 5 comments

Comments

@colhom
Copy link
Contributor

colhom commented Jun 29, 2016 •
edited
Loading

From kubernetes-e2e-gce-federation logs:

+ local -r bucket=kubernetes-release-dev
++ gsutil cat gs://kubernetes-release-dev/ci/latest.txt
+ build_version=v1.4.0-alpha.0.1035+d30fd0cb0c23ab
+ echo 'Using published version kubernetes-release-dev/v1.4.0-alpha.0.1035+d30fd0cb0c23ab (from ci/latest)'
+ fetch_tars_from_gcs gs://kubernetes-release-dev/ci v1.4.0-alpha.0.1035+d30fd0cb0c23ab
+ local -r gspath=gs://kubernetes-release-dev/ci
+ local -r build_version=v1.4.0-alpha.0.1035+d30fd0cb0c23ab
+ echo 'Pulling binaries from GCS; using server version gs://kubernetes-release-dev/ci/v1.4.0-alpha.0.1035+d30fd0cb0c23ab.'
+ gsutil -mq cp gs://kubernetes-release-dev/ci/v1.4.0-alpha.0.1035+d30fd0cb0c23ab/kubernetes.tar.gz gs://kubernetes-release-dev/ci/v1.4.0-alpha.0.1035+d30fd0cb0c23ab/kubernetes-test.tar.gz .
Using published version kubernetes-release-dev/v1.4.0-alpha.0.1035+d30fd0cb0c23ab (from ci/latest)
Pulling binaries from GCS; using server version gs://kubernetes-release-dev/ci/v1.4.0-alpha.0.1035+d30fd0cb0c23ab.

It has pulled a tarball published by kubernetes-build, not kubernetes-federation-build.

That later causes this error:

FATAL: tagfile /workspace/kubernetes/hack/e2e-internal/../../cluster/../cluster/gce/../../cluster/gce/../../cluster/../federation/manifests/federated-image.tag does not exist. Make sure that you have run build/push-federation-images.sh

I've fixed this error once before ( #146 ) by having kubernetes-federation-build and kubernetes-e2e-gce-federation use an entirely separate ci bucket, so something must have changed since that last PR was merged.

\cc @quinton-hoole @nikhiljindal @ixdy @spxtr
@colhom colhom mentioned this issue Jun 29, 2016
Merged
@ixdy
Copy link
Member

ixdy commented Jun 29, 2016

cc @zmerlynn

@zmerlynn
Copy link
Member

Oh! Your build job is overwriting the new bucket! Thanks for this bug!

@zmerlynn zmerlynn mentioned this issue Jun 29, 2016
Merged
@zmerlynn
Copy link
Member

Okay, I understand why this is corrupting the main kubernetes build pipeline now, that's cool. Now, how are you overriding what bucket gets pulled? Overriding KUBE_GCS_RELEASE_BUCKET in federation jobs? I'm going to need to change that after I reintroduce kubernetes/kubernetes#28193.

@ixdy
Copy link
Member

ixdy commented Jun 29, 2016

Yup: https://github.com/kubernetes/test-infra/blob/master/jenkins/job-configs/kubernetes-jenkins/kubernetes-build.yaml#L77

I forgot about this override too.

@zmerlynn zmerlynn mentioned this issue Jun 29, 2016
Merged
@zmerlynn
Copy link
Member

BTW, this issue should actually be fixed right now, but I'm leaving it open because I plan to reintroduce the PRs that briefly caused it.

zmerlynn added a commit to zmerlynn/test-infra that referenced this issue Jun 29, 2016
@zmerlynn
gce-federation: Add KUBE_GCS_DEV_RELEASE_BUCKET
7cdd098 
In anticipation of re-adding kubernetes/kubernetes#28193

Fixes kubernetes#240 for good
@zmerlynn zmerlynn mentioned this issue Jun 29, 2016
Merged
zmerlynn added a commit to zmerlynn/test-infra that referenced this issue Jun 29, 2016
@zmerlynn
gce-federation: Add KUBE_GCS_DEV_RELEASE_BUCKET
da45c46 
In anticipation of re-adding kubernetes/kubernetes#28193

Fixes kubernetes#240 for good
zmerlynn added a commit to zmerlynn/kubernetes that referenced this issue Jun 29, 2016
@zmerlynn
Revert "Revert "Merge pull request kubernetes#28193 from zmerlynn/pul…
2d6c632 
…l-ci-elsewhere""

Bring back kubernetes#28193. We caught a break in
kubernetes/test-infra#240 and discovered the
previous issue, fixed in
kubernetes/test-infra#241 and
kubernetes/test-infra#244, so I have a pretty
good handle on what was causing the previous bringup issues (and it
wasn't kubernetes#28193). By the time this merges, we'll have good signal on GKE
in the `kubernetes-e2e-gke-updown` job.

This reverts commit ee1d480.
@zmerlynn zmerlynn mentioned this issue Jun 29, 2016
Merged
k8s-github-robot pushed a commit to kubernetes/kubernetes that referenced this issue Jun 29, 2016
Merge pull request #28246 from zmerlynn/revert-revert-28193
5f4aaaa 
Automatic merge from submit-queue

Revert Revert of #28193

Bring back #28193. We caught a break in kubernetes/test-infra#240 and discovered the previous issue, fixed in kubernetes/test-infra#241 and kubernetes/test-infra#244, so I have a pretty good handle on what was causing the previous bringup issues (and it wasn't #28193). By the time this merges, we'll have good signal on GKE in several concurrent GKE jobs.

This reverts commit ee1d480.

[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()
@cjcullen cjcullen mentioned this issue Jun 30, 2016
Closed
michelleN pushed a commit to michelleN/community that referenced this issue Nov 19, 2016
@zmerlynn
Revert "Revert "Merge pull request #28193 from zmerlynn/pull-ci-elsew…
219d085 
…here""

Bring back #28193. We caught a break in
kubernetes/test-infra#240 and discovered the
previous issue, fixed in
kubernetes/test-infra#241 and
kubernetes/test-infra#244, so I have a pretty
good handle on what was causing the previous bringup issues (and it
wasn't #28193). By the time this merges, we'll have good signal on GKE
in the `kubernetes-e2e-gke-updown` job.

This reverts commit ee1d48033366cfbb2e32fc98af6d37c0789e03c2.
michelleN pushed a commit to michelleN/community that referenced this issue Nov 21, 2016
@zmerlynn
Revert "Revert "Merge pull request #28193 from zmerlynn/pull-ci-elsew…
02557b9 
…here""

Bring back #28193. We caught a break in
kubernetes/test-infra#240 and discovered the
previous issue, fixed in
kubernetes/test-infra#241 and
kubernetes/test-infra#244, so I have a pretty
good handle on what was causing the previous bringup issues (and it
wasn't #28193). By the time this merges, we'll have good signal on GKE
in the `kubernetes-e2e-gke-updown` job.

This reverts commit ee1d48033366cfbb2e32fc98af6d37c0789e03c2.
michelleN pushed a commit to michelleN/community that referenced this issue Nov 21, 2016
@zmerlynn
Revert "Revert "Merge pull request #28193 from zmerlynn/pull-ci-elsew…
1302b7f 
…here""

Bring back #28193. We caught a break in
kubernetes/test-infra#240 and discovered the
previous issue, fixed in
kubernetes/test-infra#241 and
kubernetes/test-infra#244, so I have a pretty
good handle on what was causing the previous bringup issues (and it
wasn't #28193). By the time this merges, we'll have good signal on GKE
in the `kubernetes-e2e-gke-updown` job.

This reverts commit ee1d48033366cfbb2e32fc98af6d37c0789e03c2.
michelleN pushed a commit to michelleN/community that referenced this issue Nov 30, 2016
@zmerlynn
Revert "Revert "Merge pull request #28193 from zmerlynn/pull-ci-elsew…
c4953f1 
…here""

Bring back #28193. We caught a break in
kubernetes/test-infra#240 and discovered the
previous issue, fixed in
kubernetes/test-infra#241 and
kubernetes/test-infra#244, so I have a pretty
good handle on what was causing the previous bringup issues (and it
wasn't #28193). By the time this merges, we'll have good signal on GKE
in the `kubernetes-e2e-gke-updown` job.

This reverts commit ee1d48033366cfbb2e32fc98af6d37c0789e03c2.
michelleN pushed a commit to michelleN/community that referenced this issue Nov 30, 2016
@zmerlynn
Revert "Revert "Merge pull request #28193 from zmerlynn/pull-ci-elsew…
6e800f4 
…here""

Bring back #28193. We caught a break in
kubernetes/test-infra#240 and discovered the
previous issue, fixed in
kubernetes/test-infra#241 and
kubernetes/test-infra#244, so I have a pretty
good handle on what was causing the previous bringup issues (and it
wasn't #28193). By the time this merges, we'll have good signal on GKE
in the `kubernetes-e2e-gke-updown` job.

This reverts commit ee1d48033366cfbb2e32fc98af6d37c0789e03c2.
munnerz added a commit to munnerz/test-infra that referenced this issue Aug 29, 2018
@munnerz
Merge pull request kubernetes#240 from jetstack/munnerz-patch-3-1
27477a5 
Bump bazel version in minikube-in-go
mborsz pushed a commit to mborsz/test-infra that referenced this issue Dec 14, 2018
@k8s-ci-robot
Merge pull request kubernetes#240 from krzysied/cluster_loader_measur…
387c94c 
…ements_resource_usage_data

ClusterLoader - Adding resource data types
grantr pushed a commit to grantr/test-infra that referenced this issue Feb 21, 2020
@adrcunha @knative-prow-robot
Clearly display test failures on releases (kubernetes#240)
dc08958 
AKA "display a banner when tests failed for a release, instead of just silently aborting the release"

Bonus: factor out common code in tests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@colhom @zmerlynn @ixdy