pkg/rhcos/ami: Update name filter to "redhat-coreos-*" #572

wking · 2018-10-30T16:27:01Z

The OS folks have a new pipeline for publishing these AMIs, and they're setting these new names. Current output:

$ curl -sL https://rhcos-release-browser-coreos.int.open.paas.redhat.com/storage/releases/maipo/47.33/meta.json | jq .amis
[
  {
    "name": "us-east-1",
    "hvm": "ami-0fd0205c3c81d0412"
  }
]
$ AWS_DEFAULT_REGION=us-east-1 aws ec2 describe-images --image-ids ami-0fd0205c3c81d0412 --output json
{
   "Images": [
       {
           "VirtualizationType": "hvm",
           "Description": "Red Hat CoreOS \"maipo\" (OpenShift) 47.33",
           "Tags": [
               {
                   "Value": "redhat-coreos-maipo-47.33-hvm",
                   "Key": "Name"
               }
           ],
           "Hypervisor": "xen",
           "EnaSupport": true,
           "SriovNetSupport": "simple",
           "ImageId": "ami-0fd0205c3c81d0412",
           "State": "available",
           "BlockDeviceMappings": [
               {
                   "DeviceName": "/dev/xvda",
                   "Ebs": {
                       "Encrypted": false,
                       "DeleteOnTermination": true,
                       "VolumeType": "gp2",
                       "VolumeSize": 8,
                       "SnapshotId": "snap-08318a8024617da0b"
                   }
               },
               {
                   "DeviceName": "/dev/xvdb",
                   "VirtualName": "ephemeral0"
               }
           ],
           "Architecture": "x86_64",
           "ImageLocation": "531415883065/redhat-coreos-maipo-47.33-hvm",
           "RootDeviceType": "ebs",
           "OwnerId": "531415883065",
           "RootDeviceName": "/dev/xvda",
           "CreationDate": "2018-10-30T15:27:07.000Z",
           "Public": false,
           "ImageType": "machine",
           "Name": "redhat-coreos-maipo-47.33-hvm"
       }
   ]
}

For the previous pipeline, see the output in d01ac5d (#290).

CC @abhinavdahiya, @crawford, @cgwalters.

The OS folks have a new pipeline for publishing these AMIs, and they're setting these new names. Current output: $ curl -sL https://rhcos-release-browser-coreos.int.open.paas.redhat.com/storage/releases/maipo/47.33/meta.json | jq .amis [ { "name": "us-east-1", "hvm": "ami-0fd0205c3c81d0412" } ] $ AWS_DEFAULT_REGION=us-east-1 aws ec2 describe-images --image-ids ami-0fd0205c3c81d0412 --output json { "Images": [ { "VirtualizationType": "hvm", "Description": "Red Hat CoreOS \"maipo\" (OpenShift) 47.33", "Tags": [ { "Value": "redhat-coreos-maipo-47.33-hvm", "Key": "Name" } ], "Hypervisor": "xen", "EnaSupport": true, "SriovNetSupport": "simple", "ImageId": "ami-0fd0205c3c81d0412", "State": "available", "BlockDeviceMappings": [ { "DeviceName": "/dev/xvda", "Ebs": { "Encrypted": false, "DeleteOnTermination": true, "VolumeType": "gp2", "VolumeSize": 8, "SnapshotId": "snap-08318a8024617da0b" } }, { "DeviceName": "/dev/xvdb", "VirtualName": "ephemeral0" } ], "Architecture": "x86_64", "ImageLocation": "531415883065/redhat-coreos-maipo-47.33-hvm", "RootDeviceType": "ebs", "OwnerId": "531415883065", "RootDeviceName": "/dev/xvda", "CreationDate": "2018-10-30T15:27:07.000Z", "Public": false, "ImageType": "machine", "Name": "redhat-coreos-maipo-47.33-hvm" } ] } For the previous pipeline, see the output in d01ac5d (pkg/rhcos: Default to the most-recent AMI, 2018-09-19, openshift#290).

cgwalters · 2018-10-30T16:35:25Z

Why don't we get the AMI information from the release metadata in #554 ?

abhinavdahiya · 2018-10-30T16:46:08Z

Why don't we get the AMI information from the release metadata in #281 ?

It seems like https://rhcos-release-browser-coreos.int.open.paas.redhat.com/storage/releases can only be accessed on vpn, thats why?

cgwalters · 2018-10-30T17:07:56Z

Right, though we are discussing changing that.

cgwalters · 2018-10-30T17:08:50Z

(Also I meant #554 - edited my first comment too)

wking · 2018-10-30T17:11:32Z

Why don't we get the AMI information from the release metadata in #554 ?

Because this is a one-line change that gets us working while we work out how to use the JSON metadata ;). This AWS-scraping approach will also continue to work in regions beyond us-east-1, while the curl in my initial comment shows that the new JSON only gives the AMI for that region.

wking · 2018-10-30T17:12:32Z

e2e-aws:

level=fatal msg="Error executing openshift-install: no RHCOS AMIs found in us-east-1"

which is currently blocked on getting access to the new AMIs from the CI account.

cgwalters · 2018-10-30T17:26:32Z

while the curl in my initial comment shows that the new JSON only gives the AMI for that region.

That's because it's only that region right now.

wking · 2018-10-30T17:52:33Z

while the curl in my initial comment shows that the new JSON only gives the AMI for that region.

That's because it's only that region right now.

Can we fix that (again)? @coverprice has been pushing openshift-dev users into us-east-2 and us-west-1 regions where we have higher resource quotas.

wking · 2018-10-30T22:14:21Z

CI access to the new pipeline's AMIs is in flight with this just built.

wking · 2018-10-30T22:17:05Z

$ AWS_PROFILE=ci AWS_DEFAULT_REGION=us-east-1 aws ec2 describe-images --filter 'Name=name,Values=redhat-coreos-*' --query 'sort_by(Images, &CreationDate)[-1]' --output json
{
    "VirtualizationType": "hvm", 
    "Description": "Red Hat CoreOS \"maipo\" (OpenShift) 47.14", 
    "Hypervisor": "xen", 
    "EnaSupport": true, 
    "SriovNetSupport": "simple", 
    "ImageId": "ami-0c353b2d94b0a6247", 
    "State": "available", 
    "BlockDeviceMappings": [
        {
            "DeviceName": "/dev/xvda", 
            "Ebs": {
                "Encrypted": false, 
                "DeleteOnTermination": true, 
                "VolumeType": "gp2", 
                "VolumeSize": 8, 
                "SnapshotId": "snap-058e45208e113a230"
            }
        }, 
        {
            "DeviceName": "/dev/xvdb", 
            "VirtualName": "ephemeral0"
        }
    ], 
    "Architecture": "x86_64", 
    "ImageLocation": "531415883065/redhat-coreos-maipo-47.14-hvm", 
    "RootDeviceType": "ebs", 
    "OwnerId": "531415883065", 
    "RootDeviceName": "/dev/xvda", 
    "CreationDate": "2018-10-26T20:15:56.000Z", 
    "Public": false, 
    "ImageType": "machine", 
    "Name": "redhat-coreos-maipo-47.14-hvm"
}

/retest

wking · 2018-10-30T23:13:56Z

e2e-aws:

• Failure in Spec Teardown (AfterEach) [628.028 seconds]
[sig-storage] Secrets
/tmp/openshift/build-rpms/rpm/BUILD/origin-4.0.0/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/common/secrets_volume.go:33
  should be consumable from pods in volume with defaultMode set [NodeConformance] [Conformance] [Suite:openshift/conformance/parallel/minimal] [Suite:k8s] [AfterEach]
  /tmp/openshift/build-rpms/rpm/BUILD/origin-4.0.0/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:684

  Oct 30 22:58:13.503: Couldn't delete ns: "e2e-tests-secrets-djgnb": namespace e2e-tests-secrets-djgnb was not deleted with limit: timed out waiting for the condition, namespaced content other than pods remain (&errors.errorString{s:"namespace e2e-tests-secrets-djgnb was not deleted with limit: timed out waiting for the condition, namespaced content other than pods remain"})

  /tmp/openshift/build-rpms/rpm/BUILD/origin-4.0.0/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:319

/retest

abhinavdahiya · 2018-10-30T23:19:31Z

/lgtm

openshift-ci-robot · 2018-10-30T23:19:38Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: abhinavdahiya, wking

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [abhinavdahiya,wking]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Oct 30, 2018

openshift-ci-robot requested review from crawford and smarterclayton October 30, 2018 16:27

wking force-pushed the new-rhcos-ami-name branch from 089283f to 8c0d7b6 Compare October 30, 2018 16:28

openshift-ci-robot assigned abhinavdahiya Oct 30, 2018

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Oct 30, 2018

openshift-merge-robot merged commit fa4c243 into openshift:master Oct 31, 2018

wking mentioned this pull request Oct 31, 2018

rhcos: implement image discovery for new pipeline #554

Merged

wking deleted the new-rhcos-ami-name branch October 31, 2018 20:28

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pkg/rhcos/ami: Update name filter to "redhat-coreos-*" #572

pkg/rhcos/ami: Update name filter to "redhat-coreos-*" #572

wking commented Oct 30, 2018 •

edited

Loading

cgwalters commented Oct 30, 2018 •

edited

Loading

abhinavdahiya commented Oct 30, 2018

cgwalters commented Oct 30, 2018

cgwalters commented Oct 30, 2018

wking commented Oct 30, 2018 •

edited

Loading

wking commented Oct 30, 2018

cgwalters commented Oct 30, 2018

wking commented Oct 30, 2018

wking commented Oct 30, 2018

wking commented Oct 30, 2018

wking commented Oct 30, 2018

abhinavdahiya commented Oct 30, 2018

openshift-ci-robot commented Oct 30, 2018

pkg/rhcos/ami: Update name filter to "redhat-coreos-*" #572

pkg/rhcos/ami: Update name filter to "redhat-coreos-*" #572

Conversation

wking commented Oct 30, 2018 • edited Loading

cgwalters commented Oct 30, 2018 • edited Loading

abhinavdahiya commented Oct 30, 2018

cgwalters commented Oct 30, 2018

cgwalters commented Oct 30, 2018

wking commented Oct 30, 2018 • edited Loading

wking commented Oct 30, 2018

cgwalters commented Oct 30, 2018

wking commented Oct 30, 2018

wking commented Oct 30, 2018

wking commented Oct 30, 2018

wking commented Oct 30, 2018

abhinavdahiya commented Oct 30, 2018

openshift-ci-robot commented Oct 30, 2018

wking commented Oct 30, 2018 •

edited

Loading

cgwalters commented Oct 30, 2018 •

edited

Loading

wking commented Oct 30, 2018 •

edited

Loading