aws-test: Add a rhcos_test=smoketested tag #314

cgwalters · 2018-09-25T18:39:06Z

This way the installer can do a query for those.

cgwalters · 2018-09-25T18:39:10Z

(Untested)

jlebon · 2018-09-25T18:40:46Z

Jenkinsfile.aws-test

@@ -59,6 +59,9 @@ node(NODE) {
                    fi

                    cp ${dirpath}/aws-${AWS_REGION}-smoketested.json ${images}/aws-${AWS_REGION}-tested.json
+                    aws ec2 create-tags \
+                        --resources \${ami_intermedia} \
+                        --tags rhcos_test=smoketested


This is bikeshed, though maybe this should just be tested? Given that right now we use smoketested to refer to the first kola run in the cloud job.

This is bikeshed, though maybe this should just be tested?

+1 to generic names until we have several test classes to distinguish between. If we already have plans for more specific tests, than +1 to namespacing with smoke now ;).

Beating the horse further...we discussed tagging in #201 and #150

There seemed to be consensus around #201 (comment)

While that is about the container tags, there was agreement that the container tags and image tags should be the same.

OK right so...based on that comment you're saying the tag should be alpha right?

I updated with that change ⬇️

Jenkinsfile.aws-test

This way the installer can do a query for those.

jlebon · 2018-09-26T15:20:30Z

/lgtm

wking · 2018-10-03T17:00:59Z

Is this working? Searching now, I don't see any alpha-tagged AMIs from the CI account:

$ AWS_PROFILE=ci aws sts get-caller-identity --query Account --output text
460538899914
$ AWS_PROFILE=ci AWS_DEFAULT_REGION=us-east-1 aws ec2 describe-images --filter Name=tag:rhcos_tag,Values=alpha --output json
{
    "Images": []
}
$ AWS_PROFILE=ci AWS_DEFAULT_REGION=us-east-1 aws ec2 describe-images --filter 'Name=name,Values=rhcos*' --query 'sort_by(Images, &CreationDate)[].Name' --output json
[
    "rhcos_dev_5e68e84-hvm", 
    "rhcos_dev_31c37ee-hvm", 
    "rhcos_dev_6152149-hvm", 
    "rhcos_dev_f54819e-hvm", 
    "rhcos_dev_d6f5511-hvm", 
    "rhcos_dev_bb37fd6-hvm", 
    "rhcos_dev_859effb-hvm", 
    "rhcos_dev_7b23b3e-hvm", 
    "rhcos_dev_720aa72-hvm", 
    "rhcos_dev_3e4adc1-hvm", 
    "rhcos_dev_601a8a9-hvm", 
    "rhcos_dev_d96fb50-hvm", 
    "rhcos_dev_62d22c1-hvm", 
    "rhcos_dev_3b8f6c0-hvm", 
    "rhcos_dev_622abd5-hvm", 
    "rhcos_dev_c1ca7d1-hvm", 
    "rhcos_dev_c427d12-hvm", 
    "rhcos_dev_5582faf-hvm", 
    "rhcos_dev_c9a6bb4-hvm", 
    "rhcos_dev_d68316b-hvm", 
    "rhcos_dev_de0a1f7-hvm", 
    "rhcos_dev_2e74ae4-hvm", 
    "rhcos_dev_7ca7835-hvm", 
    "rhcos_dev_6930a92-hvm", 
    "rhcos_dev_2501688-hvm", 
    "rhcos_dev_22740ad-hvm", 
    "rhcos_dev_5ddeccf-hvm", 
    "rhcos_dev_3a7887d-hvm", 
    "rhcos_dev_d7e9c37-hvm", 
    "rhcos_dev_7536381-hvm", 
    "rhcos_dev_3089f99-hvm", 
    "rhcos_dev_c0f6e2b-hvm", 
    "rhcos_dev_0de27f7-hvm", 
    "rhcos_dev_d379d2b-hvm", 
    "rhcos_dev_4.0.6271-hvm", 
    "rhcos_dev_4.0.6272-hvm", 
    "rhcos_dev_4.0.6275-hvm", 
    "rhcos_dev_4.0.6276-hvm", 
    "rhcos_dev_4.0.6295-hvm", 
    "rhcos_dev_4.0.6457-hvm", 
    "rhcos_dev_4.0.6464-hvm", 
    "rhcos_dev_4.0.6468-hvm", 
    "rhcos_dev_4.0.6469-hvm"
]

It's also possible that Name=tag:rhcos_tag,Values=alpha is not the right incantation.

openshift/os@76b3ae5d (aws-test: Add a rhcos_test=smoketested tag, 2018-09-25, openshift/os#314) started adding rhcos_tag=alpha tags to tested AMIs. The likely plan for RHCOS tagging is [1], although they don't have stable docs for that yet. With this commit, we allow the user to specify whichever channel they want, and we search for that tag. If they give us nonsense, we'll now hit AWS, see no images in the nonsense channel, and report that back to the caller. Users that don't care about release channels and just want to use the latest image regardless of channel, can set 'channel' to an empty string. [1]: openshift/os#201 (comment)

miabbott · 2018-10-05T13:57:12Z

@wking I think the syntax might be off based on this #322 and some reading of aws ec2 describe-images help

$ aws ec2 describe-images --filters "Name=tag:rhcos_tag,Values=alpha" --output json
{                                                    
    "Images": [                                                                                
        {                                                                                      
            "Architecture": "x86_64",                                                          
            "CreationDate": "2018-09-27T18:14:11.000Z",                                        
            "ImageId": "ami-01be9707253b2d46d",                                                                                                                                                                    
            "ImageLocation": "531415883065/rhcos_dev_0de27f7-hvm",                                                         
            "ImageType": "machine",                                
            "Public": true,                                       
            "OwnerId": "531415883065",                             
            "State": "available",                                                              
            "BlockDeviceMappings": [                               
                {                                                                       
                    "DeviceName": "/dev/xvda",                     
                    "Ebs": {                                                                                                                                                                                       
                        "Encrypted": false,                                                                                
                        "DeleteOnTermination": true,               
                        "SnapshotId": "snap-0617705761c5d0050",    
                        "VolumeSize": 8,                                                           
                        "VolumeType": "gp2"
                    }
                },
                {
                    "DeviceName": "/dev/xvdb",
                    "VirtualName": "ephemeral0"
                }
            ],
            "Description": "Red Hat CoreOS 4.0.6225 (0de27f731fcdedd767ea99e3b341065146b570628bf9d76d3d6837eb524550da)",
            "EnaSupport": true,
            "Hypervisor": "xen",
            "Name": "rhcos_dev_0de27f7-hvm",
            "RootDeviceName": "/dev/xvda",
            "RootDeviceType": "ebs",
            "SriovNetSupport": "simple",
            "Tags": [
                {
                    "Key": "Name",
                    "Value": "rhcos_dev_0de27f7-hvm"
                },
                {
                    "Key": "ostree_version",
                    "Value": "4.0.6225"
                },
                {
                    "Key": "ostree_commit",
                    "Value": "0de27f731fcdedd767ea99e3b341065146b570628bf9d76d3d6837eb524550da"
                },
                {
                    "Key": "rhcos_tag",
                    "Value": "alpha"
                }
            ],
            "VirtualizationType": "hvm"
        },
...

wking · 2018-10-05T15:06:58Z

That looks like what I ran ;). But guessing based on your success, it looks like the tags are only being pushed to the rh-dev account:

$ AWS_PROFILE=rh-dev AWS_DEFAULT_REGION=us-east-1 aws ec2 describe-images --filters "Name=tag:rhcos_tag,Values=alpha" --query 'sort_by(Images, &CreationDate)[-1].Name' --output text
rhcos_dev_4.0.6562-hvm
$ AWS_PROFILE=ci AWS_DEFAULT_REGION=us-east-1 aws ec2 describe-images --filters "Name=tag:rhcos_tag,Values=alpha" --query 'sort_by(Images, &CreationDate)[-1].Name' --output text
None
$ AWS_PROFILE=rh-dev aws sts get-caller-identity --query Account --output text
531415883065
$ AWS_PROFILE=ci aws sts get-caller-identity --query Account --output text
460538899914

I thought these AMIs were public? Are AMI tags account-specific? If so, that basically sinks this idea, as far as I can tell.

miabbott · 2018-10-05T16:39:49Z

That looks like what I ran ;). But guessing based on your success, it looks like the tags are only being pushed to the rh-dev account:

Heh...yeah, didn't pay enough attention.

We do make the AMI public here: https://github.com/openshift/os/blob/master/Jenkinsfile.aws-test#L89-L91

And according to the docs here, it looks like we are doing the right thing.

@wking Could you open a separate issue for what you are seeing, so we can take this out of PR comments? 😄

miabbott · 2018-10-05T18:01:52Z

FWIW

$ aws ec2 describe-image-attribute --image-id ami-0a08f82608c65f0bc --attribute launchPermission
ami-0a08f82608c65f0bc
LAUNCHPERMISSIONS       all

¯_(ツ)_/¯

openshift-ci-robot requested review from ashcrow and sdemos September 25, 2018 18:39

openshift-ci-robot added the size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. label Sep 25, 2018

cgwalters force-pushed the smoketested-tag branch from 6b6e99b to 0ac784a Compare September 25, 2018 18:39

jlebon reviewed Sep 25, 2018

View reviewed changes

cgwalters force-pushed the smoketested-tag branch from 0ac784a to d65dbf2 Compare September 25, 2018 18:43

jlebon reviewed Sep 25, 2018

View reviewed changes

Jenkinsfile.aws-test Outdated Show resolved Hide resolved

aws-test: Add a rhcos_test=smoketested tag

76b3ae5

This way the installer can do a query for those.

cgwalters force-pushed the smoketested-tag branch from d65dbf2 to 76b3ae5 Compare September 26, 2018 15:12

openshift-ci-robot assigned jlebon Sep 26, 2018

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Sep 26, 2018

openshift-merge-robot merged commit deb0971 into openshift:master Sep 26, 2018

wking mentioned this pull request Oct 3, 2018

oscontainer tags #201

Closed

wking mentioned this pull request Oct 3, 2018

pkg/rhcos/ami: Start supporting RHCOS channels openshift/installer#409

Closed

wking mentioned this pull request Oct 5, 2018

AMI tags not shared between accounts #353

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

aws-test: Add a rhcos_test=smoketested tag #314

aws-test: Add a rhcos_test=smoketested tag #314

cgwalters commented Sep 25, 2018

cgwalters commented Sep 25, 2018

jlebon Sep 25, 2018

wking Sep 25, 2018

miabbott Sep 25, 2018

cgwalters Sep 26, 2018

cgwalters Sep 26, 2018

jlebon commented Sep 26, 2018

wking commented Oct 3, 2018

miabbott commented Oct 5, 2018

wking commented Oct 5, 2018 •

edited

Loading

miabbott commented Oct 5, 2018

miabbott commented Oct 5, 2018

aws-test: Add a rhcos_test=smoketested tag #314

aws-test: Add a rhcos_test=smoketested tag #314

Conversation

cgwalters commented Sep 25, 2018

cgwalters commented Sep 25, 2018

jlebon Sep 25, 2018

Choose a reason for hiding this comment

wking Sep 25, 2018

Choose a reason for hiding this comment

miabbott Sep 25, 2018

Choose a reason for hiding this comment

cgwalters Sep 26, 2018

Choose a reason for hiding this comment

cgwalters Sep 26, 2018

Choose a reason for hiding this comment

jlebon commented Sep 26, 2018

wking commented Oct 3, 2018

miabbott commented Oct 5, 2018

wking commented Oct 5, 2018 • edited Loading

miabbott commented Oct 5, 2018

miabbott commented Oct 5, 2018

wking commented Oct 5, 2018 •

edited

Loading