service/s3control: GetBucketLifecycleConfigurationOutput Missing Rules[].Filter.And.Tags

[bflad]

Confirm by changing [ ] to [x] below to ensure that it's a bug:

• I've gone though Developer Guide and API reference
• I've checked AWS Forums and StackOverflow for answers
• I've searched for previous similar issues and didn't find any solution

Describe the bug

The AWS Go SDK does not include the LifecycleRuleAndOperator.Tags when parsing the API response.

Version of AWS SDK for Go?

v1.35.7

Version of Go (go version)?

go version go1.15.2 darwin/amd64

To Reproduce (observed behavior)

This code should be very close -- our access to an Outpost was removed before I could fully run this reproduction code while trying to pare it down from real world Terraform AWS Provider code.

package main

import (
	"log"

	"github.com/aws/aws-sdk-go/aws"
	"github.com/aws/aws-sdk-go/aws/session"
	"github.com/aws/aws-sdk-go/service/s3control"
)

func main() {
	// Fill these in correctly
	accountID := "123456789012"
	outpostID := "op-1234567890123456"
	bucketName := "lifecycleruleandoperator-testing"
	region := "us-east-1"

	config := aws.NewConfig().WithLogLevel(aws.LogDebugWithHTTPBody).WithRegion(region)
	sess := session.Must(session.NewSession(config))
	svc := s3control.New(sess)

	createBucketOutput, err := svc.CreateBucket(&s3control.CreateBucketInput{
		Bucket:    aws.String(bucketName),
		OutpostId: aws.String(outpostID),
	})

	if err != nil {
		log.Printf("CreateBucket error: %s", err)
		return
	}

	bucketARN := aws.StringValue(createBucketOutput.BucketArn)

	_, err = svc.PutBucketLifecycleConfiguration(&s3control.PutBucketLifecycleConfigurationInput{
		AccountId: aws.String(accountID),
		Bucket:    aws.String(bucketARN),
		LifecycleConfiguration: &s3control.LifecycleConfiguration{
			Rules: []*s3control.LifecycleRule{
				{
					Expiration: &s3control.LifecycleExpiration{
						Days: aws.Int64(7),
					},
					Filter: &s3control.LifecycleRuleFilter{
						And: &s3control.LifecycleRuleAndOperator{
							Tags: []*s3control.S3Tag{
								{
									Key:   aws.String("key1"),
									Value: aws.String("value1"),
								},
								{
									Key:   aws.String("key2"),
									Value: aws.String("value2"),
								},
							},
						},
					},
					ID:     aws.String("test"),
					Status: aws.String(s3control.ExpirationStatusEnabled),
				},
			},
		},
	})

	if err != nil {
		log.Printf("PutBucketLifecycleConfiguration error: %s", err)
		return
	}

	output, err := svc.GetBucketLifecycleConfiguration(&s3control.GetBucketLifecycleConfigurationInput{
		AccountId: aws.String(accountID),
		Bucket:    aws.String(bucketARN),
	})

	if err != nil {
		log.Printf("GetBucketLifecycleConfiguration error: %s", err)
		return
	}

	log.Printf("GetBucketLifecycleConfigurationOutput: %+v", output)
}

Here's the Terraform AWS Provider logging I had for this before our access was removed (note the lack of information under And):

2020/10/12 11:04:02 [DEBUG] [aws-sdk-go] <?xml version="1.0"?>
<GetBucketLifecycleConfigurationResult>
  <Rules>
    <Rule>
      <Filter>
        <And>
          <Tags>
            <Tag>
              <Value>value2</Value>
              <Key>key2</Key>
            </Tag>
            <Tag>
              <Value>value1updated</Value>
              <Key>key1</Key>
            </Tag>
          </Tags>
        </And>
      </Filter>
      <Expiration>
        <Days>365</Days>
        <ExpiredObjectDeleteMarker>false</ExpiredObjectDeleteMarker>
      </Expiration>
      <ID>test</ID>
      <Status>Enabled</Status>
    </Rule>
  </Rules>
</GetBucketLifecycleConfigurationResult>

GetBucketLifecycleConfigurationOutput: {
  Expiration: {
    Days: 365,
    ExpiredObjectDeleteMarker: false
  },
  Filter: {
    And: {

    }
  },
  ID: "test",
  Status: "Enabled"
}

Expected behavior

And.Tags to contain Tags from XML response.

Additional context

The s3 API model includes additional information for the Tags type:

    "LifecycleRuleAndOperator":{
      "type":"structure",
      "members":{
        "Prefix":{"shape":"Prefix"},
        "Tags":{
          "shape":"TagSet",
          "flattened":true,
          "locationName":"Tag"
        }
      }
    },
...
    "Tag":{
      "type":"structure",
      "required":[
        "Key",
        "Value"
      ],
      "members":{
        "Key":{"shape":"ObjectKey"},
        "Value":{"shape":"Value"}
      }
    },
    "TagCount":{"type":"integer"},
    "TagSet":{
      "type":"list",
      "member":{
        "shape":"Tag",
        "locationName":"Tag"
      }
    },

Which generates the following in the service code:

type LifecycleRuleAndOperator struct {
	_ struct{} `type:"structure"`

	// Prefix identifying one or more objects to which the rule applies.
	Prefix *string `type:"string"`

	// All of these tags must exist in the object's tag set in order for the rule
	// to apply.
	Tags []*Tag `locationName:"Tag" locationNameList:"Tag" type:"list" flattened:"true"`
}
// ...
type Tag struct {
	_ struct{} `type:"structure"`

	// Name of the object key.
	//
	// Key is a required field
	Key *string `min:"1" type:"string" required:"true"`

	// Value of the tag.
	//
	// Value is a required field
	Value *string `type:"string" required:"true"`
}

While s3control API model does not contain this extra type information:

    "LifecycleRuleAndOperator":{
      "type":"structure",
      "members":{
        "Prefix":{"shape":"Prefix"},
        "Tags":{"shape":"S3TagSet"}
      }
    },
...
    "S3Tag":{
      "type":"structure",
      "required":[
        "Key",
        "Value"
      ],
      "members":{
        "Key":{"shape":"TagKeyString"},
        "Value":{"shape":"TagValueString"}
      }
    },
    "S3TagSet":{
      "type":"list",
      "member":{"shape":"S3Tag"}
    },

Which generates the following service code:

type LifecycleRuleAndOperator struct {
	_ struct{} `type:"structure"`

	// Prefix identifying one or more objects to which the rule applies.
	Prefix *string `type:"string"`

	// All of these tags must exist in the object's tag set in order for the rule
	// to apply.
	Tags []*S3Tag `type:"list"`
}
// ...
type S3Tag struct {
	_ struct{} `type:"structure"`

	// Key is a required field
	Key *string `min:"1" type:"string" required:"true"`

	// Value is a required field
	Value *string `type:"string" required:"true"`
}

[jasdel]

Thanks for creating this issue @bflad. It looks like this is a bug in the Amazon S3 Control API model definition for the LifecycleRuleAndOperator shape, usage of S3TagSet

We'll raise this issue with the S3 team to let them know about the modeling issue, and work with them to resolve this issue.

[github-actions]

We have noticed this issue has not received attention in 1 year. We will close this issue for now. If you think this is in error, please feel free to comment and reopen the issue.