Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

resource/aws_kinesis_firehose_delivery_stream: Use IAM timeout constant for retries, add LakeFormation permissions retries and configuration to tests #17254

Merged
merged 2 commits into from
Jan 29, 2021
Merged

resource/aws_kinesis_firehose_delivery_stream: Use IAM timeout constant for retries, add LakeFormation permissions retries and configuration to tests #17254

merged 2 commits into from
Jan 29, 2021

Commits on Jan 29, 2021

  1. resource/aws_kinesis_firehose_delivery_stream: Use IAM timeout consta… 

    …nt for retries, add LakeFormation permissions retries and configuration to tests

Reference: #16752

Previously:

```
=== CONT  TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_HiveJsonSerDe_Empty
resource_aws_kinesis_firehose_delivery_stream_test.go:638: Step 1/2 error: Error running apply:
Error: error creating Kinesis Firehose Delivery Stream: InvalidArgumentException: Access was denied when calling Glue. Please ensure that the role specified in the data format conversion configuration has the necessary permissions. Insufficient Lake Formation permission(s) on tf-acc-test-4731441258578020859 (Service: AWSGlue; Status Code: 400; Error Code: AccessDeniedException; Request ID: 67116cf3-6102-4d1e-9229-a8c0e63cf9f7; Proxy: null)
--- FAIL: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_HiveJsonSerDe_Empty (21.32s)

=== CONT  TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_Deserializer_Update
resource_aws_kinesis_firehose_delivery_stream_test.go:596: Step 1/3 error: Error running apply:
Error: error creating Kinesis Firehose Delivery Stream: InvalidArgumentException: Access was denied when calling Glue. Please ensure that the role specified in the data format conversion configuration has the necessary permissions. Insufficient Lake Formation permission(s) on tf-acc-test-1453880257072042205 (Service: AWSGlue; Status Code: 400; Error Code: AccessDeniedException; Request ID: a5a8ef8d-e7c8-419b-a5a3-b762145c6783; Proxy: null)
--- FAIL: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_Deserializer_Update (30.13s)

=== CONT  TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_OpenXJsonSerDe_Empty
resource_aws_kinesis_firehose_delivery_stream_test.go:669: Step 1/2 error: Error running apply:
Error: error creating Kinesis Firehose Delivery Stream: InvalidArgumentException: Access was denied when calling Glue. Please ensure that the role specified in the data format conversion configuration has the necessary permissions. Insufficient Lake Formation permission(s) on tf-acc-test-4296742326842474514 (Service: AWSGlue; Status Code: 400; Error Code: AccessDeniedException; Request ID: 67b204a4-290f-4b8b-bba7-ec850759a4fe; Proxy: null)
--- FAIL: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_OpenXJsonSerDe_Empty (18.58s)

=== CONT  TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_OrcSerDe_Empty
resource_aws_kinesis_firehose_delivery_stream_test.go:700: Step 1/2 error: Error running apply:
Error: error creating Kinesis Firehose Delivery Stream: InvalidArgumentException: Access was denied when calling Glue. Please ensure that the role specified in the data format conversion configuration has the necessary permissions. Insufficient Lake Formation permission(s) on tf-acc-test-4205955522949248362 (Service: AWSGlue; Status Code: 400; Error Code: AccessDeniedException; Request ID: 9cd73bb5-9a58-4c35-b2da-4e3f12e17415; Proxy: null)
--- FAIL: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_OrcSerDe_Empty (21.11s)

=== CONT  TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_ParquetSerDe_Empty
resource_aws_kinesis_firehose_delivery_stream_test.go:731: Step 1/2 error: Error running apply:
Error: error creating Kinesis Firehose Delivery Stream: InvalidArgumentException: Access was denied when calling Glue. Please ensure that the role specified in the data format conversion configuration has the necessary permissions. Insufficient Lake Formation permission(s) on tf-acc-test-2371862365551213044 (Service: AWSGlue; Status Code: 400; Error Code: AccessDeniedException; Request ID: 2e0188ba-98ba-496b-99f1-804376dc5862; Proxy: null)
--- FAIL: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_ParquetSerDe_Empty (25.47s)

=== CONT  TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_Serializer_Update
resource_aws_kinesis_firehose_delivery_stream_test.go:762: Step 1/3 error: Error running apply:
Error: error creating Kinesis Firehose Delivery Stream: InvalidArgumentException: Access was denied when calling Glue. Please ensure that the role specified in the data format conversion configuration has the necessary permissions. Insufficient Lake Formation permission(s) on tf-acc-test-2168117662921768660 (Service: AWSGlue; Status Code: 400; Error Code: AccessDeniedException; Request ID: aa73610e-cac0-44a6-8e0a-fded3e5c6bd9; Proxy: null)
--- FAIL: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_Serializer_Update (25.85s)

=== CONT  TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_Enabled
resource_aws_kinesis_firehose_delivery_stream_test.go:490: Step 3/4 error: Error running apply:
Error: Error Updating Kinesis Firehose Delivery Stream: "tf-acc-test-8695271398619453258"
InvalidArgumentException: Access was denied when calling Glue. Please ensure that the role specified in the data format conversion configuration has the necessary permissions. Insufficient Lake Formation permission(s) on tf-acc-test-8695271398619453258 (Service: AWSGlue; Status Code: 400; Error Code: AccessDeniedException; Request ID: c6b9cf64-3918-4140-b85b-fe53c0a4406b; Proxy: null)
--- FAIL: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_Enabled (111.38s)
```

Output from acceptance testing in AWS Commercial:

```
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_basic (131.86s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_disappears (90.21s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ElasticsearchConfigEndpointUpdates (678.89s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ElasticsearchConfigUpdates (975.34s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ElasticsearchWithVpcConfigUpdates (1432.78s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_Deserializer_Update (160.49s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_Enabled (176.11s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_HiveJsonSerDe_Empty (135.95s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_OpenXJsonSerDe_Empty (131.68s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_OrcSerDe_Empty (120.16s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_ParquetSerDe_Empty (136.73s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_Serializer_Update (120.12s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_ErrorOutputPrefix (124.47s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_ExternalUpdate (162.47s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_KinesisStreamSource (95.95s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_ProcessingConfiguration_Empty (126.45s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3basic (136.44s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3KmsKeyArn (124.11s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3Updates (176.36s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_HttpEndpointConfiguration (135.22s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_HttpEndpointConfiguration_RetryDuration (126.68s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_missingProcessingConfiguration (126.21s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_RedshiftConfigUpdates (437.94s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3basic (104.37s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3basicWithSSE (295.11s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3basicWithSSEAndKeyArn (260.98s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3basicWithSSEAndKeyType (248.31s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3basicWithTags (140.47s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3ConfigUpdates (197.64s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3KinesisStreamSource (94.39s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3WithCloudwatchLogging (81.40s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_SplunkConfigUpdates (156.62s)
```

Output from acceptance testing in AWS GovCloud (US):

```
--- FAIL: TestAccAWSKinesisFirehoseDeliveryStream_RedshiftConfigUpdates (18.19s) # unrelated; did not succeed while acquiring capacity
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_basic (112.04s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_disappears (82.08s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ElasticsearchConfigEndpointUpdates (727.48s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ElasticsearchConfigUpdates (640.24s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ElasticsearchWithVpcConfigUpdates (1538.54s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_Deserializer_Update (134.15s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_Enabled (162.17s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_HiveJsonSerDe_Empty (96.38s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_OpenXJsonSerDe_Empty (101.65s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_OrcSerDe_Empty (101.56s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_ParquetSerDe_Empty (110.19s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_Serializer_Update (95.24s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_ErrorOutputPrefix (127.33s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_ExternalUpdate (122.80s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_KinesisStreamSource (103.54s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_ProcessingConfiguration_Empty (91.54s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3basic (120.64s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3KmsKeyArn (107.85s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3Updates (158.79s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_HttpEndpointConfiguration (108.81s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_HttpEndpointConfiguration_RetryDuration (113.39s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_missingProcessingConfiguration (101.69s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3basic (66.63s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3basicWithSSE (212.50s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3basicWithSSEAndKeyArn (191.54s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3basicWithSSEAndKeyType (218.06s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3basicWithTags (126.11s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3ConfigUpdates (169.25s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3KinesisStreamSource (105.90s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3WithCloudwatchLogging (94.65s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_SplunkConfigUpdates (148.60s)
```
    @bflad @YakDriver
    bflad authored and YakDriver committed Jan 29, 2021
    Configuration menu
    Copy the full SHA
    051aab9 View commit details
    Browse the repository at this point in the history

  2. Update CHANGELOG for #17254

    @bflad @YakDriver
    bflad authored and YakDriver committed Jan 29, 2021
    Configuration menu
    Copy the full SHA
    fb792e7 View commit details
    Browse the repository at this point in the history