Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

r/glue_crawler - support mongodb target #15934

Merged
merged 7 commits into from
Nov 4, 2020
Merged

r/glue_crawler - support mongodb target #15934

merged 7 commits into from
Nov 4, 2020

Conversation

DrFaust92
Copy link
Collaborator

@DrFaust92 DrFaust92 commented Oct 30, 2020 •
edited
Loading

Community Note

  • Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for pull request followers and do not help prioritize the request

Release note for CHANGELOG:

resource_aws_ glue_crawler - support mongodb target

Output from acceptance testing:

$ make testacc TESTARGS='-run= TestAccAWSGlueCrawler_'
--- PASS: TestAccAWSGlueCrawler_disappears (63.98s)
--- PASS: TestAccAWSGlueCrawler_Role_Name_Path (68.00s)
--- PASS: TestAccAWSGlueCrawler_Role_ARN_NoPath (72.37s)
--- PASS: TestAccAWSGlueCrawler_Role_ARN_Path (72.38s)
--- PASS: TestAccAWSGlueCrawler_S3Target_ConnectionName (98.68s)
--- PASS: TestAccAWSGlueCrawler_RemoveTablePrefix (103.74s)
--- PASS: TestAccAWSGlueCrawler_S3Target (108.69s)
--- PASS: TestAccAWSGlueCrawler_Configuration (109.71s)
--- PASS: TestAccAWSGlueCrawler_DynamodbTarget (109.81s)
--- PASS: TestAccAWSGlueCrawler_S3Target_Exclusions (110.36s)
--- PASS: TestAccAWSGlueCrawler_TablePrefix (110.43s)
--- PASS: TestAccAWSGlueCrawler_SchemaChangePolicy (110.90s)
--- PASS: TestAccAWSGlueCrawler_JdbcTarget_Exclusions (113.60s)
--- PASS: TestAccAWSGlueCrawler_SecurityConfiguration (116.05s)
--- PASS: TestAccAWSGlueCrawler_Tags (139.93s)
--- PASS: TestAccAWSGlueCrawler_Schedule (141.61s)
--- PASS: TestAccAWSGlueCrawler_mongoDBTarget_multiple (141.61s)
--- PASS: TestAccAWSGlueCrawler_S3Target_Multiple (141.84s)
--- PASS: TestAccAWSGlueCrawler_Description (101.13s)
--- PASS: TestAccAWSGlueCrawler_CatalogTarget (176.19s)
--- PASS: TestAccAWSGlueCrawler_mongoDBTarget (103.81s)
--- PASS: TestAccAWSGlueCrawler_Classifiers (139.34s)
--- PASS: TestAccAWSGlueCrawler_mongoDBTarget_scan_all (140.11s)
--- PASS: TestAccAWSGlueCrawler_JdbcTarget (108.07s)
--- PASS: TestAccAWSGlueCrawler_DynamodbTarget_scanRate (138.67s)
--- PASS: TestAccAWSGlueCrawler_JdbcTarget_Multiple (146.42s)
--- PASS: TestAccAWSGlueCrawler_DynamodbTarget_scanAll (141.91s)
--- PASS: TestAccAWSGlueCrawler_CatalogTarget_Multiple (261.86s)

@DrFaust92 DrFaust92 requested a review from a team October 30, 2020 09:34
@ghost ghost added size/XL Managed by automation to categorize the size of a PR. service/glue Issues and PRs that pertain to the glue service. documentation Introduces or discusses updates to documentation. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure. labels Oct 30, 2020
@DrFaust92 DrFaust92 added the enhancement Requests to existing resources that expand the functionality or scope. label Oct 30, 2020
ewbankkit
ewbankkit approved these changes Oct 30, 2020
View reviewed changes
Copy link
Contributor

@ewbankkit ewbankkit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

$ make testacc TEST=./aws/ TESTARGS='-run=TestAccAWSGlueCrawler_'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSGlueCrawler_ -timeout 120m
=== RUN   TestAccAWSGlueCrawler_DynamodbTarget
=== PAUSE TestAccAWSGlueCrawler_DynamodbTarget
=== RUN   TestAccAWSGlueCrawler_DynamodbTarget_scanAll
=== PAUSE TestAccAWSGlueCrawler_DynamodbTarget_scanAll
=== RUN   TestAccAWSGlueCrawler_DynamodbTarget_scanRate
=== PAUSE TestAccAWSGlueCrawler_DynamodbTarget_scanRate
=== RUN   TestAccAWSGlueCrawler_JdbcTarget
=== PAUSE TestAccAWSGlueCrawler_JdbcTarget
=== RUN   TestAccAWSGlueCrawler_JdbcTarget_Exclusions
=== PAUSE TestAccAWSGlueCrawler_JdbcTarget_Exclusions
=== RUN   TestAccAWSGlueCrawler_JdbcTarget_Multiple
=== PAUSE TestAccAWSGlueCrawler_JdbcTarget_Multiple
=== RUN   TestAccAWSGlueCrawler_mongoDBTarget
=== PAUSE TestAccAWSGlueCrawler_mongoDBTarget
=== RUN   TestAccAWSGlueCrawler_mongoDBTarget_scan_all
=== PAUSE TestAccAWSGlueCrawler_mongoDBTarget_scan_all
=== RUN   TestAccAWSGlueCrawler_mongoDBTarget_multiple
=== PAUSE TestAccAWSGlueCrawler_mongoDBTarget_multiple
=== RUN   TestAccAWSGlueCrawler_S3Target
=== PAUSE TestAccAWSGlueCrawler_S3Target
=== RUN   TestAccAWSGlueCrawler_S3Target_ConnectionName
=== PAUSE TestAccAWSGlueCrawler_S3Target_ConnectionName
=== RUN   TestAccAWSGlueCrawler_S3Target_Exclusions
=== PAUSE TestAccAWSGlueCrawler_S3Target_Exclusions
=== RUN   TestAccAWSGlueCrawler_S3Target_Multiple
=== PAUSE TestAccAWSGlueCrawler_S3Target_Multiple
=== RUN   TestAccAWSGlueCrawler_CatalogTarget
=== PAUSE TestAccAWSGlueCrawler_CatalogTarget
=== RUN   TestAccAWSGlueCrawler_CatalogTarget_Multiple
=== PAUSE TestAccAWSGlueCrawler_CatalogTarget_Multiple
=== RUN   TestAccAWSGlueCrawler_disappears
=== PAUSE TestAccAWSGlueCrawler_disappears
=== RUN   TestAccAWSGlueCrawler_Classifiers
=== PAUSE TestAccAWSGlueCrawler_Classifiers
=== RUN   TestAccAWSGlueCrawler_Configuration
=== PAUSE TestAccAWSGlueCrawler_Configuration
=== RUN   TestAccAWSGlueCrawler_Description
=== PAUSE TestAccAWSGlueCrawler_Description
=== RUN   TestAccAWSGlueCrawler_Role_ARN_NoPath
=== PAUSE TestAccAWSGlueCrawler_Role_ARN_NoPath
=== RUN   TestAccAWSGlueCrawler_Role_ARN_Path
=== PAUSE TestAccAWSGlueCrawler_Role_ARN_Path
=== RUN   TestAccAWSGlueCrawler_Role_Name_Path
=== PAUSE TestAccAWSGlueCrawler_Role_Name_Path
=== RUN   TestAccAWSGlueCrawler_Schedule
=== PAUSE TestAccAWSGlueCrawler_Schedule
=== RUN   TestAccAWSGlueCrawler_SchemaChangePolicy
=== PAUSE TestAccAWSGlueCrawler_SchemaChangePolicy
=== RUN   TestAccAWSGlueCrawler_TablePrefix
=== PAUSE TestAccAWSGlueCrawler_TablePrefix
=== RUN   TestAccAWSGlueCrawler_RemoveTablePrefix
=== PAUSE TestAccAWSGlueCrawler_RemoveTablePrefix
=== RUN   TestAccAWSGlueCrawler_Tags
=== PAUSE TestAccAWSGlueCrawler_Tags
=== RUN   TestAccAWSGlueCrawler_SecurityConfiguration
=== PAUSE TestAccAWSGlueCrawler_SecurityConfiguration
=== CONT  TestAccAWSGlueCrawler_DynamodbTarget
=== CONT  TestAccAWSGlueCrawler_disappears
=== CONT  TestAccAWSGlueCrawler_CatalogTarget_Multiple
=== CONT  TestAccAWSGlueCrawler_DynamodbTarget_scanRate
=== CONT  TestAccAWSGlueCrawler_CatalogTarget
=== CONT  TestAccAWSGlueCrawler_S3Target_Multiple
=== CONT  TestAccAWSGlueCrawler_SecurityConfiguration
=== CONT  TestAccAWSGlueCrawler_S3Target_Exclusions
=== CONT  TestAccAWSGlueCrawler_Tags
=== CONT  TestAccAWSGlueCrawler_S3Target_ConnectionName
=== CONT  TestAccAWSGlueCrawler_RemoveTablePrefix
=== CONT  TestAccAWSGlueCrawler_S3Target
=== CONT  TestAccAWSGlueCrawler_TablePrefix
=== CONT  TestAccAWSGlueCrawler_mongoDBTarget_multiple
=== CONT  TestAccAWSGlueCrawler_SchemaChangePolicy
=== CONT  TestAccAWSGlueCrawler_mongoDBTarget_scan_all
=== CONT  TestAccAWSGlueCrawler_mongoDBTarget
=== CONT  TestAccAWSGlueCrawler_JdbcTarget_Multiple
=== CONT  TestAccAWSGlueCrawler_JdbcTarget_Exclusions
=== CONT  TestAccAWSGlueCrawler_JdbcTarget
=== CONT  TestAccAWSGlueCrawler_CatalogTarget
    resource_aws_glue_crawler_test.go:719: Step 1/3 error: Error running apply: 2020/10/30 09:46:27 [DEBUG] Using modified User-Agent: Terraform/0.12.26 HashiCorp-terraform-exec/0.10.0
        
        Error: error creating Glue crawler: InvalidInputException: Insufficient Lake Formation permission(s) on tf-acc-test-3798935878093255471_table_0 (Service: AmazonDataCatalog; Status Code: 400; Error Code: AccessDeniedException; Request ID: c1cdd1ba-e85c-4799-9182-c3c64e786fc1; Proxy: null)
        
        
=== CONT  TestAccAWSGlueCrawler_CatalogTarget_Multiple
    resource_aws_glue_crawler_test.go:791: Step 1/4 error: Error running apply: 2020/10/30 09:46:30 [DEBUG] Using modified User-Agent: Terraform/0.12.26 HashiCorp-terraform-exec/0.10.0
        
        Error: error creating Glue crawler: InvalidInputException: Insufficient Lake Formation permission(s) on tf-acc-test-4728270018400728346_table_0 (Service: AmazonDataCatalog; Status Code: 400; Error Code: AccessDeniedException; Request ID: 2d0a4738-5f0a-4e7f-ac07-10ea22fb3aed; Proxy: null)
        
        
--- FAIL: TestAccAWSGlueCrawler_CatalogTarget (60.83s)
=== CONT  TestAccAWSGlueCrawler_DynamodbTarget_scanAll
--- FAIL: TestAccAWSGlueCrawler_CatalogTarget_Multiple (69.31s)
=== CONT  TestAccAWSGlueCrawler_Role_ARN_NoPath
--- PASS: TestAccAWSGlueCrawler_disappears (114.35s)
=== CONT  TestAccAWSGlueCrawler_Schedule
--- PASS: TestAccAWSGlueCrawler_S3Target_ConnectionName (138.00s)
=== CONT  TestAccAWSGlueCrawler_Role_Name_Path
--- PASS: TestAccAWSGlueCrawler_Role_ARN_NoPath (130.61s)
=== CONT  TestAccAWSGlueCrawler_Role_ARN_Path
--- PASS: TestAccAWSGlueCrawler_DynamodbTarget (212.67s)
=== CONT  TestAccAWSGlueCrawler_Configuration
--- PASS: TestAccAWSGlueCrawler_S3Target_Exclusions (213.01s)
=== CONT  TestAccAWSGlueCrawler_Description
--- PASS: TestAccAWSGlueCrawler_RemoveTablePrefix (214.73s)
=== CONT  TestAccAWSGlueCrawler_Classifiers
--- PASS: TestAccAWSGlueCrawler_mongoDBTarget (214.78s)
--- PASS: TestAccAWSGlueCrawler_JdbcTarget_Exclusions (214.97s)
--- PASS: TestAccAWSGlueCrawler_TablePrefix (216.05s)
--- PASS: TestAccAWSGlueCrawler_SchemaChangePolicy (216.07s)
--- PASS: TestAccAWSGlueCrawler_SecurityConfiguration (216.54s)
--- PASS: TestAccAWSGlueCrawler_JdbcTarget (216.55s)
--- PASS: TestAccAWSGlueCrawler_S3Target (222.37s)
--- PASS: TestAccAWSGlueCrawler_Role_Name_Path (106.49s)
--- PASS: TestAccAWSGlueCrawler_S3Target_Multiple (265.39s)
--- PASS: TestAccAWSGlueCrawler_JdbcTarget_Multiple (266.22s)
--- PASS: TestAccAWSGlueCrawler_mongoDBTarget_multiple (266.29s)
--- PASS: TestAccAWSGlueCrawler_mongoDBTarget_scan_all (266.83s)
--- PASS: TestAccAWSGlueCrawler_DynamodbTarget_scanRate (266.86s)
--- PASS: TestAccAWSGlueCrawler_Tags (272.02s)
--- PASS: TestAccAWSGlueCrawler_Role_ARN_Path (78.64s)
--- PASS: TestAccAWSGlueCrawler_DynamodbTarget_scanAll (222.04s)
--- PASS: TestAccAWSGlueCrawler_Schedule (187.87s)
--- PASS: TestAccAWSGlueCrawler_Configuration (92.41s)
--- PASS: TestAccAWSGlueCrawler_Description (92.26s)
--- PASS: TestAccAWSGlueCrawler_Classifiers (105.63s)
FAIL
FAIL	github.com/terraform-providers/terraform-provider-aws/aws	320.448s
FAIL
GNUmakefile:27: recipe for target 'testacc' failed
make: *** [testacc] Error 1

The failures are unrelated to this change and have been noticed previously.

@breathingdust
Copy link
Member

LGTM 🚀 Thanks @DrFaust92

Verified Acceptance Tests in Commercial (us-west-2)

make testacc TEST=./aws/ TESTARGS='-run=TestAccAWSGlueCrawler_'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSGlueCrawler_ -timeout 120m
=== RUN   TestAccAWSGlueCrawler_DynamodbTarget
=== PAUSE TestAccAWSGlueCrawler_DynamodbTarget
=== RUN   TestAccAWSGlueCrawler_DynamodbTarget_scanAll
=== PAUSE TestAccAWSGlueCrawler_DynamodbTarget_scanAll
=== RUN   TestAccAWSGlueCrawler_DynamodbTarget_scanRate
=== PAUSE TestAccAWSGlueCrawler_DynamodbTarget_scanRate
=== RUN   TestAccAWSGlueCrawler_JdbcTarget
=== PAUSE TestAccAWSGlueCrawler_JdbcTarget
=== RUN   TestAccAWSGlueCrawler_JdbcTarget_Exclusions
=== PAUSE TestAccAWSGlueCrawler_JdbcTarget_Exclusions
=== RUN   TestAccAWSGlueCrawler_JdbcTarget_Multiple
=== PAUSE TestAccAWSGlueCrawler_JdbcTarget_Multiple
=== RUN   TestAccAWSGlueCrawler_mongoDBTarget
=== PAUSE TestAccAWSGlueCrawler_mongoDBTarget
=== RUN   TestAccAWSGlueCrawler_mongoDBTarget_scan_all
=== PAUSE TestAccAWSGlueCrawler_mongoDBTarget_scan_all
=== RUN   TestAccAWSGlueCrawler_mongoDBTarget_multiple
=== PAUSE TestAccAWSGlueCrawler_mongoDBTarget_multiple
=== RUN   TestAccAWSGlueCrawler_S3Target
=== PAUSE TestAccAWSGlueCrawler_S3Target
=== RUN   TestAccAWSGlueCrawler_S3Target_ConnectionName
=== PAUSE TestAccAWSGlueCrawler_S3Target_ConnectionName
=== RUN   TestAccAWSGlueCrawler_S3Target_Exclusions
=== PAUSE TestAccAWSGlueCrawler_S3Target_Exclusions
=== RUN   TestAccAWSGlueCrawler_S3Target_Multiple
=== PAUSE TestAccAWSGlueCrawler_S3Target_Multiple
=== RUN   TestAccAWSGlueCrawler_CatalogTarget
=== PAUSE TestAccAWSGlueCrawler_CatalogTarget
=== RUN   TestAccAWSGlueCrawler_CatalogTarget_Multiple
=== PAUSE TestAccAWSGlueCrawler_CatalogTarget_Multiple
=== RUN   TestAccAWSGlueCrawler_disappears
=== PAUSE TestAccAWSGlueCrawler_disappears
=== RUN   TestAccAWSGlueCrawler_Classifiers
=== PAUSE TestAccAWSGlueCrawler_Classifiers
=== RUN   TestAccAWSGlueCrawler_Configuration
=== PAUSE TestAccAWSGlueCrawler_Configuration
=== RUN   TestAccAWSGlueCrawler_Description
=== PAUSE TestAccAWSGlueCrawler_Description
=== RUN   TestAccAWSGlueCrawler_Role_ARN_NoPath
=== PAUSE TestAccAWSGlueCrawler_Role_ARN_NoPath
=== RUN   TestAccAWSGlueCrawler_Role_ARN_Path
=== PAUSE TestAccAWSGlueCrawler_Role_ARN_Path
=== RUN   TestAccAWSGlueCrawler_Role_Name_Path
=== PAUSE TestAccAWSGlueCrawler_Role_Name_Path
=== RUN   TestAccAWSGlueCrawler_Schedule
=== PAUSE TestAccAWSGlueCrawler_Schedule
=== RUN   TestAccAWSGlueCrawler_SchemaChangePolicy
=== PAUSE TestAccAWSGlueCrawler_SchemaChangePolicy
=== RUN   TestAccAWSGlueCrawler_TablePrefix
=== PAUSE TestAccAWSGlueCrawler_TablePrefix
=== RUN   TestAccAWSGlueCrawler_RemoveTablePrefix
=== PAUSE TestAccAWSGlueCrawler_RemoveTablePrefix
=== RUN   TestAccAWSGlueCrawler_Tags
=== PAUSE TestAccAWSGlueCrawler_Tags
=== RUN   TestAccAWSGlueCrawler_SecurityConfiguration
=== PAUSE TestAccAWSGlueCrawler_SecurityConfiguration
=== CONT  TestAccAWSGlueCrawler_DynamodbTarget
=== CONT  TestAccAWSGlueCrawler_disappears
=== CONT  TestAccAWSGlueCrawler_Description
=== CONT  TestAccAWSGlueCrawler_mongoDBTarget_multiple
=== CONT  TestAccAWSGlueCrawler_CatalogTarget_Multiple
=== CONT  TestAccAWSGlueCrawler_CatalogTarget
=== CONT  TestAccAWSGlueCrawler_S3Target_Multiple
=== CONT  TestAccAWSGlueCrawler_S3Target_Exclusions
=== CONT  TestAccAWSGlueCrawler_S3Target_ConnectionName
=== CONT  TestAccAWSGlueCrawler_S3Target
=== CONT  TestAccAWSGlueCrawler_Schedule
=== CONT  TestAccAWSGlueCrawler_RemoveTablePrefix
=== CONT  TestAccAWSGlueCrawler_SecurityConfiguration
=== CONT  TestAccAWSGlueCrawler_Tags
=== CONT  TestAccAWSGlueCrawler_Role_ARN_NoPath
=== CONT  TestAccAWSGlueCrawler_Role_Name_Path
=== CONT  TestAccAWSGlueCrawler_SchemaChangePolicy
=== CONT  TestAccAWSGlueCrawler_Configuration
=== CONT  TestAccAWSGlueCrawler_TablePrefix
=== CONT  TestAccAWSGlueCrawler_Role_ARN_Path
2020/11/04 10:59:05 [DEBUG] Trying to get account information via sts:GetCallerIdentity
2020/11/04 10:59:05 [DEBUG] Trying to get account information via sts:GetCallerIdentity
2020/11/04 10:59:11 [DEBUG] Trying to get account information via sts:GetCallerIdentity
2020/11/04 10:59:30 [DEBUG] Waiting for state to become: [success]
2020/11/04 10:59:37 [WARN] Truncating attribute path of 0 diagnostics for TypeSet
2020/11/04 10:59:37 [WARN] Truncating attribute path of 0 diagnostics for TypeSet
2020/11/04 10:59:39 [DEBUG] Trying to get account information via sts:GetCallerIdentity
--- PASS: TestAccAWSGlueCrawler_Role_ARN_Path (38.90s)
=== CONT  TestAccAWSGlueCrawler_mongoDBTarget
--- PASS: TestAccAWSGlueCrawler_disappears (39.11s)
=== CONT  TestAccAWSGlueCrawler_mongoDBTarget_scan_all
--- PASS: TestAccAWSGlueCrawler_S3Target_ConnectionName (41.48s)
=== CONT  TestAccAWSGlueCrawler_DynamodbTarget_scanRate
2020/11/04 10:59:52 [TRACE] Waiting 500ms before next try
--- PASS: TestAccAWSGlueCrawler_Role_Name_Path (54.00s)
=== CONT  TestAccAWSGlueCrawler_JdbcTarget_Exclusions
--- PASS: TestAccAWSGlueCrawler_Role_ARN_NoPath (54.55s)
=== CONT  TestAccAWSGlueCrawler_Classifiers
2020/11/04 10:59:58 [DEBUG] Waiting for state to become: [success]
2020/11/04 11:00:04 [WARN] Truncating attribute path of 0 diagnostics for TypeSet
2020/11/04 11:00:04 [WARN] Truncating attribute path of 0 diagnostics for TypeSet
2020/11/04 11:00:04 [WARN] Truncating attribute path of 0 diagnostics for TypeSet
--- PASS: TestAccAWSGlueCrawler_S3Target (62.72s)
=== CONT  TestAccAWSGlueCrawler_JdbcTarget_Multiple
--- PASS: TestAccAWSGlueCrawler_SchemaChangePolicy (63.17s)
=== CONT  TestAccAWSGlueCrawler_DynamodbTarget_scanAll
--- PASS: TestAccAWSGlueCrawler_TablePrefix (63.22s)
=== CONT  TestAccAWSGlueCrawler_JdbcTarget
2020/11/04 11:00:07 [INFO] AWS Auth provider used: "SharedCredentialsProvider"
--- PASS: TestAccAWSGlueCrawler_S3Target_Exclusions (66.39s)
--- PASS: TestAccAWSGlueCrawler_DynamodbTarget (69.29s)
--- PASS: TestAccAWSGlueCrawler_CatalogTarget (69.38s)
2020/11/04 11:00:13 [DEBUG] Trying to get account information via sts:GetCallerIdentity
--- PASS: TestAccAWSGlueCrawler_Description (72.06s)
2020/11/04 11:00:18 [DEBUG] Getting IAM Policy Version: {
  PolicyArn: "arn:aws:iam::aws:policy/service-role/AWSGlueServiceRole",
  VersionId: "v4"
}
2020/11/04 11:00:18 [DEBUG] Waiting for state to become: [success]
--- PASS: TestAccAWSGlueCrawler_SecurityConfiguration (76.73s)
--- PASS: TestAccAWSGlueCrawler_Configuration (76.90s)
--- PASS: TestAccAWSGlueCrawler_Schedule (81.17s)
--- PASS: TestAccAWSGlueCrawler_RemoveTablePrefix (82.14s)
--- PASS: TestAccAWSGlueCrawler_mongoDBTarget_multiple (83.20s)
--- PASS: TestAccAWSGlueCrawler_S3Target_Multiple (85.18s)
2020/11/04 11:00:29 [DEBUG] deleting Glue crawler: tf-acc-test-2019825100332017867
2020/11/04 11:00:31 [DEBUG] Getting IAM Policy: {
  PolicyArn: "arn:aws:iam::aws:policy/service-role/AWSGlueServiceRole"
}
2020/11/04 11:00:31 [DEBUG] Waiting for state to become: [success]
--- PASS: TestAccAWSGlueCrawler_mongoDBTarget (52.80s)
2020/11/04 11:00:35 [DEBUG] Trying to get account information via sts:GetCallerIdentity
--- PASS: TestAccAWSGlueCrawler_Tags (92.87s)
2020/11/04 11:00:40 [DEBUG] Trying to get account information via sts:GetCallerIdentity
--- PASS: TestAccAWSGlueCrawler_CatalogTarget_Multiple (104.15s)
--- PASS: TestAccAWSGlueCrawler_JdbcTarget_Exclusions (53.95s)
--- PASS: TestAccAWSGlueCrawler_mongoDBTarget_scan_all (72.81s)
--- PASS: TestAccAWSGlueCrawler_DynamodbTarget_scanRate (78.70s)
--- PASS: TestAccAWSGlueCrawler_JdbcTarget (59.44s)
--- PASS: TestAccAWSGlueCrawler_Classifiers (70.64s)
--- PASS: TestAccAWSGlueCrawler_JdbcTarget_Multiple (67.80s)
--- PASS: TestAccAWSGlueCrawler_DynamodbTarget_scanAll (67.45s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	132.210s

Verified Acceptance Tests in GovCloud (us-gov-west-1)

make testacc TEST=./aws/ TESTARGS='-run=TestAccAWSGlueCrawler_'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSGlueCrawler_ -timeout 120m
=== RUN   TestAccAWSGlueCrawler_DynamodbTarget
=== PAUSE TestAccAWSGlueCrawler_DynamodbTarget
=== RUN   TestAccAWSGlueCrawler_DynamodbTarget_scanAll
=== PAUSE TestAccAWSGlueCrawler_DynamodbTarget_scanAll
=== RUN   TestAccAWSGlueCrawler_DynamodbTarget_scanRate
=== PAUSE TestAccAWSGlueCrawler_DynamodbTarget_scanRate
=== RUN   TestAccAWSGlueCrawler_JdbcTarget
=== PAUSE TestAccAWSGlueCrawler_JdbcTarget
=== RUN   TestAccAWSGlueCrawler_JdbcTarget_Exclusions
=== PAUSE TestAccAWSGlueCrawler_JdbcTarget_Exclusions
=== RUN   TestAccAWSGlueCrawler_JdbcTarget_Multiple
=== PAUSE TestAccAWSGlueCrawler_JdbcTarget_Multiple
=== RUN   TestAccAWSGlueCrawler_mongoDBTarget
=== PAUSE TestAccAWSGlueCrawler_mongoDBTarget
=== RUN   TestAccAWSGlueCrawler_mongoDBTarget_scan_all
=== PAUSE TestAccAWSGlueCrawler_mongoDBTarget_scan_all
=== RUN   TestAccAWSGlueCrawler_mongoDBTarget_multiple
=== PAUSE TestAccAWSGlueCrawler_mongoDBTarget_multiple
=== RUN   TestAccAWSGlueCrawler_S3Target
=== PAUSE TestAccAWSGlueCrawler_S3Target
=== RUN   TestAccAWSGlueCrawler_S3Target_ConnectionName
=== PAUSE TestAccAWSGlueCrawler_S3Target_ConnectionName
=== RUN   TestAccAWSGlueCrawler_S3Target_Exclusions
=== PAUSE TestAccAWSGlueCrawler_S3Target_Exclusions
=== RUN   TestAccAWSGlueCrawler_S3Target_Multiple
=== PAUSE TestAccAWSGlueCrawler_S3Target_Multiple
=== RUN   TestAccAWSGlueCrawler_CatalogTarget
=== PAUSE TestAccAWSGlueCrawler_CatalogTarget
=== RUN   TestAccAWSGlueCrawler_CatalogTarget_Multiple
=== PAUSE TestAccAWSGlueCrawler_CatalogTarget_Multiple
=== RUN   TestAccAWSGlueCrawler_disappears
=== PAUSE TestAccAWSGlueCrawler_disappears
=== RUN   TestAccAWSGlueCrawler_Classifiers
=== PAUSE TestAccAWSGlueCrawler_Classifiers
=== RUN   TestAccAWSGlueCrawler_Configuration
=== PAUSE TestAccAWSGlueCrawler_Configuration
=== RUN   TestAccAWSGlueCrawler_Description
=== PAUSE TestAccAWSGlueCrawler_Description
=== RUN   TestAccAWSGlueCrawler_Role_ARN_NoPath
=== PAUSE TestAccAWSGlueCrawler_Role_ARN_NoPath
=== RUN   TestAccAWSGlueCrawler_Role_ARN_Path
=== PAUSE TestAccAWSGlueCrawler_Role_ARN_Path
=== RUN   TestAccAWSGlueCrawler_Role_Name_Path
=== PAUSE TestAccAWSGlueCrawler_Role_Name_Path
=== RUN   TestAccAWSGlueCrawler_Schedule
=== PAUSE TestAccAWSGlueCrawler_Schedule
=== RUN   TestAccAWSGlueCrawler_SchemaChangePolicy
=== PAUSE TestAccAWSGlueCrawler_SchemaChangePolicy
=== RUN   TestAccAWSGlueCrawler_TablePrefix
=== PAUSE TestAccAWSGlueCrawler_TablePrefix
=== RUN   TestAccAWSGlueCrawler_RemoveTablePrefix
=== PAUSE TestAccAWSGlueCrawler_RemoveTablePrefix
=== RUN   TestAccAWSGlueCrawler_Tags
=== PAUSE TestAccAWSGlueCrawler_Tags
=== RUN   TestAccAWSGlueCrawler_SecurityConfiguration
=== PAUSE TestAccAWSGlueCrawler_SecurityConfiguration
=== CONT  TestAccAWSGlueCrawler_DynamodbTarget
=== CONT  TestAccAWSGlueCrawler_disappears
=== CONT  TestAccAWSGlueCrawler_Tags
=== CONT  TestAccAWSGlueCrawler_mongoDBTarget_multiple
=== CONT  TestAccAWSGlueCrawler_CatalogTarget_Multiple
=== CONT  TestAccAWSGlueCrawler_CatalogTarget
=== CONT  TestAccAWSGlueCrawler_S3Target_Multiple
=== CONT  TestAccAWSGlueCrawler_S3Target_Exclusions
=== CONT  TestAccAWSGlueCrawler_S3Target_ConnectionName
=== CONT  TestAccAWSGlueCrawler_Schedule
=== CONT  TestAccAWSGlueCrawler_S3Target
=== CONT  TestAccAWSGlueCrawler_RemoveTablePrefix
=== CONT  TestAccAWSGlueCrawler_SecurityConfiguration
=== CONT  TestAccAWSGlueCrawler_JdbcTarget_Exclusions
=== CONT  TestAccAWSGlueCrawler_mongoDBTarget_scan_all
=== CONT  TestAccAWSGlueCrawler_mongoDBTarget
=== CONT  TestAccAWSGlueCrawler_Role_Name_Path
=== CONT  TestAccAWSGlueCrawler_Configuration
=== CONT  TestAccAWSGlueCrawler_TablePrefix
=== CONT  TestAccAWSGlueCrawler_JdbcTarget_Multiple
2020/11/04 11:01:50 [DEBUG] Trying to get account information via sts:GetCallerIdentity
2020/11/04 11:01:50 [DEBUG] Trying to get account information via sts:GetCallerIdentity
2020/11/04 11:01:51 [DEBUG] Trying to get account information via sts:GetCallerIdentity
2020/11/04 11:02:13 [DEBUG] Getting IAM Policy: {
  PolicyArn: "arn:aws-us-gov:iam::aws:policy/service-role/AWSGlueServiceRole"
}
2020/11/04 11:02:13 [DEBUG] Waiting for state to become: [success]
2020/11/04 11:02:23 [DEBUG] Waiting for state to become: [success]
--- PASS: TestAccAWSGlueCrawler_Role_Name_Path (36.15s)
=== CONT  TestAccAWSGlueCrawler_Classifiers
--- PASS: TestAccAWSGlueCrawler_S3Target_ConnectionName (36.94s)
=== CONT  TestAccAWSGlueCrawler_SchemaChangePolicy
2020/11/04 11:02:28 [DEBUG] Waiting for state to become: [success]
2020/11/04 11:02:29 [DEBUG] Trying to get account information via sts:GetCallerIdentity
2020/11/04 11:02:29 [DEBUG] Trying to get account information via sts:GetCallerIdentity
2020/11/04 11:02:30 [WARN] Glue Crawler (tf-acc-test-3623191015994565968) not found, removing from state
--- PASS: TestAccAWSGlueCrawler_disappears (46.15s)
=== CONT  TestAccAWSGlueCrawler_Role_ARN_Path
2020/11/04 11:02:37 [DEBUG] Trying to get account information via sts:GetCallerIdentity
--- PASS: TestAccAWSGlueCrawler_SecurityConfiguration (52.20s)
=== CONT  TestAccAWSGlueCrawler_DynamodbTarget_scanRate
--- PASS: TestAccAWSGlueCrawler_Configuration (54.21s)
=== CONT  TestAccAWSGlueCrawler_DynamodbTarget_scanAll
--- PASS: TestAccAWSGlueCrawler_CatalogTarget (56.60s)
=== CONT  TestAccAWSGlueCrawler_JdbcTarget
--- PASS: TestAccAWSGlueCrawler_S3Target_Exclusions (56.66s)
=== CONT  TestAccAWSGlueCrawler_Role_ARN_NoPath
--- PASS: TestAccAWSGlueCrawler_S3Target (57.54s)
=== CONT  TestAccAWSGlueCrawler_Description
2020/11/04 11:02:48 [DEBUG] Waiting for state to become: [success]
--- PASS: TestAccAWSGlueCrawler_JdbcTarget_Exclusions (58.22s)
2020/11/04 11:02:49 [DEBUG] Trying to get account information via sts:GetCallerIdentity
--- PASS: TestAccAWSGlueCrawler_Tags (61.65s)
--- PASS: TestAccAWSGlueCrawler_mongoDBTarget_scan_all (72.62s)
--- PASS: TestAccAWSGlueCrawler_RemoveTablePrefix (74.69s)
--- PASS: TestAccAWSGlueCrawler_JdbcTarget_Multiple (78.05s)
--- PASS: TestAccAWSGlueCrawler_mongoDBTarget (78.11s)
--- PASS: TestAccAWSGlueCrawler_mongoDBTarget_multiple (78.89s)
2020/11/04 11:03:12 [INFO] AWS Auth provider used: "SharedCredentialsProvider"
--- PASS: TestAccAWSGlueCrawler_Role_ARN_Path (37.62s)
--- PASS: TestAccAWSGlueCrawler_SchemaChangePolicy (46.83s)
--- PASS: TestAccAWSGlueCrawler_Role_ARN_NoPath (29.35s)
--- PASS: TestAccAWSGlueCrawler_Schedule (88.72s)
--- PASS: TestAccAWSGlueCrawler_Classifiers (57.26s)
2020/11/04 11:03:26 [INFO] AWS Auth provider used: "SharedCredentialsProvider"
2020/11/04 11:03:26 [DEBUG] Trying to get account information via sts:GetCallerIdentity
--- PASS: TestAccAWSGlueCrawler_JdbcTarget (42.54s)
--- PASS: TestAccAWSGlueCrawler_CatalogTarget_Multiple (99.15s)
--- PASS: TestAccAWSGlueCrawler_Description (44.02s)
--- PASS: TestAccAWSGlueCrawler_TablePrefix (103.13s)
--- PASS: TestAccAWSGlueCrawler_S3Target_Multiple (106.34s)
--- PASS: TestAccAWSGlueCrawler_DynamodbTarget_scanAll (53.41s)
--- PASS: TestAccAWSGlueCrawler_DynamodbTarget_scanRate (56.57s)
--- PASS: TestAccAWSGlueCrawler_DynamodbTarget (130.99s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	132.449s

@breathingdust breathingdust added this to the v3.14.0 milestone Nov 4, 2020
@breathingdust breathingdust merged commit ed409e5 into hashicorp:master Nov 4, 2020
breathingdust added a commit that referenced this pull request Nov 4, 2020
@breathingdust
Update CHANGELOG.md for #15934
f595c6d
@ghost
Copy link

ghost commented Nov 6, 2020

This has been released in version 3.14.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

@ghost
Copy link

ghost commented Dec 5, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

@ghost ghost locked as resolved and limited conversation to collaborators Dec 5, 2020
@DrFaust92 DrFaust92 deleted the r/glue_crawler_mongodb_target branch April 15, 2021 10:39
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@ewbankkit ewbankkit ewbankkit approved these changes

Assignees
No one assigned
Labels
documentation Introduces or discusses updates to documentation. enhancement Requests to existing resources that expand the functionality or scope. service/glue Issues and PRs that pertain to the glue service. size/XL Managed by automation to categorize the size of a PR. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure.
Projects
None yet
Milestone
v3.14.0
Development

Successfully merging this pull request may close these issues.
3 participants
@DrFaust92 @breathingdust @ewbankkit