-
Notifications
You must be signed in to change notification settings - Fork 398
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
aws_glue_connection: Add multiple connection types, add check mode #503
aws_glue_connection: Add multiple connection types, add check mode #503
Conversation
Examples: ``` - community.aws.aws_glue_connection: name: My connection availability_zone: us-east-1a connection_properties: JDBC_ENFORCE_SSL: "false" connection_type: NETWORK description: My test connection security_groups: - test subnet_id: subnet-123abc state: present ```
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for taking the time to submit this PR.
Rather than replacing the existing tasks in the integration tests it would be better if the new tasks were added to the tests. This helps to ensure that old playbooks should continue working.
I'd recommend using the jittered backoff rather than a plain backoff: https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/
Additionally, we have some magic in AnsibleAWSModule.client() which decorates the client calls. This means you can retry just the API call by adding aws_retry=True
to the call rather than needing to retry all of the code around it. See for example https://github.com/ansible-collections/community.aws/pull/421/files
retry_decorator = AWSRetry.jittered_backoff(retries=10)
connection = module.client('ec2', retry_decorator=retry_decorator)
...
response = connection.describe_egress_only_internet_gateways(aws_retry=True)
It's not as important for this module because it's mostly making one call per function, but if (for example) you made the describe call directly inside the create_or_update_glue_connection function, and the describe call failed due to limits, it would be possible for the change to be made during the first run, the describe fails, and then when it re-runs the whole function there's nothing to change, so it'll result in changed=False when a change has happened.
Improve retries - use retries with jittered backoff: https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/
Hi @ichekaldin, Thanks for the updates. I'm still a little concerned that the original tests have been completely overwritten. One of the reasons for integration tests is to ensure that playbooks which used to work continue to work. One option that some people use is splitting the tests into multiple task-files and then "including" them. This means you can just add a new set of tasks to test a new feature. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Once the tests pass I'll get this merged.
Thanks for taking the time to raise this PR.
This is mostly to re-trigger the tests.
…nsible-collections#503) * Add multiple connection types and support for check mode Examples: ``` - community.aws.aws_glue_connection: name: My connection availability_zone: us-east-1a connection_properties: JDBC_ENFORCE_SSL: "false" connection_type: NETWORK description: My test connection security_groups: - test subnet_id: subnet-123abc state: present ``` * Add retries. * Add description of how to create a Glue network connection Co-authored-by: Mark Chappell <[email protected]>
…nsible-collections#503) * Add multiple connection types and support for check mode Examples: ``` - community.aws.aws_glue_connection: name: My connection availability_zone: us-east-1a connection_properties: JDBC_ENFORCE_SSL: "false" connection_type: NETWORK description: My test connection security_groups: - test subnet_id: subnet-123abc state: present ``` * Add retries. * Add description of how to create a Glue network connection Co-authored-by: Mark Chappell <[email protected]>
…nsible-collections#503) * Add multiple connection types and support for check mode Examples: ``` - community.aws.aws_glue_connection: name: My connection availability_zone: us-east-1a connection_properties: JDBC_ENFORCE_SSL: "false" connection_type: NETWORK description: My test connection security_groups: - test subnet_id: subnet-123abc state: present ``` * Add retries. * Add description of how to create a Glue network connection Co-authored-by: Mark Chappell <[email protected]>
…case (#518) glue_connection - Avoid converting connection_parameter keys to lowercase SUMMARY This is a follow to my own PR #503. This is a cosmetic change that prevents converting keys in connection_parameters dict to lowercase. ISSUE TYPE Bugfix Pull Request COMPONENT NAME aws_glue_connection ADDITIONAL INFORMATION As an example, this: - community.aws.glue_connection: name: test-connection connection_parameters: JDBC_ENFORCE_SSL: "false" ... is a valid value, while this: - community.aws.glue_connection: name: test-connection connection_parameters: jdbc_enforce_ssl: "false" ... is not. This PR simply aligns the module output to the expected input. Reviewed-by: Mark Chappell <None>
…case (#518) glue_connection - Avoid converting connection_parameter keys to lowercase SUMMARY This is a follow to my own PR #503. This is a cosmetic change that prevents converting keys in connection_parameters dict to lowercase. ISSUE TYPE Bugfix Pull Request COMPONENT NAME aws_glue_connection ADDITIONAL INFORMATION As an example, this: - community.aws.glue_connection: name: test-connection connection_parameters: JDBC_ENFORCE_SSL: "false" ... is a valid value, while this: - community.aws.glue_connection: name: test-connection connection_parameters: jdbc_enforce_ssl: "false" ... is not. This PR simply aligns the module output to the expected input. Reviewed-by: Mark Chappell <None> (cherry picked from commit b03919d)
…case (#518) (#1321) [PR #518/b03919db backport][stable-4] glue_connection - Avoid converting connection_parameter keys to lowercase This is a backport of PR #518 as merged into main (b03919d). SUMMARY This is a follow to my own PR #503. This is a cosmetic change that prevents converting keys in connection_parameters dict to lowercase. ISSUE TYPE Bugfix Pull Request COMPONENT NAME aws_glue_connection ADDITIONAL INFORMATION As an example, this: - community.aws.glue_connection: name: test-connection connection_parameters: JDBC_ENFORCE_SSL: "false" ... is a valid value, while this: - community.aws.glue_connection: name: test-connection connection_parameters: jdbc_enforce_ssl: "false" ... is not. This PR simply aligns the module output to the expected input. Reviewed-by: Mark Chappell <None>
SUMMARY
Add multiple connection types to aws_glue_connection, add support for check mode.
ISSUE TYPE
COMPONENT NAME
aws_glue_connection
ADDITIONAL INFORMATION
Example: