aws_ssm: ability to customize s3 endpoint for vpc interface endpoint #1619
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Docs Build 📝Thank you for contribution!✨ This PR has been merged and your docs changes will be incorporated when they are next published. |
ansibullbot
added
community_review
connection
markuman
reviewed
Dec 13, 2022
|
Hi, did I need to change something or is the PR just waiting for a re-review? :) |
|
recheck |
markuman
reviewed
Jan 12, 2023
...ragments/1619-add-s3-bucket-endpoint-url-var-for-private-network-vpc-interface-endpoints.yml
Outdated
Show resolved
Hide resolved
markuman
changed the title
tremble
reviewed
Jan 13, 2023
|
recheck |
Co-authored-by: Markus Bergholz <[email protected]>
…rivate-network-vpc-interface-endpoints.yml Co-authored-by: Markus Bergholz <[email protected]>
tremble
force-pushed
the
feature/s3_custom_endpoint_url_for_private_vpc_endpoints
branch
from
30ab080 to
4b9c102
Compare
|
@markuman I've rebased this and added some integration tests. Could you cast a quick eye over this one please. |
markuman
approved these changes
Jan 31, 2023
|
regate |
Backport to stable-5: 💚 backport PR created✅ Backport PR branch: Backported as #1684 🤖 @patchback |
patchback bot
pushed a commit
that referenced
this pull request
Jan 31, 2023
…1619) aws_ssm: ability to customize s3 endpoint for vpc interface endpoint Depends-On: ansible/ansible-zuul-jobs#1743 SUMMARY Add a new variable for setting the s3 endpoint url ISSUE TYPE Feature Pull Request COMPONENT NAME connection aws_ssm.py ADDITIONAL INFORMATION If you try to running SSM commands on EC2 instances in private networks only with vpc interface endpoints. You are not able to access S3 service because the generated URL is wrong. For now this plugin only works for s3 vpc gateway endpoints. Not for s3 vpc interface endpoints. To simply fix this. We need the possibility to set the interface endpoint url. So I added a new parameter to the connection plugin. How to test - name: test ssm on an EC2 instance hosts: router vars: ansible_python_interpreter: /usr/bin/python3 ansible_connection: aws_ssm ansible_aws_ssm_region: 'eu-west-1' ansible_aws_ssm_bucket_name: testbucket ansible_aws_ssm_bucket_endpoint_url: "https://vpce-00000000000000-10ygvtbr.s3.eu-west-1.vpce.amazonaws.com" tasks: - name: list files in opt folder shell: echo "running on $(curl -s http://169.254.169.254/latest/meta-data/instance-id)" Output before change XEC curl 'https://testbucket.s3.amazonaws.com/i-0b5e11951ab6b12cd//home/ssm-user/.ansible/tmp/ansible-tmp-1610187779.9946203-6452-52757228453165/AnsiballZ_setup.py?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIAZEY6F5MCGGCDZQGI%2F20210109%2F**eu-central-1**%2Fs3%2Faws4_request&X-Amz-Date=20210109T102300Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEKr%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaDGV1LWNlbnRyYWwtMSJIMEYCIQDRoCbT6dS9geijC00Xhr4nKdDrfKSE0ULsEXNgjM3vUwIhAMoAiDJJSGehMKvcmUlZDHc17WcV3Wnw4lsCED4MH%2BkMKroDCHMQABoMNjI4NzM5MTQwMzU2IgylHaq9VXBqdPex8fsqlwMzbTC5nczwsbUzXkpdw1MWndywQnjxp%2BnZoYcHMba6TGM57osVwt6hQoYxKA04co63FOr%2FtvhmmLGdphxeEGBPRjyTCNB%2Bdtr%2BwfKmjyls7WmBQF4jRMm2xPMUSd3EBnitCOpRvHPtp4xsuIX59QKCZmUNKBYIn2USx18mcSrWpI1emQGkmgewn9EOxUT168X9unNnvmUerokKgD5f1dZvpnIEmUyPYhYFCkJAdmLa5E5CIWe4UFfULLwDwTqYe6akqSAhBUeMrzWvebp7oXkER%2BymsmdGdAl4nFKNDtJ5suSkcGooliKsFhrHKEb1gN4UH%2FldPSFZqCEOayiWByk6SK7yEkhqI7wbc5Ufwv68AimpRddA5dU95kXUL3tgBYq5QcSeXStdd%2B6nQ3vRDBJx%2BETvR2dGOeZv%2Bu6p1iLaT5wnMgMcSnPQWCTja%2Bnf7Lp%2Bkmd4pR9yfTYaPa%2FVdblsVAXtfDURQ7wHwV6DJJavt26oUXNOOjEXg4FDraLzGSNWGFjMkbxLSBFNEyKBB9g3Hk8hV4YOwjC4%2BeX%2FBTrqAfEXoF92NloBKePOvKXzFcp8YT8yC0p35rXYqa0GA5d9ZNaGewFw6ks9VMUTSht3SZ2ns2qCYF6p73ISe88pgrUWGwFaxZnbNxP1dvfpNH3X9zQ9oVyjKfD9dwPfnOYpx6j48dZZhdgZ6n2H13h3Ckf7hmebHo7po%2BWrXkc8K1Bo07YSFyFMffieXBk0NvrBPGNGtKTEJ3m%2FfF4vkM4lnEN2xWaS0umgwMQrCqfhKD3Gpf%2BdglVJ2oBRHb3ho7dfie48ohpAd%2B6j752PjR2SuaA3Gokns8scHLBB7dvkGlmqb3vrX8TYsc7CZg%3D%3D&X-Amz-Signature=a992947e6682f66239d42c08a6b8ba2da136a572bc15a2184763846fd5a39504' -o '/home/ssm-user/.ansible/tmp/ansible-tmp-1610187779.9946203-6452-52757228453165/AnsiballZ_setup.py' Output after change XEC curl 'https://testbucket.vpce-00000000000000-10ygvtbr.s3.eu-west-1.vpce.amazonaws.com/i-0b5e11951ab6b12cd//home/ssm-user/.ansible/tmp/ansible-tmp-1610187779.9946203-6452-52757228453165/AnsiballZ_setup.py?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIAZEY6F5MCGGCDZQGI%2F20210109%2F**eu-central-1**%2Fs3%2Faws4_request&X-Amz-Date=20210109T102300Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEKr%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaDGV1LWNlbnRyYWwtMSJIMEYCIQDRoCbT6dS9geijC00Xhr4nKdDrfKSE0ULsEXNgjM3vUwIhAMoAiDJJSGehMKvcmUlZDHc17WcV3Wnw4lsCED4MH%2BkMKroDCHMQABoMNjI4NzM5MTQwMzU2IgylHaq9VXBqdPex8fsqlwMzbTC5nczwsbUzXkpdw1MWndywQnjxp%2BnZoYcHMba6TGM57osVwt6hQoYxKA04co63FOr%2FtvhmmLGdphxeEGBPRjyTCNB%2Bdtr%2BwfKmjyls7WmBQF4jRMm2xPMUSd3EBnitCOpRvHPtp4xsuIX59QKCZmUNKBYIn2USx18mcSrWpI1emQGkmgewn9EOxUT168X9unNnvmUerokKgD5f1dZvpnIEmUyPYhYFCkJAdmLa5E5CIWe4UFfULLwDwTqYe6akqSAhBUeMrzWvebp7oXkER%2BymsmdGdAl4nFKNDtJ5suSkcGooliKsFhrHKEb1gN4UH%2FldPSFZqCEOayiWByk6SK7yEkhqI7wbc5Ufwv68AimpRddA5dU95kXUL3tgBYq5QcSeXStdd%2B6nQ3vRDBJx%2BETvR2dGOeZv%2Bu6p1iLaT5wnMgMcSnPQWCTja%2Bnf7Lp%2Bkmd4pR9yfTYaPa%2FVdblsVAXtfDURQ7wHwV6DJJavt26oUXNOOjEXg4FDraLzGSNWGFjMkbxLSBFNEyKBB9g3Hk8hV4YOwjC4%2BeX%2FBTrqAfEXoF92NloBKePOvKXzFcp8YT8yC0p35rXYqa0GA5d9ZNaGewFw6ks9VMUTSht3SZ2ns2qCYF6p73ISe88pgrUWGwFaxZnbNxP1dvfpNH3X9zQ9oVyjKfD9dwPfnOYpx6j48dZZhdgZ6n2H13h3Ckf7hmebHo7po%2BWrXkc8K1Bo07YSFyFMffieXBk0NvrBPGNGtKTEJ3m%2FfF4vkM4lnEN2xWaS0umgwMQrCqfhKD3Gpf%2BdglVJ2oBRHb3ho7dfie48ohpAd%2B6j752PjR2SuaA3Gokns8scHLBB7dvkGlmqb3vrX8TYsc7CZg%3D%3D&X-Amz-Signature=a992947e6682f66239d42c08a6b8ba2da136a572bc15a2184763846fd5a39504' -o '/home/ssm-user/.ansible/tmp/ansible-tmp-1610187779.9946203-6452-52757228453165/AnsiballZ_setup.py' Reviewed-by: Markus Bergholz <[email protected]> Reviewed-by: Mark Chappell <None> (cherry picked from commit 433f6c5)
softwarefactory-project-zuul bot
pushed a commit
that referenced
this pull request
Jan 31, 2023
…1619) (#1684) [PR #1619/433f6c5e backport][stable-5] aws_ssm: ability to customize s3 endpoint for vpc interface endpoint This is a backport of PR #1619 as merged into main (433f6c5). Depends-On: ansible/ansible-zuul-jobs#1743 SUMMARY Add a new variable for setting the s3 endpoint url ISSUE TYPE Feature Pull Request COMPONENT NAME connection aws_ssm.py ADDITIONAL INFORMATION If you try to running SSM commands on EC2 instances in private networks only with vpc interface endpoints. You are not able to access S3 service because the generated URL is wrong. For now this plugin only works for s3 vpc gateway endpoints. Not for s3 vpc interface endpoints. To simply fix this. We need the possibility to set the interface endpoint url. So I added a new parameter to the connection plugin. How to test - name: test ssm on an EC2 instance hosts: router vars: ansible_python_interpreter: /usr/bin/python3 ansible_connection: aws_ssm ansible_aws_ssm_region: 'eu-west-1' ansible_aws_ssm_bucket_name: testbucket ansible_aws_ssm_bucket_endpoint_url: "https://vpce-00000000000000-10ygvtbr.s3.eu-west-1.vpce.amazonaws.com" tasks: - name: list files in opt folder shell: echo "running on $(curl -s http://169.254.169.254/latest/meta-data/instance-id)" Output before change XEC curl 'https://testbucket.s3.amazonaws.com/i-0b5e11951ab6b12cd//home/ssm-user/.ansible/tmp/ansible-tmp-1610187779.9946203-6452-52757228453165/AnsiballZ_setup.py?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIAZEY6F5MCGGCDZQGI%2F20210109%2F**eu-central-1**%2Fs3%2Faws4_request&X-Amz-Date=20210109T102300Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEKr%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaDGV1LWNlbnRyYWwtMSJIMEYCIQDRoCbT6dS9geijC00Xhr4nKdDrfKSE0ULsEXNgjM3vUwIhAMoAiDJJSGehMKvcmUlZDHc17WcV3Wnw4lsCED4MH%2BkMKroDCHMQABoMNjI4NzM5MTQwMzU2IgylHaq9VXBqdPex8fsqlwMzbTC5nczwsbUzXkpdw1MWndywQnjxp%2BnZoYcHMba6TGM57osVwt6hQoYxKA04co63FOr%2FtvhmmLGdphxeEGBPRjyTCNB%2Bdtr%2BwfKmjyls7WmBQF4jRMm2xPMUSd3EBnitCOpRvHPtp4xsuIX59QKCZmUNKBYIn2USx18mcSrWpI1emQGkmgewn9EOxUT168X9unNnvmUerokKgD5f1dZvpnIEmUyPYhYFCkJAdmLa5E5CIWe4UFfULLwDwTqYe6akqSAhBUeMrzWvebp7oXkER%2BymsmdGdAl4nFKNDtJ5suSkcGooliKsFhrHKEb1gN4UH%2FldPSFZqCEOayiWByk6SK7yEkhqI7wbc5Ufwv68AimpRddA5dU95kXUL3tgBYq5QcSeXStdd%2B6nQ3vRDBJx%2BETvR2dGOeZv%2Bu6p1iLaT5wnMgMcSnPQWCTja%2Bnf7Lp%2Bkmd4pR9yfTYaPa%2FVdblsVAXtfDURQ7wHwV6DJJavt26oUXNOOjEXg4FDraLzGSNWGFjMkbxLSBFNEyKBB9g3Hk8hV4YOwjC4%2BeX%2FBTrqAfEXoF92NloBKePOvKXzFcp8YT8yC0p35rXYqa0GA5d9ZNaGewFw6ks9VMUTSht3SZ2ns2qCYF6p73ISe88pgrUWGwFaxZnbNxP1dvfpNH3X9zQ9oVyjKfD9dwPfnOYpx6j48dZZhdgZ6n2H13h3Ckf7hmebHo7po%2BWrXkc8K1Bo07YSFyFMffieXBk0NvrBPGNGtKTEJ3m%2FfF4vkM4lnEN2xWaS0umgwMQrCqfhKD3Gpf%2BdglVJ2oBRHb3ho7dfie48ohpAd%2B6j752PjR2SuaA3Gokns8scHLBB7dvkGlmqb3vrX8TYsc7CZg%3D%3D&X-Amz-Signature=a992947e6682f66239d42c08a6b8ba2da136a572bc15a2184763846fd5a39504' -o '/home/ssm-user/.ansible/tmp/ansible-tmp-1610187779.9946203-6452-52757228453165/AnsiballZ_setup.py' Output after change XEC curl 'https://testbucket.vpce-00000000000000-10ygvtbr.s3.eu-west-1.vpce.amazonaws.com/i-0b5e11951ab6b12cd//home/ssm-user/.ansible/tmp/ansible-tmp-1610187779.9946203-6452-52757228453165/AnsiballZ_setup.py?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIAZEY6F5MCGGCDZQGI%2F20210109%2F**eu-central-1**%2Fs3%2Faws4_request&X-Amz-Date=20210109T102300Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEKr%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaDGV1LWNlbnRyYWwtMSJIMEYCIQDRoCbT6dS9geijC00Xhr4nKdDrfKSE0ULsEXNgjM3vUwIhAMoAiDJJSGehMKvcmUlZDHc17WcV3Wnw4lsCED4MH%2BkMKroDCHMQABoMNjI4NzM5MTQwMzU2IgylHaq9VXBqdPex8fsqlwMzbTC5nczwsbUzXkpdw1MWndywQnjxp%2BnZoYcHMba6TGM57osVwt6hQoYxKA04co63FOr%2FtvhmmLGdphxeEGBPRjyTCNB%2Bdtr%2BwfKmjyls7WmBQF4jRMm2xPMUSd3EBnitCOpRvHPtp4xsuIX59QKCZmUNKBYIn2USx18mcSrWpI1emQGkmgewn9EOxUT168X9unNnvmUerokKgD5f1dZvpnIEmUyPYhYFCkJAdmLa5E5CIWe4UFfULLwDwTqYe6akqSAhBUeMrzWvebp7oXkER%2BymsmdGdAl4nFKNDtJ5suSkcGooliKsFhrHKEb1gN4UH%2FldPSFZqCEOayiWByk6SK7yEkhqI7wbc5Ufwv68AimpRddA5dU95kXUL3tgBYq5QcSeXStdd%2B6nQ3vRDBJx%2BETvR2dGOeZv%2Bu6p1iLaT5wnMgMcSnPQWCTja%2Bnf7Lp%2Bkmd4pR9yfTYaPa%2FVdblsVAXtfDURQ7wHwV6DJJavt26oUXNOOjEXg4FDraLzGSNWGFjMkbxLSBFNEyKBB9g3Hk8hV4YOwjC4%2BeX%2FBTrqAfEXoF92NloBKePOvKXzFcp8YT8yC0p35rXYqa0GA5d9ZNaGewFw6ks9VMUTSht3SZ2ns2qCYF6p73ISe88pgrUWGwFaxZnbNxP1dvfpNH3X9zQ9oVyjKfD9dwPfnOYpx6j48dZZhdgZ6n2H13h3Ckf7hmebHo7po%2BWrXkc8K1Bo07YSFyFMffieXBk0NvrBPGNGtKTEJ3m%2FfF4vkM4lnEN2xWaS0umgwMQrCqfhKD3Gpf%2BdglVJ2oBRHb3ho7dfie48ohpAd%2B6j752PjR2SuaA3Gokns8scHLBB7dvkGlmqb3vrX8TYsc7CZg%3D%3D&X-Amz-Signature=a992947e6682f66239d42c08a6b8ba2da136a572bc15a2184763846fd5a39504' -o '/home/ssm-user/.ansible/tmp/ansible-tmp-1610187779.9946203-6452-52757228453165/AnsiballZ_setup.py' Reviewed-by: Mark Chappell <None>
very n1. Thank you very much for the support. |
lubbyhst
deleted the
feature/s3_custom_endpoint_url_for_private_vpc_endpoints
branch
|
@lubbyhst, thanks for taking the time to open the PR. Sorry it took a while, the aws_ssm connection integration tests had been broken for a while and I really didn't want to merge new features without fixing the tests. |
abikouo
pushed a commit
to abikouo/community.aws
that referenced
this pull request
Oct 24, 2023
Refactor ARN validation code SUMMARY Adds resource_id and resource_type to parse_aws_arn() return value. Adds validate_aws_arn() to handle common pattern matching for ARNs. ISSUE TYPE Feature Pull Request COMPONENT NAME ec2_instance iam_user ADDITIONAL INFORMATION Related to ansible-collections#1846 - We've been doing things like assuming the aws partition. Reviewed-by: Alina Buzachis
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Depends-On: ansible/ansible-zuul-jobs#1743
SUMMARY
Add a new variable for setting the s3 endpoint url
ISSUE TYPE
COMPONENT NAME
connection aws_ssm.py
ADDITIONAL INFORMATION
If you try to running SSM commands on EC2 instances in private networks only with vpc interface endpoints. You are not able to access S3 service because the generated URL is wrong. For now this plugin only works for s3 vpc gateway endpoints. Not for s3 vpc interface endpoints.
To simply fix this. We need the possibility to set the interface endpoint url. So I added a new parameter to the connection plugin.
How to test
Output before change
Output after change