-
Notifications
You must be signed in to change notification settings - Fork 42
feat: enable SSH access to users for debugging cloud instances #2001
Conversation
It will allow importing public SSH keys from Github so we can easily provide a way for developers to connect to the cloud instances. See https://github.com/dustinkirkland/ssh-import-id
quietPeriod(10) | ||
} | ||
triggers { | ||
cron '0 0 * * 0' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should run this daily, removing the DEVELOPER_MODE logic and never destroy the stack and runners.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure, though what if they are 50 PR runs in a day? We may run into resource restrictions and tests would fail
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IIRC, Travis had a nice feature you were able to connect to the machines but only for 30 minutes
💔 Tests Failed
Expand to view the summary
Build stats
Test stats 🧪
Test errorsExpand to view the tests failures
|
This way we will be able to correlate the cloud instances with a commit SHA instead of a random UUID
Provisioning from local machine will lead to buildURL not to be populated, so we need a default value
This pull request is now in conflict. Could you fix it @mdelapenya? 🙏
|
@@ -93,6 +94,7 @@ pipeline { | |||
githubCheckNotify('PENDING') // we want to notify the upstream about the e2e the soonest | |||
stash allowEmpty: true, name: 'source', useDefaultExcludes: false | |||
setEnvVar("GO_VERSION", readFile("${env.WORKSPACE}/${env.BASE_DIR}/.go-version").trim()) | |||
setEnvVar("LABELS_STRING", "buildURL=${env.BUILD_URL} gitSha=${env.GIT_BASE_COMMIT}") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Define labels only once
/test |
@adam-stokes I think these changes are correct: the machines are receiving the desired tags, and they are kept after the pipeline finishes. We may need to iterate through the reaper pipeline in the case is not working properly, but we can merge this one and open a new ticket for those potential issues. |
@Mergifyio refresh |
…ic#2001) * chore: install ssh-import-id from pip It will allow importing public SSH keys from Github so we can easily provide a way for developers to connect to the cloud instances. See https://github.com/dustinkirkland/ssh-import-id * feat: do not destroy cloud resources if DEVELOPER_MODE is true * feat: install public SSH keys into the cloud instances * chore: enable @mdelapenya in SSH access * chore: mark EC2 instances with a proper label to be reaped * feat: add a pipeline that removes AWS cloud resources on Sundays * chore: force installation * fix: install pip * fix: run script after having the repo (order matters) * chore: use base commit as RUN_ID This way we will be able to correlate the cloud instances with a commit SHA instead of a random UUID * docs: update RUN_ID docs * chore: add @adam-stokes as SSH user * fix: do not import ssh keys as root * chore: rename tag used on AWS reaper * chore: add a label for node kind * chore: add buildURL as tag * fix: camelcase * fix: pass buildURL when creating runners * chore: use default value for buildURL Provisioning from local machine will lead to buildURL not to be populated, so we need a default value * chore: simplify reaper pipeline removing git checkout * chore: install python-pip for Suse and CentOS * chore: back to a random UUID * feat: add a VM label for the git base commit * chore: define labels once Co-authored-by: Adam Stokes <[email protected]>
…ic#2001) * chore: install ssh-import-id from pip It will allow importing public SSH keys from Github so we can easily provide a way for developers to connect to the cloud instances. See https://github.com/dustinkirkland/ssh-import-id * feat: do not destroy cloud resources if DEVELOPER_MODE is true * feat: install public SSH keys into the cloud instances * chore: enable @mdelapenya in SSH access * chore: mark EC2 instances with a proper label to be reaped * feat: add a pipeline that removes AWS cloud resources on Sundays * chore: force installation * fix: install pip * fix: run script after having the repo (order matters) * chore: use base commit as RUN_ID This way we will be able to correlate the cloud instances with a commit SHA instead of a random UUID * docs: update RUN_ID docs * chore: add @adam-stokes as SSH user * fix: do not import ssh keys as root * chore: rename tag used on AWS reaper * chore: add a label for node kind * chore: add buildURL as tag * fix: camelcase * fix: pass buildURL when creating runners * chore: use default value for buildURL Provisioning from local machine will lead to buildURL not to be populated, so we need a default value * chore: simplify reaper pipeline removing git checkout * chore: install python-pip for Suse and CentOS * chore: back to a random UUID * feat: add a VM label for the git base commit * chore: define labels once Co-authored-by: Adam Stokes <[email protected]>
💔 Build Failed
Expand to view the summary
Build stats
Test stats 🧪
Test errorsExpand to view the tests failures
|
#2031) * chore: install ssh-import-id from pip It will allow importing public SSH keys from Github so we can easily provide a way for developers to connect to the cloud instances. See https://github.com/dustinkirkland/ssh-import-id * feat: do not destroy cloud resources if DEVELOPER_MODE is true * feat: install public SSH keys into the cloud instances * chore: enable @mdelapenya in SSH access * chore: mark EC2 instances with a proper label to be reaped * feat: add a pipeline that removes AWS cloud resources on Sundays * chore: force installation * fix: install pip * fix: run script after having the repo (order matters) * chore: use base commit as RUN_ID This way we will be able to correlate the cloud instances with a commit SHA instead of a random UUID * docs: update RUN_ID docs * chore: add @adam-stokes as SSH user * fix: do not import ssh keys as root * chore: rename tag used on AWS reaper * chore: add a label for node kind * chore: add buildURL as tag * fix: camelcase * fix: pass buildURL when creating runners * chore: use default value for buildURL Provisioning from local machine will lead to buildURL not to be populated, so we need a default value * chore: simplify reaper pipeline removing git checkout * chore: install python-pip for Suse and CentOS * chore: back to a random UUID * feat: add a VM label for the git base commit * chore: define labels once Co-authored-by: Adam Stokes <[email protected]> Co-authored-by: Adam Stokes <[email protected]>
#2030) * chore: install ssh-import-id from pip It will allow importing public SSH keys from Github so we can easily provide a way for developers to connect to the cloud instances. See https://github.com/dustinkirkland/ssh-import-id * feat: do not destroy cloud resources if DEVELOPER_MODE is true * feat: install public SSH keys into the cloud instances * chore: enable @mdelapenya in SSH access * chore: mark EC2 instances with a proper label to be reaped * feat: add a pipeline that removes AWS cloud resources on Sundays * chore: force installation * fix: install pip * fix: run script after having the repo (order matters) * chore: use base commit as RUN_ID This way we will be able to correlate the cloud instances with a commit SHA instead of a random UUID * docs: update RUN_ID docs * chore: add @adam-stokes as SSH user * fix: do not import ssh keys as root * chore: rename tag used on AWS reaper * chore: add a label for node kind * chore: add buildURL as tag * fix: camelcase * fix: pass buildURL when creating runners * chore: use default value for buildURL Provisioning from local machine will lead to buildURL not to be populated, so we need a default value * chore: simplify reaper pipeline removing git checkout * chore: install python-pip for Suse and CentOS * chore: back to a random UUID * feat: add a VM label for the git base commit * chore: define labels once Co-authored-by: Adam Stokes <[email protected]> Co-authored-by: Adam Stokes <[email protected]>
✅ Pull request refreshed |
* chore: install ssh-import-id from pip It will allow importing public SSH keys from Github so we can easily provide a way for developers to connect to the cloud instances. See https://github.com/dustinkirkland/ssh-import-id * feat: do not destroy cloud resources if DEVELOPER_MODE is true * feat: install public SSH keys into the cloud instances * chore: enable @mdelapenya in SSH access * chore: mark EC2 instances with a proper label to be reaped * feat: add a pipeline that removes AWS cloud resources on Sundays * chore: force installation * fix: install pip * fix: run script after having the repo (order matters) * chore: use base commit as RUN_ID This way we will be able to correlate the cloud instances with a commit SHA instead of a random UUID * docs: update RUN_ID docs * chore: add @adam-stokes as SSH user * fix: do not import ssh keys as root * chore: rename tag used on AWS reaper * chore: add a label for node kind * chore: add buildURL as tag * fix: camelcase * fix: pass buildURL when creating runners * chore: use default value for buildURL Provisioning from local machine will lead to buildURL not to be populated, so we need a default value * chore: simplify reaper pipeline removing git checkout * chore: install python-pip for Suse and CentOS * chore: back to a random UUID * feat: add a VM label for the git base commit * chore: define labels once Co-authored-by: Adam Stokes <[email protected]> (cherry picked from commit fd75b80)
* chore: install ssh-import-id from pip It will allow importing public SSH keys from Github so we can easily provide a way for developers to connect to the cloud instances. See https://github.com/dustinkirkland/ssh-import-id * feat: do not destroy cloud resources if DEVELOPER_MODE is true * feat: install public SSH keys into the cloud instances * chore: enable @mdelapenya in SSH access * chore: mark EC2 instances with a proper label to be reaped * feat: add a pipeline that removes AWS cloud resources on Sundays * chore: force installation * fix: install pip * fix: run script after having the repo (order matters) * chore: use base commit as RUN_ID This way we will be able to correlate the cloud instances with a commit SHA instead of a random UUID * docs: update RUN_ID docs * chore: add @adam-stokes as SSH user * fix: do not import ssh keys as root * chore: rename tag used on AWS reaper * chore: add a label for node kind * chore: add buildURL as tag * fix: camelcase * fix: pass buildURL when creating runners * chore: use default value for buildURL Provisioning from local machine will lead to buildURL not to be populated, so we need a default value * chore: simplify reaper pipeline removing git checkout * chore: install python-pip for Suse and CentOS * chore: back to a random UUID * feat: add a VM label for the git base commit * chore: define labels once Co-authored-by: Adam Stokes <[email protected]> (cherry picked from commit fd75b80)
* chore: install ssh-import-id from pip It will allow importing public SSH keys from Github so we can easily provide a way for developers to connect to the cloud instances. See https://github.com/dustinkirkland/ssh-import-id * feat: do not destroy cloud resources if DEVELOPER_MODE is true * feat: install public SSH keys into the cloud instances * chore: enable @mdelapenya in SSH access * chore: mark EC2 instances with a proper label to be reaped * feat: add a pipeline that removes AWS cloud resources on Sundays * chore: force installation * fix: install pip * fix: run script after having the repo (order matters) * chore: use base commit as RUN_ID This way we will be able to correlate the cloud instances with a commit SHA instead of a random UUID * docs: update RUN_ID docs * chore: add @adam-stokes as SSH user * fix: do not import ssh keys as root * chore: rename tag used on AWS reaper * chore: add a label for node kind * chore: add buildURL as tag * fix: camelcase * fix: pass buildURL when creating runners * chore: use default value for buildURL Provisioning from local machine will lead to buildURL not to be populated, so we need a default value * chore: simplify reaper pipeline removing git checkout * chore: install python-pip for Suse and CentOS * chore: back to a random UUID * feat: add a VM label for the git base commit * chore: define labels once Co-authored-by: Adam Stokes <[email protected]> (cherry picked from commit fd75b80) # Conflicts: # .ci/Jenkinsfile # .ci/ansible/playbook.yml # .ci/ansible/tasks/install_deps.yml
… backport for 7.16 (#2032) * chore: remove unused code (#1119) * chore: remove unused code * chore: remove all references to fleet server hostname Because we assume it's a runtime dependency, provided by the initial compose file, we do not need to calculate service names, or URIs for the fleet-service endpoint. Instead, we assume it's listening in the 8220 port in the "fleet-server" hostname, which is accessible from the network created by docker-compose. * fix: use HTTP to connect to fleet-server * chore: remove fleet server policy code We do not need it anymore, as the fleet server is already bootstrapped * chore: remove all policies but system and fleet_server * Update policies.go * Update fleet.go * Update stand-alone.go Co-authored-by: Adam Stokes <[email protected]> * fix: wrong resolve conflicts * fix: wrong resolve conflicts Co-authored-by: Adam Stokes <[email protected]>
* main: (45 commits) feat: add CentOS 8 support (elastic#2034) fix: set default region for AWS cli (elastic#2053) chore: use Ansible's built-in replace instead of sed (elastic#2048) chore: split stack configuration and start into two tasks (elastic#2044) feat: enable SSH access to users for debugging cloud instances (elastic#2001) fix: use the right branch for 7.17 backports (elastic#2025) SLES15 enablement (elastic#2007) chore: bump stale agent for main (elastic#2014) Update `fetchBeatsBinary` to be reused in elastic-agent-poc (elastic#1984) chore: add resiliency when provisioning the stack (elastic#1990) chore: bump elastic-package to v0.32.1 (elastic#1959) feat: export Fetch&Download methods in the /pkg directory (elastic#1943) bump stack version 8.1.0-dbc834fd (elastic#1948) bump stack version 8.1.0-76902d39 (elastic#1946) chore: retire 7.15 adding 7.17 (elastic#1938) ci: use withAPMEnv (elastic#1917) Update main branch (elastic#1928) bump stack version 8.1.0-befff95a (elastic#1929) chore: properly evaluate how tests are skipped on CI when checking modified files (elastic#1924) bump stack version 8.1.0-60bffc32 (elastic#1921) ...
What does this PR do?
This PR does a few things:
ssh-import-id
(see https://github.com/dustinkirkland/ssh-import-id)ReaperMark=e2e-testing-vm
Why is it important?
We need developers to access the machines to troubleshoot test errors, and because we are not there yet in terms of full Observability of the test execution, we need to SSH into the machines and check logs, file states, etc.
Checklist
make unit-test
), and they are passing locallymake notice
in the proper directory)Author's Checklist
How to test this PR locally
To verify that the keys are installed, please first make a copy of your authorised keys file:
Related issues