Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add regression test suite; fix redeploys with myhosttypes #82

Merged
merged 13 commits into from
Jan 25, 2021
Merged

Add regression test suite; fix redeploys with myhosttypes #82

merged 13 commits into from
Jan 25, 2021

Conversation

dseeley
Copy link
Contributor

@dseeley dseeley commented Jan 15, 2021 •
edited
Loading

Testsuite: new scripted pipeline job (jenkinsfiles/Jenkinsfile_testsuite).

  • Creates matrix builds to test multiple combinations of parameters simultaneously.
  • The declarative 'matrix' job has many deficiencies, which this addresses. This job makes calls out to a helper jenkinsfile to run the deploy/clean etc, an example is in EXAMPLE/jenkinsfiles/Jenkinsfile_ops.
  • Supports multiple matrices running in parallel (messy in BlueOcean) or in series (takes longer).
  • Support matrices of: multiple redeploy schemes, multiple OSs, multiple clouds, DNS enabled/disabled, myhosttypes set/unset.
  • Fixes Automated testing suite #37

Clusterverse Fixes:

  • Prevent _scheme_rmvm_keepdisk_rollback from detaching the root volume from AWS instances
  • Prevent _scheme_addallnew_rmdisk_rollback running with myhosttypes (not a supported configuration)
  • Fix redeploy tidy cleaning more than it should.
  • Allow for gcp_dns_resource_record_set occasional failures (suspect gcp throttling due to multiple machines registering simultaneously)
  • Fix jmespath "contains" logic for some of the redeploy tests (first parameter must be an array).

Clusterverse Enhancements:

  • Deprecate the ping module for validating remote connection; it times-out on hosts that are slow to boot (e.g. centos). The fix allows the old method to remain, but issues a deprecation warning, and advice on how to fix it properly (by putting a wait_for_connection in the top-level playbook).
  • Add ability to override cluster_vars from a command-line parameter (cluster_vars_override) - allows test harness to programmatically change cluster_vars (e.g. OS image, or disk allocation)
  • Add an example mechanism to load environment variables and files that are specific to the loaded cluster_vars. Useful, for example, for loading bastion SSH keys and ansible SSH parameters at runtime.
  • Add ability to extract ssh host (& bastion) credentials from the playbooks at runtime - removes credential coupling to Jenkins. Also prevent using bastion ssh_args when current host is in same network as bastion.
  • Move the bind9 (nsupdate) definitions to be inside the (per-buildenv) cluster_vars

Dougal Seeley added 4 commits January 14, 2021 18:18
New scripted pipeline job: jenkinsfiles/Jenkinsfile_testsuite.
aa2228b 
+ Creates matrix builds to test multiple combinations of parameters simultaneously.
+ The declarative 'matrix' job has many deficiencies, which this addresses.  This job makes calls out to a helper jenkinsfile to run the deploy/clean etc, an example is in EXAMPLE/jenkinsfiles/Jenkinsfile_ops.
+ Supports multiple matrices running in parallel (messy in BlueOcean) or in series (takes longer).
+ Support matrices of: multiple redeploy schemes, multiple OSs, multiple clouds, DNS enabled/disabled, myhosttypes set/unset.

Code changes:
+ Deprecate the ping module for validating remote connection; it times-out on hosts that are slow to boot (e.g. centos).  The fix allows the old method to remain, but issues a deprecation warning, and advice on how to fix it properly (by putting a wait_for_connection in the top-level playbook).
+ Add ability to override cluster_vars from a command-line parameter (cluster_vars_override) - allows test harness to
+ Fix jmespath "contains" logic for some of the redeploy tests (first parameter must be an array).
+ Prevent _scheme_addallnew_rmdisk_rollback running with myhosttypes (not a supported configuration)
+ Fix redeploy tidy cleaning more than it should.
Fix parallel of parallel (ensure unique BUILD_HASH for each matrix). …
c31e826 
…Simplify DNS testing. Fix release_version. Improve comments.
Allow per-environment GCP credentials
07000e1
Add 'sandbox=true' to the 'Active choices' type parameters ('uno-choi…
e8cb0e9 
…ce' plugin), because the permissive-script-security plugin does not recognise these 'GroovyScript' classes, and so will not automatically add the to the approvedScriptHashes in /var/jenkins_home/scriptApproval.xml.
@dseeley dseeley force-pushed the dps_testsuite_redeploy_fixes branch from d954383 to 2230459 Compare January 17, 2021 17:42
Add a mechanism to load environment variables and files that are spec…
179be10 
…ific to the loaded cluster_vars. Useful, for example, for loading bastion SSH keys and Ansible SSH parameters at runtime.
@dseeley dseeley force-pushed the dps_testsuite_redeploy_fixes branch from 2230459 to 179be10 Compare January 17, 2021 20:03
Dougal Seeley added 3 commits January 19, 2021 00:23
Add ability to extract ssh host (& bastion) credentials from the play…
8de60ee 
…books at runtime - removes credential coupling to Jenkins.

Reorder the variables.
Update variables
1b78268
Update testsuite vars
560b555
@dseeley-sky dseeley-sky force-pushed the dps_testsuite_redeploy_fixes branch from dd78965 to 560b555 Compare January 19, 2021 17:29
@dseeley-sky dseeley-sky force-pushed the dps_testsuite_redeploy_fixes branch from 1a9583f to 610bbee Compare January 21, 2021 23:58
Dougal Seeley added 2 commits January 22, 2021 14:42
Update testsuite
1dab2bf
Code fixes:
5586b2a 
+ Prevent _scheme_rmvm_keepdisk_rollback from detaching the root volume from AWS instances
+ Only load bastion ssh_args when current host is in same network as bastion.
+ Fix deprecated wait_for_ssh to work with new bastion definitions (as much as possible).
+ Allow for gcp_dns_resource_record_set failures (suspect gcp throttling due to multiple machines registering simultaneously)
+ Update the nsupdate code to be inside cluster_vars
+ Change crlf to lf
antoineserrano
antoineserrano approved these changes Jan 25, 2021
View reviewed changes
Copy link
Contributor

@antoineserrano antoineserrano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed/tested extensively. Not merging yet, so a desired release version can be created instead of the automated job.

@dseeley-sky dseeley-sky merged commit 5c8f574 into sky-uk:master Jan 25, 2021
@dseeley dseeley deleted the dps_testsuite_redeploy_fixes branch June 28, 2021 13:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@antoineserrano antoineserrano antoineserrano approved these changes

@dipiazzg dipiazzg Awaiting requested review from dipiazzg

@sky-amoncadot sky-amoncadot Awaiting requested review from sky-amoncadot

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Automated testing suite
3 participants
@dseeley @antoineserrano @dseeley-sky