From 2fc5841fe73b812153a39051cd96685121864ac8 Mon Sep 17 00:00:00 2001 From: Conor Schaefer Date: Mon, 1 Oct 2018 11:19:19 +0200 Subject: [PATCH 1/7] Converts underscores to hyphens in upgrade targets Stylistic nit. All the Makefile targets use hyphens, rather than underscores, to the "upgrade" targets were inconsistent. Partly I want to avoid holding shift where possible, but more important, I don't want to have to remember when I do and do not have to use shift for autocomplete to work. --- Makefile | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/Makefile b/Makefile index c0e939e6bf..cfb2ef3c4b 100644 --- a/Makefile +++ b/Makefile @@ -163,20 +163,20 @@ staging-xenial: ## Creates local staging VMs based on Xenial, autodetecting plat clean: ## DANGER! Purges all site-specific info and developer files from project. @./devops/clean -.PHONY: upgrade_start -upgrade_start: ## Boot up an upgrade test base environment using libvirt +.PHONY: upgrade-start +upgrade-start: ## Boot up an upgrade test base environment using libvirt @SD_UPGRADE_BASE=$(STABLE_VER) molecule converge -s upgrade -.PHONY: upgrade_destroy -upgrade_destroy: ## Destroy up an upgrade test base environment +.PHONY: upgrade-destroy +upgrade-destroy: ## Destroy up an upgrade test base environment @SD_UPGRADE_BASE=$(STABLE_VER) molecule destroy -s upgrade -.PHONY: upgrade_test_local -upgrade_test_local: ## Once an upgrade environment is running, force upgrade apt packages (local pkgs) +.PHONY: upgrade-test-local +upgrade-test-local: ## Once an upgrade environment is running, force upgrade apt packages (local pkgs) @molecule side-effect -s upgrade -.PHONY: upgrade_test_qa -upgrade_test_qa: ## Once an upgrade environment is running, force upgrade apt packages (from qa server) +.PHONY: upgrade-test-qa +upgrade-test-qa: ## Once an upgrade environment is running, force upgrade apt packages (from qa server) @QA_APTTEST=yes molecule side-effect -s upgrade # Explaination of the below shell command should it ever break. From d3495d8aa6f5dbb211debc0f2e41fee4e90c0a06 Mon Sep 17 00:00:00 2001 From: Conor Schaefer Date: Mon, 1 Oct 2018 11:31:34 +0200 Subject: [PATCH 2/7] Ensure apt-test upgrade scenario uses apt-test proxy The Makefile logic wasn't automatically converting to the apt-test repo; developers still had to run an ad-hoc Molecule command from the docs in order to convert from local debs to the apt-test proxy. Let's make that automatic, as part of the "upgrade-test-qa" Makefile target. --- Makefile | 1 + 1 file changed, 1 insertion(+) diff --git a/Makefile b/Makefile index cfb2ef3c4b..ba1bbbf0af 100644 --- a/Makefile +++ b/Makefile @@ -177,6 +177,7 @@ upgrade-test-local: ## Once an upgrade environment is running, force upgrade apt .PHONY: upgrade-test-qa upgrade-test-qa: ## Once an upgrade environment is running, force upgrade apt packages (from qa server) + @QA_APTTEST=yes molecule converge -s upgrade -- --diff -t apt @QA_APTTEST=yes molecule side-effect -s upgrade # Explaination of the below shell command should it ever break. From 0c3eed113baba95151c3bd48d5dbfc702fc8c2ed Mon Sep 17 00:00:00 2001 From: Conor Schaefer Date: Mon, 1 Oct 2018 11:32:27 +0200 Subject: [PATCH 3/7] Updates upgrade docs to use new Makefile targets We have convenient Makefile targets for the various upgrade testing scenario flows: * upgrade-start * upgrade-test-local * upgrade-test-qa Let's use those new targets in the docs where appropriate, to streamline the process for developers using the scenarios locally. --- docs/development/upgrade_testing.rst | 87 ++++++++++++---------------- 1 file changed, 37 insertions(+), 50 deletions(-) diff --git a/docs/development/upgrade_testing.rst b/docs/development/upgrade_testing.rst index fc1f5740aa..fa62e4f3bf 100644 --- a/docs/development/upgrade_testing.rst +++ b/docs/development/upgrade_testing.rst @@ -6,23 +6,23 @@ Upgrade Testing using Molecule The SecureDrop project includes Molecule scenarios for developing and testing against multi-server configurations, including a scenario to simulate the process of upgrading an existing system. This document explains how to work with this scenario to test -features that make potentially release-breaking changes such as database +features that make potentially release-breaking changes such as database schema updates. -The Molecule upgrade scenario sets up a predefined staging Securedrop virtual +The Molecule upgrade scenario sets up a predefined staging Securedrop virtual environment using Vagrant boxes built with the latest application release. -It also creates a virtualized APT repository, and modifies -the SecureDrop environment to use this APT repository instead of the FPF main -repo at https://apt.freedom.press/. +It also creates a virtualized APT repository, and modifies +the SecureDrop environment to use this APT repository instead of the FPF main +repo at https://apt.freedom.press/. -You can use this scenario to test the upgrade process, using using either -locally-built .debs or packages from the FPF test repo at +You can use this scenario to test the upgrade process, using using either +locally-built .debs or packages from the FPF test repo at https://apt-test.freedom.press/. Both options are described below. - -.. note:: The upgrade scenario uses QEMU/KVM via Vagrant's libvirt provider, in - place of the default Virtualbox provider. If you haven't already done so, - you'll need to set up the libvirt provider before proceeding. For - more information, see :ref:`libvirt_provider`. + +.. note:: The upgrade scenario uses QEMU/KVM via Vagrant's libvirt provider, in + place of the default Virtualbox provider. If you haven't already done so, + you'll need to set up the libvirt provider before proceeding. For + more information, see :ref:`libvirt_provider`. .. _upgrade_testing_local: @@ -34,18 +34,18 @@ First, build the app code packages and create the environment: .. code:: sh make build-debs - molecule converge -s upgrade + make upgrade-start The playbook will return the source interface Onion address. You can use this to -check the application version displayed in the source interface footer. -Alternatively, you can log into the Application Server VM and check the deployed +check the application version displayed in the source interface footer. +Alternatively, you can log into the *Application Server* VM and check the deployed package version directly: .. code:: sh molecule login -s upgrade -h app-staging -From the Application Server: +From the *Application Server*: .. code:: sh @@ -53,73 +53,60 @@ From the Application Server: The installed package version should match the latest release version. -To perform an upgrade using the virtualized APT repository, log out of the -Application Server and run the Molecule side-effect action: +To perform an upgrade using the virtualized APT repository, log out of the +*Application Server* and run the Molecule side-effect action: .. code:: sh - molecule side-effect -s upgrade + make upgrade-test-local -This will upgrade the SecureDrop packages on the application and -monitor servers, -using your locally-built packages and APT VM instead of the FPF main APT -repository. +This will upgrade the SecureDrop packages on the *Application* and +*Monitor Servers*, using your locally-built packages and apt VM instead of the +FPF production apt repository. -You can verify that the application version has changed either by checking the -source interface's footer or directly on the Application Server as described -above. +You can verify that the application version has changed either by checking the +source interface's footer or directly on the *Application Server* as described +above. .. _upgrade_testing_apt: Upgrade testing using apt-test.freedom.press -------------------------------------------- -You can use the upgrade scenario to test upgrades using official release -candidate packages from the FPF test APT repository. First, +You can use the upgrade scenario to test upgrades using official release +candidate packages from the FPF test APT repository. First, create the environment: .. code:: sh - make build-debs - molecule converge -s upgrade + make upgrade-start -Then, log into the Application Server: +Then, log into the *Application Server*: .. code:: sh - + molecule login -s upgrade -h app-staging -From the Application Server: +From the *Application Server*: .. code:: sh sudo apt-get update apt-cache policy securedrop-config -The installed package version should match the current release version, and the -candidate version should match your locally-built version. - -Now, log out of the app server. To switch to the apt-test proxy: +The installed package version should match the current release version. +To install the latest packages from the apt-test proxy: .. code:: sh - QA_APTTEST=yes molecule converge -s upgrade -- --diff -t apt + make upgrade-test-qa -Log back into the Application Server, and repeat the previous commands: +Log back into the *Application Server*, and repeat the previous commands: .. code:: sh sudo apt-get update - apt-cache policy securedrop-config - -This time, you should see multiple entries in the version table, corresponding -to the versions available on the FPF test APT repository. If a new release -candidate is available, you can use the molecule side-effect action from your -local terminal to perform an upgrade: - -.. code:: sh - - molecule side-effect -s upgrade + apt-cache policy securedrop-config -Navigate to the Source Interface URL again, and confirm you see the upgraded +Navigate to the Source Interface URL again, and confirm you see the upgraded version in the footer. Then proceed with testing the new version. From 4f62b29de6ebaa9f5b8e20164bb52dc828ad322f Mon Sep 17 00:00:00 2001 From: Conor Schaefer Date: Thu, 3 Jan 2019 16:36:40 -0800 Subject: [PATCH 4/7] Adds upgrade testing box for 0.11.0 We missed updating the upgrade boxes during post-release procedure for both 0.10.0 and 0.11.0. Since 0.11.0 is still the most current release, it was rather straightforward to prepare a box. Catching up with 0.10.0 was trickier, however, since the box preparation logic assumes that whatever version is posted to the prod apt repo is the correct one to prepare for; no longer the case. As such, I'm omitting metadata for the 0.10.0 version. Since I did prepare and upload those boxes before realizing the 0.10/0.11 version discrepancy, here are the checksums of what's stored in S3: { "version": "0.10.0", "providers": [ { "name": "libvirt", "url": "https://s3.amazonaws.com/securedrop-vagrant/app-staging_0.10.0.box", "checksum_type": "sha256", "checksum": "2dda15d0f2557dd434dfba4cf0b3ab58b30666ac0277cd8d32b973d954b54264" } ] } { "version": "0.10.0", "providers": [ { "name": "libvirt", "url": "https://s3.amazonaws.com/securedrop-vagrant/mon-staging_0.10.0.box", "checksum_type": "sha256", "checksum": "6e281590a7dfd7753136b7c6505c8dd1ebc4d7f8207c890a80db4871d6ac08b7" } ] } --- molecule/vagrant_packager/box_files/app_metadata.json | 11 +++++++++++ molecule/vagrant_packager/box_files/mon_metadata.json | 11 +++++++++++ 2 files changed, 22 insertions(+) diff --git a/molecule/vagrant_packager/box_files/app_metadata.json b/molecule/vagrant_packager/box_files/app_metadata.json index 6d8bb43e60..a5d9ec726f 100644 --- a/molecule/vagrant_packager/box_files/app_metadata.json +++ b/molecule/vagrant_packager/box_files/app_metadata.json @@ -34,6 +34,17 @@ "checksum": "6fb8c12821b902b0905f967d7405c620443f4f4345b18d05b59005f798a08e50" } ] + }, + { + "version": "0.11.0", + "providers": [ + { + "name": "libvirt", + "url": "https://s3.amazonaws.com/securedrop-vagrant/app-staging_0.11.0.box", + "checksum_type": "sha256", + "checksum": "cf2c4c8ac89bd132d3f75ee77e2d97b68c562e1ee36aa059bb1d4e6b37499f62" + } + ] } ] } diff --git a/molecule/vagrant_packager/box_files/mon_metadata.json b/molecule/vagrant_packager/box_files/mon_metadata.json index f126876197..a1fd724b78 100644 --- a/molecule/vagrant_packager/box_files/mon_metadata.json +++ b/molecule/vagrant_packager/box_files/mon_metadata.json @@ -34,6 +34,17 @@ "checksum": "9891c88aa7148129f2f91638d7dfed1e7815eb980bba1de8a9c075f14ae0ddeb" } ] + }, + { + "version": "0.11.0", + "providers": [ + { + "name": "libvirt", + "url": "https://s3.amazonaws.com/securedrop-vagrant/mon-staging_0.11.0.box", + "checksum_type": "sha256", + "checksum": "baba21e8799fe2093d902b332b45d7a8342adf019fa195382011fbdfa54cd1d5" + } + ] } ] } From 2db8238ec0413d080b19aeee84bfcb6b8f134b4e Mon Sep 17 00:00:00 2001 From: Conor Schaefer Date: Fri, 4 Jan 2019 11:11:19 -0800 Subject: [PATCH 5/7] Documents updating base boxes for upgrade scenario Conferred with @msheiny on the manual steps. Ideally we'd automate even these, via inspecting checksums and modifying the JSON file programatically, but the manual changes are only necessary after each SD release, which isn't a terrible burden right now. --- docs/development/upgrade_testing.rst | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/docs/development/upgrade_testing.rst b/docs/development/upgrade_testing.rst index fa62e4f3bf..f3b48d65ab 100644 --- a/docs/development/upgrade_testing.rst +++ b/docs/development/upgrade_testing.rst @@ -110,3 +110,23 @@ Log back into the *Application Server*, and repeat the previous commands: Navigate to the Source Interface URL again, and confirm you see the upgraded version in the footer. Then proceed with testing the new version. + +Updating the base boxes used for upgrade testing +------------------------------------------------ + +When a new version of SecureDrop is released, we must create and upload +new VM images, to enable testing against that base version in future upgrade +testing. The procedure is as follows: + +1. ``git checkout `` +2. ``make vagrant-package`` +3. ``mv molecule/vagrant_packager/build/app-staging{,_}.box`` +4. ``mv molecule/vagrant_packager/build/mon-staging{,_}.box`` +5. ``sha256sum molecule/vagrant_packager/build/*.box`` +6. Manually update ``molecule/vagrant_packager/box_files/*.json`` with new + version information, including URL and checksum. +7. ``cd molecule/vagrant_packager && ./push.yml`` to upload to S3 +8. Commit the local changes to JSON files and open a PR. + +Subsequent invocations of ``make upgrade-start`` will pull the latest +version of the box. From 6cc4c70700998ee9f420d163f3ebcc47fae00c56 Mon Sep 17 00:00:00 2001 From: Conor Schaefer Date: Fri, 4 Jan 2019 11:51:44 -0800 Subject: [PATCH 6/7] Confirm local debs are present for upgrade scenario If the upgrade scenario is used with the local debs strategy (i.e., QA_APT_TEST env var is not set), we should fail fast without the required local deb files. Previously, the scenario would proceed all the way to the local apt server configuration and only fail when attempting to sign the Release file with a test key. Now, the role will fail almost immediately without local debs present. Hat tip to @zenmonkeykstop for flagging during review. --- molecule/upgrade/playbook.yml | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/molecule/upgrade/playbook.yml b/molecule/upgrade/playbook.yml index 9be7bd731a..01b59cbd7d 100644 --- a/molecule/upgrade/playbook.yml +++ b/molecule/upgrade/playbook.yml @@ -1,4 +1,30 @@ --- +- name: Confirm local debs present if required + hosts: localhost + gather_facts: no + vars: + molecule_dir: "{{ lookup('env', 'MOLECULE_SCENARIO_DIRECTORY') }}" + tasks: + # Before running apt-server logic, confirm we have local + # debs built. + - name: Establish QA strategy + set_fact: + QA_APTTEST: "{{ lookup('env','QA_APTTEST')|bool }}" + + - name: Find local deb files + find: + paths: "{{ molecule_dir }}/../../build/" + patterns: "*.deb" + register: _upgrade_scenario_find_debs_result + when: not QA_APTTEST + + - name: Ensure debs were found + assert: + that: + - "_upgrade_scenario_find_debs_result.files|length >= 8" + msg: "No local debs found, run 'make build-debs'" + when: not QA_APTTEST + - name: Re-run original tor role hosts: securedrop max_fail_percentage: 0 From cadd8786df40160b08624b00eac22bbdba120b78 Mon Sep 17 00:00:00 2001 From: Conor Schaefer Date: Fri, 4 Jan 2019 14:01:52 -0800 Subject: [PATCH 7/7] Passes env var during apt-test upgrade testing Pointed out by @msheiny during review, so lifting the patch from PR discussion and implementing to resolve. The goal is to ensure that the "use apt-test repo" env var is declared during the "create" phase of the scenario, which is now handled. --- Makefile | 4 ++++ docs/development/upgrade_testing.rst | 2 +- 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/Makefile b/Makefile index ba1bbbf0af..d90755e062 100644 --- a/Makefile +++ b/Makefile @@ -167,6 +167,10 @@ clean: ## DANGER! Purges all site-specific info and developer files from project upgrade-start: ## Boot up an upgrade test base environment using libvirt @SD_UPGRADE_BASE=$(STABLE_VER) molecule converge -s upgrade +.PHONY: upgrade-start-qa +upgrade-start-qa: ## Boot up an upgrade test base env using libvirt in remote apt mode + @SD_UPGRADE_BASE=$(STABLE_VER) QA_APTTEST=yes molecule converge -s upgrade + .PHONY: upgrade-destroy upgrade-destroy: ## Destroy up an upgrade test base environment @SD_UPGRADE_BASE=$(STABLE_VER) molecule destroy -s upgrade diff --git a/docs/development/upgrade_testing.rst b/docs/development/upgrade_testing.rst index f3b48d65ab..736e5901da 100644 --- a/docs/development/upgrade_testing.rst +++ b/docs/development/upgrade_testing.rst @@ -79,7 +79,7 @@ create the environment: .. code:: sh - make upgrade-start + make upgrade-start-qa Then, log into the *Application Server*: