Skip to content

Commit

Permalink
Merge pull request #3832 from freedomofpress/docs-upgrade-testing
Browse files Browse the repository at this point in the history
Updates developer docs for upgrade testing targets
  • Loading branch information
conorsch authored Jan 14, 2019
2 parents 1c877f7 + cadd878 commit eb82c23
Show file tree
Hide file tree
Showing 5 changed files with 118 additions and 58 deletions.
21 changes: 13 additions & 8 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -163,20 +163,25 @@ staging-xenial: ## Creates local staging VMs based on Xenial, autodetecting plat
clean: ## DANGER! Purges all site-specific info and developer files from project.
@./devops/clean

.PHONY: upgrade_start
upgrade_start: ## Boot up an upgrade test base environment using libvirt
.PHONY: upgrade-start
upgrade-start: ## Boot up an upgrade test base environment using libvirt
@SD_UPGRADE_BASE=$(STABLE_VER) molecule converge -s upgrade

.PHONY: upgrade_destroy
upgrade_destroy: ## Destroy up an upgrade test base environment
.PHONY: upgrade-start-qa
upgrade-start-qa: ## Boot up an upgrade test base env using libvirt in remote apt mode
@SD_UPGRADE_BASE=$(STABLE_VER) QA_APTTEST=yes molecule converge -s upgrade

.PHONY: upgrade-destroy
upgrade-destroy: ## Destroy up an upgrade test base environment
@SD_UPGRADE_BASE=$(STABLE_VER) molecule destroy -s upgrade

.PHONY: upgrade_test_local
upgrade_test_local: ## Once an upgrade environment is running, force upgrade apt packages (local pkgs)
.PHONY: upgrade-test-local
upgrade-test-local: ## Once an upgrade environment is running, force upgrade apt packages (local pkgs)
@molecule side-effect -s upgrade

.PHONY: upgrade_test_qa
upgrade_test_qa: ## Once an upgrade environment is running, force upgrade apt packages (from qa server)
.PHONY: upgrade-test-qa
upgrade-test-qa: ## Once an upgrade environment is running, force upgrade apt packages (from qa server)
@QA_APTTEST=yes molecule converge -s upgrade -- --diff -t apt
@QA_APTTEST=yes molecule side-effect -s upgrade

# Explaination of the below shell command should it ever break.
Expand Down
107 changes: 57 additions & 50 deletions docs/development/upgrade_testing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,23 +6,23 @@ Upgrade Testing using Molecule
The SecureDrop project includes Molecule scenarios for developing and testing against
multi-server configurations, including a scenario to simulate the process of upgrading an
existing system. This document explains how to work with this scenario to test
features that make potentially release-breaking changes such as database
features that make potentially release-breaking changes such as database
schema updates.

The Molecule upgrade scenario sets up a predefined staging Securedrop virtual
The Molecule upgrade scenario sets up a predefined staging Securedrop virtual
environment using Vagrant boxes built with the latest application release.
It also creates a virtualized APT repository, and modifies
the SecureDrop environment to use this APT repository instead of the FPF main
repo at https://apt.freedom.press/.
It also creates a virtualized APT repository, and modifies
the SecureDrop environment to use this APT repository instead of the FPF main
repo at https://apt.freedom.press/.

You can use this scenario to test the upgrade process, using using either
locally-built .debs or packages from the FPF test repo at
You can use this scenario to test the upgrade process, using using either
locally-built .debs or packages from the FPF test repo at
https://apt-test.freedom.press/. Both options are described below.
.. note:: The upgrade scenario uses QEMU/KVM via Vagrant's libvirt provider, in
place of the default Virtualbox provider. If you haven't already done so,
you'll need to set up the libvirt provider before proceeding. For
more information, see :ref:`libvirt_provider`.

.. note:: The upgrade scenario uses QEMU/KVM via Vagrant's libvirt provider, in
place of the default Virtualbox provider. If you haven't already done so,
you'll need to set up the libvirt provider before proceeding. For
more information, see :ref:`libvirt_provider`.

.. _upgrade_testing_local:

Expand All @@ -34,92 +34,99 @@ First, build the app code packages and create the environment:
.. code:: sh
make build-debs
molecule converge -s upgrade
make upgrade-start
The playbook will return the source interface Onion address. You can use this to
check the application version displayed in the source interface footer.
Alternatively, you can log into the Application Server VM and check the deployed
check the application version displayed in the source interface footer.
Alternatively, you can log into the *Application Server* VM and check the deployed
package version directly:

.. code:: sh
molecule login -s upgrade -h app-staging
From the Application Server:
From the *Application Server*:

.. code:: sh
apt-cache-policy securedrop-config
The installed package version should match the latest release version.

To perform an upgrade using the virtualized APT repository, log out of the
Application Server and run the Molecule side-effect action:
To perform an upgrade using the virtualized APT repository, log out of the
*Application Server* and run the Molecule side-effect action:

.. code:: sh
molecule side-effect -s upgrade
make upgrade-test-local
This will upgrade the SecureDrop packages on the application and
monitor servers,
using your locally-built packages and APT VM instead of the FPF main APT
repository.
This will upgrade the SecureDrop packages on the *Application* and
*Monitor Servers*, using your locally-built packages and apt VM instead of the
FPF production apt repository.

You can verify that the application version has changed either by checking the
source interface's footer or directly on the Application Server as described
above.
You can verify that the application version has changed either by checking the
source interface's footer or directly on the *Application Server* as described
above.

.. _upgrade_testing_apt:

Upgrade testing using apt-test.freedom.press
--------------------------------------------

You can use the upgrade scenario to test upgrades using official release
candidate packages from the FPF test APT repository. First,
You can use the upgrade scenario to test upgrades using official release
candidate packages from the FPF test APT repository. First,
create the environment:

.. code:: sh
make build-debs
molecule converge -s upgrade
make upgrade-start-qa
Then, log into the Application Server:
Then, log into the *Application Server*:

.. code:: sh
molecule login -s upgrade -h app-staging
From the Application Server:
From the *Application Server*:

.. code:: sh
sudo apt-get update
apt-cache policy securedrop-config
The installed package version should match the current release version, and the
candidate version should match your locally-built version.

Now, log out of the app server. To switch to the apt-test proxy:
The installed package version should match the current release version.
To install the latest packages from the apt-test proxy:

.. code:: sh
QA_APTTEST=yes molecule converge -s upgrade -- --diff -t apt
make upgrade-test-qa
Log back into the Application Server, and repeat the previous commands:
Log back into the *Application Server*, and repeat the previous commands:

.. code:: sh
sudo apt-get update
apt-cache policy securedrop-config
This time, you should see multiple entries in the version table, corresponding
to the versions available on the FPF test APT repository. If a new release
candidate is available, you can use the molecule side-effect action from your
local terminal to perform an upgrade:

.. code:: sh
molecule side-effect -s upgrade
apt-cache policy securedrop-config
Navigate to the Source Interface URL again, and confirm you see the upgraded
Navigate to the Source Interface URL again, and confirm you see the upgraded
version in the footer. Then proceed with testing the new version.

Updating the base boxes used for upgrade testing
------------------------------------------------

When a new version of SecureDrop is released, we must create and upload
new VM images, to enable testing against that base version in future upgrade
testing. The procedure is as follows:

1. ``git checkout <version>``
2. ``make vagrant-package``
3. ``mv molecule/vagrant_packager/build/app-staging{,_<version>}.box``
4. ``mv molecule/vagrant_packager/build/mon-staging{,_<version>}.box``
5. ``sha256sum molecule/vagrant_packager/build/*.box``
6. Manually update ``molecule/vagrant_packager/box_files/*.json`` with new
version information, including URL and checksum.
7. ``cd molecule/vagrant_packager && ./push.yml`` to upload to S3
8. Commit the local changes to JSON files and open a PR.

Subsequent invocations of ``make upgrade-start`` will pull the latest
version of the box.
26 changes: 26 additions & 0 deletions molecule/upgrade/playbook.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,30 @@
---
- name: Confirm local debs present if required
hosts: localhost
gather_facts: no
vars:
molecule_dir: "{{ lookup('env', 'MOLECULE_SCENARIO_DIRECTORY') }}"
tasks:
# Before running apt-server logic, confirm we have local
# debs built.
- name: Establish QA strategy
set_fact:
QA_APTTEST: "{{ lookup('env','QA_APTTEST')|bool }}"

- name: Find local deb files
find:
paths: "{{ molecule_dir }}/../../build/"
patterns: "*.deb"
register: _upgrade_scenario_find_debs_result
when: not QA_APTTEST

- name: Ensure debs were found
assert:
that:
- "_upgrade_scenario_find_debs_result.files|length >= 8"
msg: "No local debs found, run 'make build-debs'"
when: not QA_APTTEST

- name: Re-run original tor role
hosts: securedrop
max_fail_percentage: 0
Expand Down
11 changes: 11 additions & 0 deletions molecule/vagrant_packager/box_files/app_metadata.json
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,17 @@
"checksum": "6fb8c12821b902b0905f967d7405c620443f4f4345b18d05b59005f798a08e50"
}
]
},
{
"version": "0.11.0",
"providers": [
{
"name": "libvirt",
"url": "https://s3.amazonaws.com/securedrop-vagrant/app-staging_0.11.0.box",
"checksum_type": "sha256",
"checksum": "cf2c4c8ac89bd132d3f75ee77e2d97b68c562e1ee36aa059bb1d4e6b37499f62"
}
]
}
]
}
11 changes: 11 additions & 0 deletions molecule/vagrant_packager/box_files/mon_metadata.json
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,17 @@
"checksum": "9891c88aa7148129f2f91638d7dfed1e7815eb980bba1de8a9c075f14ae0ddeb"
}
]
},
{
"version": "0.11.0",
"providers": [
{
"name": "libvirt",
"url": "https://s3.amazonaws.com/securedrop-vagrant/mon-staging_0.11.0.box",
"checksum_type": "sha256",
"checksum": "baba21e8799fe2093d902b332b45d7a8342adf019fa195382011fbdfa54cd1d5"
}
]
}
]
}

0 comments on commit eb82c23

Please sign in to comment.