Begins dynamic dev/prod config logic #432

conorsch · 2020-01-25T02:08:26Z

Status

Ready for review

Description of Changes

Refs #406

Changes proposed in this pull request:

Adds new config.json value "environment", accepts "dev" or "prod"
Updates Salt logic to dynamically use dev/prod vars, based on config.json "environment"
Converts URLs https://apt-test-qubes.freedom.press -> https://apt-test.freedom.press

There are a few caveats with the current implementation:

config.json is modified on-the-fly as part of e.g. make prod / make dev. Is that what we want?
The prod URLs don't actually have stable, signed packages yet—so full testing isn't possible

Testing

dev target

make clone
make clean
make all
make test

staging target

Only run on test hardware. We don't want to install CI-built artifacts in dom0.

make clone
make clean
make staging # confirm prompt
make test

Checklist

If you have made changes to the provisioning logic

Linting (make flake8) and tests (make test) pass in dom0 of a Qubes install
I have added/removed files, and have updated packaging logic in MANIFEST.in and rpm-build/SPECS/securedrop-workstation-dom0-config.spec

emkll · 2020-01-28T15:22:39Z

Added securedrop-admin logic to staging and prod scenarios (the /usr/bin/ file is brought in exclusively by the RPM), reusing existing logic from the scripts/ directory.
Added conditional logic to install securedrop-workstation-dom0-config for staging and prod environments
Added a test-only staging scenario, which should mirror prod, but use test-specific URLs.
Fixed the tests

Note that the staging environment should be used on test hardware only, and provisions the workstation VMs and their environment based on the config that's in the RPM itself (initial dom0 logic is ensured by the Makefile target). Dom0 is configured by installing the package, and the package will replace the files in /srv/salt in place, during the run.

To prevent a race condition, once an RPM is is uploaded with changes from this branch (with the securedrop-admin wrapper and script it place, we should apply the following diff, for staging and prod targets):

diff --git a/Makefile b/Makefile
index 9e83f0611..585e783f7 100644
--- a/Makefile
+++ b/Makefile
@@ -27,7 +27,8 @@ staging: ## Configures a STAGING install. To be used on test hardware ONLY
        ./scripts/configure-environment --env staging
        $(MAKE) validate
        $(MAKE) prep-salt
-       ./scripts/provision-all
+       $(MAKE) prep-dom0
+       /usr/bin/securedrop-admin --apply
 
 dom0-rpm: ## Builds rpm package to be installed on dom0
        @./scripts/build-dom0-rpm

conorsch

Thanks for the collab, @emkll, these changes are looking solid. Pushed up some more cleanup (the most likely breaking change is config.json "target" -> "environment"; that was to resolve some lingering conflicts between our branches, sorry for the override), and left a few questions throughout. If you can comment back on the in-line, I'm confident we can get this into a mergable shape early next week.

conorsch · 2020-02-01T00:58:31Z

dom0/securedrop-admin

+set -u
+set -o pipefail
+
+python3 /usr/share/securedrop-workstation-dom0-config/scripts/securedrop-admin.py "$@"


Why the indirection of this wrapper script? Seems simpler to me to copy scripts/securedrop-admin directly to /usr/bin/seucredrop-admin in the RPM spec, no...

You are correct, indirection is not required here: the first prep-dom0 pass will install the packages (and drop the securedrop-admin script in place). I will update this

conorsch · 2020-02-01T01:01:36Z

rpm-build/SPECS/securedrop-workstation-dom0-config.spec

 install -m 644 dom0/securedrop-update %{buildroot}/srv/salt/
 install -m 644 dom0/securedrop-login %{buildroot}/srv/salt/
 install -m 644 dom0/securedrop-launcher.desktop %{buildroot}/srv/salt/
 install -m 655 dom0/securedrop-handle-upgrade %{buildroot}/srv/salt/
 # The next file should get installed via RPM not via salt
 install -m 755 dom0/securedrop-update %{buildroot}/srv/salt/securedrop-update
+install -m 755 dom0/securedrop-admin %{buildroot}/%{_bindir}/securedrop-admin


Instead of:

install -m 755 dom0/securedrop-admin %{buildroot}/%{_bindir}/securedrop-admin

why not

install -m 755 scripts/securedrop-admin.py %{buildroot}/%{_bindir}/securedrop-admin

? N.B. Haven't tested, mostly questioning the need for the wrapper script indirection, as above.

conorsch · 2020-02-01T01:03:12Z

scripts/securedrop-admin.py

+        )
+    except subprocess.CalledProcessError as e:
+        print(str(e))
+        sys.exit(1)


Why catch, then sys.exit? If we simply allow the exception to be raised, we'll still have the script halt execution, as well as exit non-zero. Does the syntax here prevent preventing the traceback? (Even if it does, not sure it's worth the extra lines.)

conorsch · 2020-02-01T01:04:46Z

scripts/securedrop-admin.py

+    Runs provision-all to apply the salt state.highstate on dom0 and all VMs
+    """
+    try:
+        subprocess.check_call([os.path.join("./scripts/provision-all")])


Indirection, oh my!! Nit: shouldn't there be a SCRIPTS_PATH in the .join()? Would prefer that we integrate provision-all directly into securedrop-admin, but now is not the time... looking forward to circling back and converting to Python Qubes API wherever possible.

conorsch · 2020-02-04T01:57:38Z

Looking good with the conditional logic. The latest changes show a minor test failure, related to the updated prod/test paths for keyfiles:

======================================================================
ERROR: test_rpm_repo_public_key (test_dom0_rpm_repo.SD_Dom0_Rpm_Repo_Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/user/securedrop-workstation/tests/test_dom0_rpm_repo.py", line 33, in test_rpm_repo_public_key
    with open(self.pubkey_actual, "r") as f:
FileNotFoundError: [Errno 2] No such file or directory: '/etc/pki/rpm-gpg/RPM-GPG-KEY-securedrop-workstation'

======================================================================
FAIL: test_rpm_repo_config (test_dom0_rpm_repo.SD_Dom0_Rpm_Repo_Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/user/securedrop-workstation/tests/test_dom0_rpm_repo.py", line 54, in test_rpm_repo_config
    self.assertEqual(found_lines, wanted_lines)
AssertionError: Lists differ: ['[se[104 chars]ation-prod', 'file:///etc/pki/rpm-gpg/RPM-GPG-[152 chars]epo'] != ['[se[104 chars]ation', 'enabled=1', 'baseurl=https://yum-test[80 chars]epo']

First differing element 2:
'gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-securedrop-workstation-prod'
'gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-securedrop-workstation'

First list contains 1 additional elements.
First extra element 6:
'name=SecureDrop Workstation Qubes dom0 repo'

  ['[securedrop-workstation-dom0]',
   'gpgcheck=1',
-  'gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-securedrop-workstation-prod',
?                                                                    -----

+  'gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-securedrop-workstation',
-  'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-securedrop-workstation-test',
   'enabled=1',
   'baseurl=https://yum-test.securedrop.org/workstation/dom0/f25',
   'name=SecureDrop Workstation Qubes dom0 repo']

----------------------------------------------------------------------
Ran 49 tests in 237.949s

FAILED (failures=1, errors=1)
Makefile:114: recipe for target 'test' failed
make: *** [test] Error 1

I believe all that's necessary is a few finishing touches on the salt logic. The conditionals for test/prod state look solid on visual review, it's mostly the corresponding filepaths that should be unique across the board, effectively overturning the simplification in c378758 in favor of using explicit prod/test filepaths.

emkll · 2020-02-04T15:28:55Z

@conorsch I have removed the logic for trusting the prod key in non prod environments. In the changes you've reviewed, I have also omitted changes to sys-firewall. After implementing changes to sys-firewall, it seemed to me like the logic was overly complicated for little benefit (templating the script in /rw/config/ for example, handling the clean-up case), and may introduce unnecessary risk. If we have a compelling reason to use it in the future, we could revisit this.

I think, before merging this, we should focus on the prod-first-install story. Can we get away with not using git to ship software to the dev vm / cloned to dom0?

Using state file includes doesn't appear to expose vars imported via the include to the state performing the include. If this doesn't work for gathering dev/prod vars, we'll likely have to use pillars.

rely on config.json instead of env vars. We can rely on env vars as jinja templates are evaluated before runtime (https://docs.saltstack.com/en/latest/topics/jinja/index.html) Adds production SD release signing key Necessary for the prod logic. We don't have packages hosted in these prod URLs yet. Adds validate check for target config item Determines the "environment", e.g. dev/prod

Using a single filename for the RPM repo configuration, same as for the apt repo, regardless of dev/prod URL within that file.

Script performs one operation:

We've already folded the "apt-test-qubes" Buster-based SDW deb packages into the "apt-test" repository, backed by the dev lfs repo [0]. There's no longer any reason to maintain a Qubes-specific test apt repo; we'll instead reduce the number of apt URLs from 3 to 2: * https://apt-test.freedom.press/ (current default) * https://apt.freedom.press/ (prod, future default) [0] https://github.com/freedomofpress/securedrop-dev-packages-lfs/

The prod endpoints aren't properly configured yet, but at least we're working on it.

Ensure we can load the environment config variable

Only when target is set to staging or prod in the config.json

The cleanup here is mostly an artifact of branch collaboration. Adding back in comments that were lost in rebases, and removing extraneous comments that were left over from debugging.

If environment=dev, then we DON'T want the salt RPM installed in dom0, because it'll conflict with the direct-copy-from-git-repo strategy used by the "dev" environment. So if we're installing from dev, let's make sure that package is absent. We might want to do this earlier in the run, e.g. in the Makefile, but I think running it at all is good enough.

Superificial change. Mostly making the change because we had examples of *both* to due rebase conflicts, and we need one. Given how many occurrences of "target" elsewhere in the repo (e.g. in the qubesctl --targets flag), I argue it's more intuitive to discuss the "environment" as dev/staging/prod.

Staging environment uses CI-built packages, so let's not install those in dom0 unless we're on test-only hardware. Prompt for confirmation, defaulting to "no", if staging env is requested.

- Reduce indirection and direcly provision securedrop-admin via rpm spec - Better exception handling

* Add SCRIPTS_PATH to sys path to re-use validation script * Properly copy config.json to /srv/salt/sd

Left out staging specific variables (key, url) to ensure they are not accidentally installed in production.

emkll · 2020-02-04T18:30:20Z

Rebased on latest master and added docs for the initial provisioning case. This should now be ready for final review

conorsch · 2020-02-05T01:52:28Z

Latest changes look great, particularly the docs additions explaining the first-run steps for fetching, verifying, and installing the RPM. I'm completely satisfied with the changes here, @emkll, so over to you for a formal approval when you're ready!

emkll

Thanks @conorsch , your changes look good to me.

In order to completely test this end-to-end, I've manually built and uploaded a package built on this branch to freedomofpress/securedrop-yum-test#6

Due to the lack of packages on apt.freedom.press and yum.securedrop.org, it will not be a complete test, as the installer will fail on prod scenarios. Given the dependencies on infrastructure (and us actually releasing a beta RC, I think this PR is good to merge for now, since there should be no impact to developers. I'll approve, but not merge, to let others chime in if they disagree

Begins dynamic dev/prod config logic

conorsch requested review from kushaldas and emkll January 25, 2020 02:08

conorsch mentioned this pull request Jan 25, 2020

Use rpm in dom0 for make all #406

Closed

8 tasks

emkll added the release blocker label Jan 30, 2020

conorsch commented Feb 1, 2020

View reviewed changes

emkll force-pushed the 406-prod-make-targets branch from 5feafda to 7d80356 Compare February 4, 2020 15:22

emkll force-pushed the 406-prod-make-targets branch 2 times, most recently from 7edccf4 to b1a6877 Compare February 4, 2020 18:18

Conor Schaefer and others added 16 commits February 4, 2020 13:27

Evaluate: Conditional Salt via includes

24e8432

Using state file includes doesn't appear to expose vars imported via the include to the state performing the include. If this doesn't work for gathering dev/prod vars, we'll likely have to use pillars.

Removes "test" from yum repo filename

24b3281

Using a single filename for the RPM repo configuration, same as for the apt repo, regardless of dev/prod URL within that file.

Adds script to set "environment" in config

42cac48

Script performs one operation:

Provides dev/prod Makefile targets

39c8552

The prod endpoints aren't properly configured yet, but at least we're working on it.

Rename validate config for re-use

ca7a7b2

Add basic securedrop-admin script to handle staging and prod scenarios

1d88089

Add dom/*.yml to manifest and rpm spec

6de1fdf

Ensure we can load the environment config variable

Conditionally install securedrop-workstation-dom0-config

1e65661

Only when target is set to staging or prod in the config.json

Use conditional RPM GPG keyfile for sys-firewall

0d4dc08

Fix tests for split logic for RPM repo and gpg keys

2df12d6

Adds Makefile target for staging

3b76df1

Cleans up config-munging comments in Salt files

c7d5fa3

The cleanup here is mostly an artifact of branch collaboration. Adding back in comments that were lost in rebases, and removing extraneous comments that were left over from debugging.

Conor Schaefer and others added 6 commits February 4, 2020 13:27

Prompts for confirmation if environment=staging

a48d49b

Staging environment uses CI-built packages, so let's not install those in dom0 unless we're on test-only hardware. Prompt for confirmation, defaulting to "no", if staging env is requested.

Address review comments:

b02b172

- Reduce indirection and direcly provision securedrop-admin via rpm spec - Better exception handling

Bump version to 0.1.2

5392a04

Fix securedrop-admin script

44570ec

* Add SCRIPTS_PATH to sys path to re-use validation script * Properly copy config.json to /srv/salt/sd

Add docs for bootstrapping staging and prod environments

dc7daf7

Left out staging specific variables (key, url) to ensure they are not accidentally installed in production.

Remove legacy usb autoattach config

7b90662

emkll force-pushed the 406-prod-make-targets branch from b1a6877 to 7b90662 Compare February 4, 2020 18:28

emkll mentioned this pull request Feb 5, 2020

Adds 0.1.2 dom0-config package with split dev/prod logic freedomofpress/securedrop-yum-test#6

Merged

emkll approved these changes Feb 5, 2020

View reviewed changes

conorsch merged commit 2c45093 into master Feb 5, 2020

This was referenced Feb 5, 2020

Enforce dom0 salt state as part of preflight updates #427

Closed

Fix split logic tests in dev, bump kernel version #443

Merged

cfm pushed a commit that referenced this pull request Apr 1, 2024

Merge pull request #432 from freedomofpress/406-prod-make-targets

0236b40

Begins dynamic dev/prod config logic

legoktm deleted the 406-prod-make-targets branch May 28, 2024 15:25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Begins dynamic dev/prod config logic #432

Begins dynamic dev/prod config logic #432

conorsch commented Jan 25, 2020 •

edited

Loading

emkll commented Jan 28, 2020

conorsch left a comment

conorsch Feb 1, 2020

emkll Feb 3, 2020

conorsch Feb 1, 2020

conorsch Feb 1, 2020

conorsch Feb 1, 2020

conorsch commented Feb 4, 2020

emkll commented Feb 4, 2020

emkll commented Feb 4, 2020

conorsch commented Feb 5, 2020

emkll left a comment

Begins dynamic dev/prod config logic #432

Begins dynamic dev/prod config logic #432

Conversation

conorsch commented Jan 25, 2020 • edited Loading

Status

Description of Changes

Testing

dev target

staging target

Checklist

If you have made changes to the provisioning logic

emkll commented Jan 28, 2020

conorsch left a comment

Choose a reason for hiding this comment

conorsch Feb 1, 2020

Choose a reason for hiding this comment

emkll Feb 3, 2020

Choose a reason for hiding this comment

conorsch Feb 1, 2020

Choose a reason for hiding this comment

conorsch Feb 1, 2020

Choose a reason for hiding this comment

conorsch Feb 1, 2020

Choose a reason for hiding this comment

conorsch commented Feb 4, 2020

emkll commented Feb 4, 2020

emkll commented Feb 4, 2020

conorsch commented Feb 5, 2020

emkll left a comment

Choose a reason for hiding this comment

conorsch commented Jan 25, 2020 •

edited

Loading