make build-debs target is failing during dpkg-source command

[zenmonkeykstop]

Description

make build-debs and make build-debs-notest runs are failing at the following command:

 dpkg-source --before-build securedrop-app-code_1.3.0~rc1+xenial_amd64\n debian/rules clean\n dpkg-source -b securedrop-app-code_1.3.0~rc1+xenial_amd64

Steps to Reproduce

Run make build-debs against current develop branch

Expected Behavior

Build completes successfully.

Actual Behavior

Build fails at command above.

Comments

@emkll recommends the following role changes for debugging:

diff --git a/install_files/ansible-base/roles/build-securedrop-app-code-deb-pkg/tasks/main.yml b/install_files/ansible-base/roles/build-securedrop-app-code-deb-pkg/tasks/main.yml
index b639d34c7..d696ed2b5 100644
--- a/install_files/ansible-base/roles/build-securedrop-app-code-deb-pkg/tasks/main.yml
+++ b/install_files/ansible-base/roles/build-securedrop-app-code-deb-pkg/tasks/main.yml
@@ -31,6 +31,12 @@
 
 - include: sass.yml
 
+- name: upgrade all packages in case unlikely bug in a system dependency
+  shell: |
+    sudo apt update && sudo apt -y upgrade
+  tags:
+    - pip
+
 - name: Install SecureDrop Python requirements in container
   shell: |
     pip3 install --no-binary :all: --require-hashes -r {{ securedrop_app_code_prep_dir }}/requirements.txt
@@ -100,6 +106,13 @@
     - "{{ securedrop_app_code_deb_dir }}/var/www/securedrop/static/.webassets-cache"
     - "{{ securedrop_app_code_deb_dir }}/var/www/securedrop/.well-known/pki-validation"
 
+- name: Sleep for 60 seconds and timeout
+    wait_for:
+      delay: 3600
+
+# docker exec -it <container id> /bin/bash
+# Edit files/run the build command and debug
+
 - name: Build securedrop-app-code Debian package
   command: dpkg-buildpackage -us -uc
   args:
diff --git a/molecule/builder-xenial/molecule.yml b/molecule/builder-xenial/molecule.yml
index f8f0f4670..4db0cd0b9 100644
--- a/molecule/builder-xenial/molecule.yml
+++ b/molecule/builder-xenial/molecule.yml
@@ -7,27 +7,28 @@ platforms:
   - name: xenial-sd-app
     groups:
       - builders
-  - name: xenial-sd-generic-ossec-agent
-    groups:
-      - builders
-  - name: xenial-sd-generic-ossec-server
-    groups:
-      - builders
-  - name: xenial-sd-generic-ossec-agent2
-    groups:
-      - builders
-  - name: xenial-sd-generic-ossec-server2
-    groups:
-      - builders
-  - name: xenial-sd-grsec
-    groups:
-      - builders
-  - name: xenial-sd-config
-    groups:
-      - builders
-  - name: xenial-sd-keyring
-    groups:
-      - builders
+  # we can comment these out for performance reasons for debugging
+  # - name: xenial-sd-generic-ossec-agent
+  #   groups:
+  #     - builders
+  # - name: xenial-sd-generic-ossec-server
+  #   groups:
+  #     - builders
+  # - name: xenial-sd-generic-ossec-agent2
+  #   groups:
+  #     - builders
+  # - name: xenial-sd-generic-ossec-server2
+  #   groups:
+  #     - builders
+  # - name: xenial-sd-grsec
+  #   groups:
+  #     - builders
+  # - name: xenial-sd-config
+  #   groups:
+  #     - builders
+  # - name: xenial-sd-keyring
+  #   groups:
+  #     - builders
   - name: xenial-sd-sec-update
     groups:
       - builders

[zenmonkeykstop]

This is probably related (note that wheel 0.34.1 was released today):

    Downloading wheel-0.34.1.tar.gz (55 kB)
    Ignoring unknown cache-control directive: immutable
    Updating cache with response from \"https://files.pythonhosted.org/packages/f9/85/120a5ad1565398efb1184eacaa597e925abbf8fff8cc748877add71257f1/wheel-0.34.1.tar.gz\"
    Caching due to etag
  Cleaning up...
    Removing source in /tmp/pip-install-k3iggafc/setuptools
  Removed build tracker: '/tmp/pip-req-tracker-e53o5a42'
  ERROR: Exception:
  Traceback (most recent call last):
    File \"/tmp/securedrop-app-code_1.3.0~rc1+xenial_amd64/debian/securedrop-app-code/opt/venvs/securedrop-app-code/lib/python3.5/site-packages/pip/_internal/cli/base_command.py\", line 186, in _main
      status = self.run(options, args)
    File \"/tmp/securedrop-app-code_1.3.0~rc1+xenial_amd64/debian/securedrop-app-code/opt/venvs/securedrop-app-code/lib/python3.5/site-packages/pip/_internal/commands/install.py\", line 331, in run
      resolver.resolve(requirement_set)
    File \"/tmp/securedrop-app-code_1.3.0~rc1+xenial_amd64/debian/securedrop-app-code/opt/venvs/securedrop-app-code/lib/python3.5/site-packages/pip/_internal/legacy_resolve.py\", line 177, in resolve
      discovered_reqs.extend(self._resolve_one(requirement_set, req))
    File \"/tmp/securedrop-app-code_1.3.0~rc1+xenial_amd64/debian/securedrop-app-code/opt/venvs/securedrop-app-code/lib/python3.5/site-packages/pip/_internal/legacy_resolve.py\", line 333, in _resolve_one
      abstract_dist = self._get_abstract_dist_for(req_to_install)
    File \"/tmp/securedrop-app-code_1.3.0~rc1+xenial_amd64/debian/securedrop-app-code/opt/venvs/securedrop-app-code/lib/python3.5/site-packages/pip/_internal/legacy_resolve.py\", line 282, in _get_abstract_dist_for
      abstract_dist = self.preparer.prepare_linked_requirement(req)
    File \"/tmp/securedrop-app-code_1.3.0~rc1+xenial_amd64/debian/securedrop-app-code/opt/venvs/securedrop-app-code/lib/python3.5/site-packages/pip/_internal/operations/prepare.py\", line 482, in prepare_linked_requirement
      hashes=hashes,
    File \"/tmp/securedrop-app-code_1.3.0~rc1+xenial_amd64/debian/securedrop-app-code/opt/venvs/securedrop-app-code/lib/python3.5/site-packages/pip/_internal/operations/prepare.py\", line 287, in unpack_url
      hashes=hashes,
    File \"/tmp/securedrop-app-code_1.3.0~rc1+xenial_amd64/debian/securedrop-app-code/opt/venvs/securedrop-app-code/lib/python3.5/site-packages/pip/_internal/operations/prepare.py\", line 164, in unpack_http_url
      unpack_file(from_path, location, content_type)
    File \"/tmp/securedrop-app-code_1.3.0~rc1+xenial_amd64/debian/securedrop-app-code/opt/venvs/securedrop-app-code/lib/python3.5/site-packages/pip/_internal/utils/unpacking.py\", line 261, in unpack_file
      untar_file(filename, location)
    File \"/tmp/securedrop-app-code_1.3.0~rc1+xenial_amd64/debian/securedrop-app-code/opt/venvs/securedrop-app-code/lib/python3.5/site-packages/pip/_internal/utils/unpacking.py\", line 222, in untar_file
      with open(path, 'wb') as destfp:
  UnicodeEncodeError: 'ascii' codec can't encode characters in position 72-74: ordinal not in range(128)

[zenmonkeykstop]

This looks to be the offending pip command:

  Running command '/tmp/securedrop-app-code_1.3.0~rc1+xenial_amd64/debian/securedrop-app-code/opt/venvs/securedrop-app-code/bin/python' '/tmp/securedrop-app-code_1.3.0~rc1+xenial_amd64/debian/securedrop-app-code/opt/venvs/securedrop-app-code/lib/python3.5/site-packages/pip' install --ignore-installed --no-user --prefix /tmp/pip-build-env-tzw3qhas/overlay --no-warn-script-location -v --no-binary :all: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=18.5' wheel 'cffi>=1.8,!=1.11.3; python_implementation != '\"'\"'PyPy'\"'\"''

[zenmonkeykstop]

Fixed with the wheel release addressing the upstream issue here: pypa/wheel#331