Test built Debian packages #1785

legoktm · 2024-02-06T15:17:13Z

Description

We should run various tests against built Debian packages. Previously the builder repository ran a very trivial test that verified unpacking and the conffiles hack (https://github.com/freedomofpress/securedrop-builder/blob/ace977adaff920e70756aa2e8e9c773f956db4a5/tests/test_deb_package.py), but we have more useful ones in the server repo: https://github.com/freedomofpress/securedrop/blob/develop/builder/tests/test_securedrop_deb_package.py that should be used here.

piuparts tests package installation, upgrade and removal. We don't care that much about the last one so it's just a warning for now. For each package, piuparts will install the package CI just built and ensure that goes smoothly, making sure all the dependencies are present and the postinst works. Issues that were found are being fixed in the next commit. At a later stage, once the released packages are passing piuparts too, we can have it automatically grab the latest package from apt.freedom.press, install that, and then test the upgrade path to the new CI-built packages. This CI setup relies on running docker-in-docker, which is hacky but seems to work. Unfortunately it doesn't work for the client (starts apparmor), workstation-config (starts systemd) and workstation-viewer (pulls in grsec kernel). In the future we can look at switching into a chrooted setup that might fix the last two. Getting this to work in GitHub Actions was inspired by <https://github.com/evgeni/action-piuparts/> but I didn't really copy any code. Refs #1785.

legoktm · 2024-02-16T22:17:32Z

For upstream tooling we can reuse:

piuparts: Have CI run piuparts #1844, which is less useful in a container than I expected.
lintian, which spotted a number of things. Going to set up a PR for this.

lintian is the Debian package linter and catches a number of things we can improve in our packages. For now I've suppressed everything and annotated them with TODO/FIXME for others to take on or address incrementally. lintian is run at the end of `make build-debs` unless the "FAST" environment variable is set. Any warning or error causes the build to fail, but it can be suppressed in the corresponding lintian-overrides file. Refs #1785.

lintian is the Debian package linter and catches a number of things we can improve in our packages. For now I've suppressed everything and annotated them with TODO/FIXME for others to take on or address incrementally. lintian is run at the end of `make build-debs` unless the "FAST" environment variable is set. Any warning or error causes the build to fail, but it can be suppressed in the corresponding lintian-overrides file. Internally there's a bit of work so we can identify the *.changes file that we just built, so we first build into a temporary directory, copy the files into our repository's build, then run lintian in the temporary directory. Refs #1785.

piuparts tests package installation, upgrade and removal. We don't care that much about the last one so it's just a warning for now. For each package, piuparts will install the package CI just built and ensure that goes smoothly, making sure all the dependencies are present and the postinst works. Issues that were found are being fixed in the next commit. At a later stage, once the released packages are passing piuparts too, we can have it automatically grab the latest package from apt.freedom.press, install that, and then test the upgrade path to the new CI-built packages. This CI setup relies on running docker-in-docker, which is hacky but seems to work. Unfortunately it doesn't work for the client (starts apparmor), workstation-config (starts systemd) and workstation-viewer (pulls in grsec kernel). In the future we can look at switching into a chrooted setup that might fix the last two. Getting this to work in GitHub Actions was inspired by <https://github.com/evgeni/action-piuparts/> but I didn't really copy any code. Refs #1785.

lintian is the Debian package linter and catches a number of things we can improve in our packages. For now I've suppressed everything and annotated them with TODO/FIXME for others to take on or address incrementally. lintian is run at the end of `make build-debs` unless the "FAST" environment variable is set. Any warning or error causes the build to fail, but it can be suppressed in the corresponding lintian-overrides file. Internally there's a bit of work so we can identify the *.changes file that we just built, so we first build into a temporary directory, copy the files into our repository's build, then run lintian in the temporary directory. Refs #1785.

piuparts tests package installation, upgrade and removal. We don't care that much about the last one so it's just a warning for now. For each package, piuparts will install the package CI just built and ensure that goes smoothly, making sure all the dependencies are present and the postinst works. Issues that were found are being fixed in the next commit. At a later stage, once the released packages are passing piuparts too, we can have it automatically grab the latest package from apt.freedom.press, install that, and then test the upgrade path to the new CI-built packages. This CI setup relies on running docker-in-docker, which is hacky but seems to work. Unfortunately it doesn't work for the client (starts apparmor), workstation-config (starts systemd) and workstation-viewer (pulls in grsec kernel). In the future we can look at switching into a chrooted setup that might fix the last two. Getting this to work in GitHub Actions was inspired by <https://github.com/evgeni/action-piuparts/> but I didn't really copy any code. Refs #1785.

legoktm · 2024-02-29T01:29:31Z

One of the things I want to explore is running the tests against the installed Debian package. Because we use a virtualenv, this actually makes it straightforward to install the dev dependencies in:

root@b028701e9ffc:/src/build# apt install ./securedrop-client_0.9.0+bookworm_all.deb python3-poetry xvfb sqlite3
...
root@b028701e9ffc:/src/client# source /opt/venvs/securedrop-client/bin/activate
(securedrop-client) root@b028701e9ffc:/src/client# poetry env info

Virtualenv
Python:         3.11.2
Implementation: CPython
Path:           /opt/venvs/securedrop-client
Executable:     /opt/venvs/securedrop-client/bin/python
Valid:          True

System
Platform:   linux
OS:         posix
Python:     3.11.2
Path:       /usr
Executable: /usr/bin/python3.11

Then install the dependencies, without installing the root package:

(securedrop-client) root@b028701e9ffc:/src/client# poetry install --only=dev --no-root
Installing dependencies from lock file

Package operations: 74 installs, 11 updates, 0 removals
...

Now the trick is we run the tests from a different directory, so that way the local securedrop_client package doesn't override the venv-installed one.

(securedrop-client) root@b028701e9ffc:/src/client# cd ../
(securedrop-client) root@b028701e9ffc:/src# xvfb-run -a pytest -v client/tests/ --ignore=client/tests/functional/ --ignore=client/tests/integration/
============================================================================================================ test session starts =============================================================================================================
platform linux -- Python 3.11.2, pytest-7.4.3, pluggy-1.3.0 -- /opt/venvs/securedrop-client/bin/python
cachedir: .pytest_cache
Test order randomisation NOT enabled. Enable with --random-order or --random-order-bucket=<bucket_type>
PyQt5 5.15.8 -- Qt runtime 5.15.2 -- Qt compiled 5.15.2
rootdir: /src/client
configfile: pytest.ini
plugins: xdist-3.3.1, random-order-1.1.0, qt-4.2.0, vcr-1.0.2, mock-3.12.0, cov-4.1.0, flaky-3.7.0
collected 892 items                                                                                                                                                                                                                          

client/tests/test_alembic.py::test_alembic_head_matches_db_models PASSED                                                                                                                                                               [  0%]
client/tests/test_alembic.py::test_alembic_migration_upgrade[d7c8af95bc8e] PASSED                                                                                                                                                      [  0%]
client/tests/test_alembic.py::test_alembic_migration_upgrade[414627c04463] PASSED                                                                                                                                                      [  0%]
client/tests/test_alembic.py::test_alembic_migration_upgrade_with_data[d7c8af95bc8e] PASSED                                                                                                                                            [  0%]

This means the tests need to support being run from a different directory, which some don't yet.

I'm not sure yet how this fits into the whole big picture, are we going to be running the tests in yet another variation? Or would this supplant our normal test runs? How straightforward do we make this to run locally?

legoktm · 2024-03-05T00:01:13Z

cross-linking Without MANIFEST.in, client component does not build a working package #1874 as a regression that we should be able to prevent with appropriate tests

lintian is the Debian package linter and catches a number of things we can improve in our packages. For now I've suppressed everything and annotated them with TODO/FIXME for others to take on or address incrementally. lintian is run at the end of `make build-debs` unless the "FAST" environment variable is set. Any warning or error causes the build to fail, but it can be suppressed in the corresponding lintian-overrides file. Internally there's a bit of work so we can identify the *.changes file that we just built, so we first build into a temporary directory, copy the files into our repository's build, then run lintian in the temporary directory. Refs #1785.

lintian is the Debian package linter and catches a number of things we can improve in our packages. For now I've suppressed everything and annotated them with TODO/FIXME for others to take on or address incrementally. lintian is run at the end of `make build-debs` unless the "FAST" environment variable is set. Any warning or error causes the build to fail, but it can be suppressed in the corresponding lintian-overrides file. A number of file and directories that may exist in the tree are excluded from the package build process through d/source/options to prevent lintian from seeing them. Internally there's a bit of work so we can identify the *.changes file that we just built, so we first build into a temporary directory, copy the files into our repository's build, then run lintian in the temporary directory. Refs #1785.

legoktm mentioned this issue Feb 16, 2024

Have CI run piuparts #1844

Merged

2 tasks

legoktm mentioned this issue Feb 19, 2024

Run lintian on Debian packages #1845

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Test built Debian packages #1785

Test built Debian packages #1785

legoktm commented Feb 6, 2024

legoktm commented Feb 16, 2024

legoktm commented Feb 29, 2024

legoktm commented Mar 5, 2024

Test built Debian packages #1785

Test built Debian packages #1785

Comments

legoktm commented Feb 6, 2024

Description

legoktm commented Feb 16, 2024

legoktm commented Feb 29, 2024

legoktm commented Mar 5, 2024