Fixes #205 installs libreoffice in svs disp #263

kushaldas · 2019-06-03T14:54:32Z

Might take time due to high amount of download for libreoffice.

How to test?

Remove old sd-svs-disp qvm-remove sd-svs-disp
Remove old sd-svs-disp template qvm-remove sd-svs-disp-template
make sd-svs-disp

emkll · 2019-06-03T16:14:06Z

Can confirm LibreOffice is properly installed. However, when opening LibreOffice writer, a few grsecurity errors can be observed in syslog:

Jun  3 12:02:03 localhost kernel: [   26.393123] grsec: denied RWX mmap of <anonymous mapping> by /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java[java:995] uid/euid:1000/1000 gid/egid:1000/1000, parent /usr/lib/libreoffice/program/javaldx[osl_executeProc:993] uid/euid:1000/1000 gid/egid:1000/1000
Jun  3 12:02:03 localhost kernel: [   26.409745] grsec: denied RWX mmap of <anonymous mapping> by /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java[java:1000] uid/euid:1000/1000 gid/egid:1000/1000, parent /usr/lib/libreoffice/program/javaldx[osl_executeProc:997] uid/euid:1000/1000 gid/egid:1000/1000
Jun  3 12:02:03 localhost kernel: [   26.618611] grsec: denied RWX mprotect of <anonymous mapping> by /usr/lib/libreoffice/program/soffice.bin[soffice.bin:1002] uid/euid:1000/1000 gid/egid:1000/1000, parent /usr/lib/libreoffice/program/oosplash[osl_executeProc:1001] uid/euid:1000/1000 gid/egid:1000/1000
Jun  3 12:02:03 localhost qubes-gui[542]: XGetWindowAttributes for 0x1200001 failed in handle_crossing, ret=0x0
Jun  3 12:02:32 localhost kernel: [   55.874278] grsec: denied RWX mmap of <anonymous mapping> by /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java[java:1050] uid/euid:1000/1000 gid/egid:1000/1000, parent /usr/lib/libreoffice/program/javaldx[osl_executeProc:1047] uid/euid:1000/1000 gid/egid:1000/1000
Jun  3 12:02:32 localhost kernel: [   55.889528] grsec: denied RWX mmap of <anonymous mapping> by /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java[java:1053] uid/euid:1000/1000 gid/egid:1000/1000, parent /usr/lib/libreoffice/program/javaldx[osl_executeProc:1051] uid/euid:1000/1000 gid/egid:1000/1000
Jun  3 12:02:32 localhost kernel: [   56.091364] grsec: denied RWX mprotect of <anonymous mapping> by /usr/lib/libreoffice/program/soffice.bin[soffice.bin:1056] uid/euid:1000/1000 gid/egid:1000/1000, parent /usr/lib/libreoffice/program/oosplash[osl_executeProc:1055] uid/euid:1000/1000 gid/egid:1000/1000

As a test for this PR, I submitted a .docx file to a SecureDrop instance and tried opening it in the client.

It seems like the MIME type is not properly handled in sd-svs-disp: handlers are not properly:

kushaldas · 2019-06-03T16:22:52Z

@emkll Yup, I will submit a proper PR in the paxctl.conf in the other repo so that the grsec errors go away. Thanks for confirming that error.

The error was mentioned at freedomofpress/securedrop-workstation#263 (comment)

emkll

I've proceeded with a full review using the svs-disp config package provided by https://github.com/freedomofpress/securedrop-debian-packaging/pull/47/files.

I submitted a .docx file to the source interface of a test SecureDrop instance and attempted to open it in the SecureDrop workstation. I've experienced 2 issues:

The package ca-certificates-java is not fully installed (likely due to paxctld, see investigation in 1) below)
MIME handling does not work and does not open LibreOffice, likely because the extension is missing (see investigation in 2) below)

1. In sd-svs-disp, apt reports that a package is not fully installed:

1 not fully installed or removed.

It's an issue with ca-certificates-java:

sudo dpkg --status ca-certificates-java
Package: ca-certificates-java
Status: install ok half-configured

It seems like the paxflag was not applied to java?

Jun  4 15:40:38 localhost kernel: [ 1309.201730] grsec: denied RWX mmap of <anonymous mapping> by /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java[java:2147] uid/euid:0/0 gid/egid:0/0, parent /var/lib/dpkg/info/ca-certificates-java.postinst[ca-certificates:2133] uid/euid:0/0 gid/egid:0/0
Jun  4 15:40:39 localhost kernel: [ 1310.117476] grsec: denied RWX mmap of <anonymous mapping> by /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java[java:2945] uid/euid:0/0 gid/egid:0/0, parent /etc/ca-certificates/update.d/jks-keystore[jks-keystore:2926] uid/euid:0/0 gid/egid:0/0
Jun  4 17:12:40 localhost kernel: [ 6830.956819] grsec: denied RWX mmap of <anonymous mapping> by /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java[java:3114] uid/euid:0/0 gid/egid:0/0, parent /var/lib/dpkg/info/ca-certificates-java.postinst[ca-certificates:3100] uid/euid:0/0 gid/egid:0/0
Jun  4 17:12:41 localhost kernel: [ 6832.077619] grsec: denied RWX mmap of <anonymous mapping> by /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java[java:3910] uid/euid:0/0 gid/egid:0/0, parent /etc/ca-certificates/update.d/jks-keystore[jks-keystore:3891] uid/euid:0/0 gid/egid:0/0

This might be because paxctld is dead, and the paxflag wasn't applied:

user@sd-svs-disp:~$ sudo service paxctld status
● paxctld.service - PaX flags maintenance daemon
   Loaded: loaded (/lib/systemd/system/paxctld.service; disabled; vendor preset:
   Active: inactive (dead)

2. Unfortunately a doc submission does not get opened by libreoffice, but rather by the archive manager as shown in #263 (comment) .

This is because xdg-open relies on the file extension and not magic numbers (file appears to rely on a file's magic numbers)

user@sd-svs-disp:$ xdg-mime query filetype 1-single-celled_authorship-doc 
application/zip
user@sd-svs-disp:$ file --mime-type 1-single-celled_authorship-doc 
1-single-celled_authorship-doc: application/vnd.openxmlformats-officedocument.wordprocessingml.document

Changing the extension does change the association:

user@sd-svs-disp:$ mv 1-single-celled_authorship-doc 1-single-celled_authorship-doc.doc
user@sd-svs-disp:$ xdg-mime query filetype 1-single-celled_authorship-doc.doc 
application/msword

We could either:
a) Write a script that receives the file from sd-svs, looks at magic numbers and opens it with the correct application, or
b) See if xdg-mime supports using only magic numbers (however preliminary research came back empty so far)
c) Add extensions to submissions on the SecureDrop server code

Happy to hear what other's thoughts on this are, it sound like a) would be the most reliable approach and would allow us to control what files are and aren't supported

emkll · 2019-06-06T15:41:35Z

can you confirm that after the following steps:

make remove-sd-svs-disp
qvm-remove sd-svs-disp-template
make sd-svs-disp

the output of sudo dpkg --status ca-certificates-java in sd-svs-disp after running is as follows?

Package: ca-certificates-java
Status: install ok installed

It still shows as half-configured for me on the latest commit (9e2e5e3). apt-get -f install resolves locally in the template for me, but requires manual user intervention

kushaldas · 2019-06-06T15:47:43Z

I have the following:

$ sudo dpkg --status ca-certificates-java
Package: ca-certificates-java
Status: install ok installed
Priority: optional
Section: java
Installed-Size: 40
Maintainer: Debian Java Maintainers <[email protected]>
Architecture: all
Multi-Arch: foreign
Version: 20170929~deb9u3
Depends: ca-certificates (>= 20121114), openjdk-7-jre-headless | java7-runtime-headless, libnss3 (>= 3.12.10-2~)
Conffiles:
 /etc/ca-certificates/update.d/jks-keystore 6365dfbb739b0b3e522ec39eeba0b80e
 /etc/default/cacerts 0ded97abeff69c2362939e2e881e214a
Description: Common CA certificates (JKS keystore)
 This package uses the hooks of the ca-certificates package to update the
 cacerts JKS keystore used for many java runtimes.

emkll

Thanks @kushaldas for your patience!

I have pushed 934b48c to sidestep/solve the first issue brought up in the previous review (In sd-svs-disp, apt reports that a package is not fully installed: ca-certificates-java) . I've also added a bit ensuring order of operation between paxctld service running and installing LibreOffice, just in case.
I have also opened #266 to track the second item of the previous review (doc submission does not get opened by libreoffice, but rather by the archive manager)

Please review my changes and merge/comment, otherwise LGTM!

emkll · 2019-06-06T20:33:38Z

Also noticed the sd-svs-disp tests weren't all running, pushed d08eb8d to resolve, and added a test for paxctld running

Also restarts and enables paxctl service

…ffice Setting the paxctld in the requires block of the libreoffice install will ensure the service is always started prior to installing the package. ca-certificates-java was posing some issues at install time. As it is listed in "recommends" and not "requires", we can sidestep the issue and also reduce total amount of packages installed.

Some sd-svs-disp tests were not running, moving sd-svs-disp tests to their own file resolves.

eloquence · 2019-06-06T22:54:14Z

c) Add extensions to submissions on the SecureDrop server code

As far as I can tell, extensions and filenames are supported just fine, the client just doesn't extract them right now (freedomofpress/securedrop-client#163). Test this yourself by decompressing a submission with gzip using the -N parameter (otherwise the old base name will be used).

Given this, will this issue be implicitly resolved once we resolve freedomofpress/securedrop-client#163 ?

Used to determine whether paxctld is running inside the VMs. The most important is sd-svs-disp, since that's where we'll be installing libreoffice, and libreoffice requires pax flags to be set on java dependencies in order for the package to install successfully. Removes the previous implementation of a service check in favor of a more generalized approach. Overall, the config test suite should probably be reorganized.

Using Salt to force the service to run, so it's active immediately. More important for the child VMs is using the qvm-service calls to enable the service on boot of other, non-Template but Template-based VMs. Since we're now properly enforcing the paxctld service state in all VMs, removes the previous implementation specific to sd-svs-disp.

conorsch

Solid improvements here. Deferring discussion on the mimetype behavior to #266. With a few more test tweaks, this is ready to merge. As it stands, I can confirm that libreoffice is installed without issue. Will follow up with another test or two, then pass to @kushaldas for final sign-off before merge.

conorsch · 2019-06-06T23:18:03Z

dom0/sd-svs-disp-files.sls

+paxctld:
+  service.running:
+    - enable: True
+    - reload: True


The task should have a more explicit name, as all tasks exist in the same namespace once the various states have been merged. Compare with the task below: sd-svs-disp-install-libreoffice, rather than libreoffice.

More relevant for predictable service state is that we need to inform Qubes (via dom0) that a given VM should start a service. See description in #231, specifically the mention of qvm-service. Following that approach will not only resolve concerns about the libreoffice install in this PR, but also allow us to resolve #231, since we'll be managed the service on all the requisite VMs, not just one.

conorsch · 2019-06-06T23:28:34Z

tests/test_svs_disp.py

+
+    def test_sd_client_package_installed(self):
+        pkg = "securedrop-workstation-svs-disp"
+        self.assertTrue(self._package_is_installed(pkg))


Great catch on re-enabling these tests! Some of the test names still refer to sd-svs (specifically test_sd_client_package_installed), will clean up for clarity.

conorsch · 2019-06-06T23:29:18Z

dom0/sd-svs-disp-files.sls

+        interval: 60
+    - install_recommends: False
+    - require:
+      - service: paxctld


Given requested changes to task name, this requirement name should be updated. Better yet, if we ensure paxctld is running in a parent state, then we needn't redeclare it here.

conorsch · 2019-06-06T23:30:36Z

tests/test_svs_disp.py

+        cmd = ["qvm-run", "-p", "sd-svs-disp",
+               "/usr/sbin/service paxctld status"]
+        p = subprocess.check_output(cmd)
+        self.assertTrue("active (running)".encode() in p)


👍 on checking the service in the config states. Although we've only been bitten by PaX flags on sd-svs-disp so far, we in fact what the paxctld service running in other VMs, as well (see #231). Will adjust the test to run against other machines, as well.

conorsch · 2019-06-06T23:32:05Z

dom0/sd-svs-disp-files.sls

+    - retry:
+        attempts: 3
+        interval: 60
+    - install_recommends: False


Given that the install_recommends: False sidesteps the PaX flags problem, do we still need the retry logic? Doesn't hurt to keep it in, but it seems like we resolve the problem without resorting to retries.

conorsch · 2019-06-07T00:00:28Z

Will also rebase on latest master (7729205), to include the recently merged #259, which had a substantial diff, and new tests.

conorsch

Able to run libreoffice just fine now. With the new paxctld service logic, merge of this PR will close #231, as well as close #205, as original intended.

@kushaldas Please have a look, since @emkll and I both pushed changes here. Fine to merge if you agree!

kushaldas · 2019-06-07T11:51:16Z

@conorsch @emkll Thanks for the pushes, these look fine to me. 🦄 🌈

kushaldas requested a review from conorsch June 3, 2019 14:54

kushaldas added a commit to freedomofpress/securedrop-builder that referenced this pull request Jun 4, 2019

Fixes libreoffice grsec error

d1d6827

The error was mentioned at freedomofpress/securedrop-workstation#263 (comment)

kushaldas mentioned this pull request Jun 4, 2019

Fixes libreoffice grsec error freedomofpress/securedrop-builder#47

Closed

emkll suggested changes Jun 4, 2019

View reviewed changes

kushaldas force-pushed the svs_disp_libre branch from 6b3d0a3 to 9e2e5e3 Compare June 5, 2019 12:22

emkll mentioned this pull request Jun 6, 2019

Open LibreOffice files in sd-svs-disp VMs #266

Closed

emkll approved these changes Jun 6, 2019

View reviewed changes

kushaldas and others added 3 commits June 6, 2019 15:34

Fixes #205 installs libreoffice in svs disp

c435ac6

Also restarts and enables paxctl service

Fix svs_disp tests and add test for paxctld

16d4446

Some sd-svs-disp tests were not running, moving sd-svs-disp tests to their own file resolves.

Conor Schaefer added 2 commits June 6, 2019 16:19

conorsch reviewed Jun 6, 2019

View reviewed changes

conorsch force-pushed the svs_disp_libre branch from d08eb8d to 5b548ad Compare June 7, 2019 00:02

conorsch approved these changes Jun 7, 2019

View reviewed changes

kushaldas merged commit b49c5f2 into master Jun 7, 2019

kushaldas deleted the svs_disp_libre branch June 7, 2019 11:51

emkll mentioned this pull request Jun 25, 2019

paxctl service is not launched on startup in sd-workstation-based VMs #231

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fixes #205 installs libreoffice in svs disp #263

Fixes #205 installs libreoffice in svs disp #263

kushaldas commented Jun 3, 2019

emkll commented Jun 3, 2019

kushaldas commented Jun 3, 2019

emkll left a comment

emkll commented Jun 6, 2019

kushaldas commented Jun 6, 2019

emkll left a comment

emkll commented Jun 6, 2019

eloquence commented Jun 6, 2019 •

edited

Loading

conorsch left a comment

conorsch Jun 6, 2019

conorsch Jun 6, 2019

conorsch Jun 6, 2019

conorsch Jun 6, 2019

conorsch Jun 6, 2019

conorsch commented Jun 7, 2019

conorsch left a comment

kushaldas commented Jun 7, 2019

Fixes #205 installs libreoffice in svs disp #263

Fixes #205 installs libreoffice in svs disp #263

Conversation

kushaldas commented Jun 3, 2019

How to test?

emkll commented Jun 3, 2019

kushaldas commented Jun 3, 2019

emkll left a comment

Choose a reason for hiding this comment

1. In sd-svs-disp, apt reports that a package is not fully installed:

2. Unfortunately a doc submission does not get opened by libreoffice, but rather by the archive manager as shown in #263 (comment) .

emkll commented Jun 6, 2019

kushaldas commented Jun 6, 2019

emkll left a comment

Choose a reason for hiding this comment

emkll commented Jun 6, 2019

eloquence commented Jun 6, 2019 • edited Loading

conorsch left a comment

Choose a reason for hiding this comment

conorsch Jun 6, 2019

Choose a reason for hiding this comment

conorsch Jun 6, 2019

Choose a reason for hiding this comment

conorsch Jun 6, 2019

Choose a reason for hiding this comment

conorsch Jun 6, 2019

Choose a reason for hiding this comment

conorsch Jun 6, 2019

Choose a reason for hiding this comment

conorsch commented Jun 7, 2019

conorsch left a comment

Choose a reason for hiding this comment

kushaldas commented Jun 7, 2019

eloquence commented Jun 6, 2019 •

edited

Loading