Release SecureDrop Client 0.2.1 #1125

eloquence · 2020-07-08T04:05:19Z

The following PRs (excluding docs and dev-only changes) have landed since 0.2.0:

Prevent addition of duplicated api jobs (app: generic approach to prevent addition of duplicated api jobs #975)
Move CSS to CSS files and add new tests for stylesheets (Move CSS to CSS files and add new tests for stylesheets #1082, fix placeholder label->widget #1112)
Delete source widgets for current sync update (delete source widgets for current sync update #1114, fixes Source list doesn't update right away when you delete multiple sources #1113, client crashes when sources are reordered #1116, add-source scheduler crashes app because of unhandled exception #1117)
Reduce minimum size (reduce minimum size #1103, fixes Support 1366x768 screen resolution #1074)

Now is a good time to prepare another release. Per SemVer I would call this a patch-level release since it comprises only bugfixes / internal changes, hence 0.2.1 as the proposed release version.

Rough breakdown of the release process (borrowing from freedomofpress/securedrop-workstation#547):

Define test plan based on changelogs in the above PRs, link to test plan from here.
Perform QA using nightly packages (QA reports should be posted as responses to this ticket.)
Final package release
Let pilot orgs know of any fixes that are relevant for them

The text was updated successfully, but these errors were encountered:

eloquence · 2020-07-09T22:26:03Z

Environment type: Qubes staging
Client version: 0.2.0-dev-20200709-060143
Workstation version: 0.4.0
Server: 1.4.1 (staging environment running in Qubes)

Scenario: Online mode

Login

when SecureDrop desktop icon is double-clicked, preflight updater is displayed
After preflight updater runs, when user clicks Continue, login dialog is displayed
In login dialog:
- show/hide password functionality works
- incorrect password cannot log in
- 2FA token reuse cannot log in after password failure
- invalid 2FA token cannot log in
- valid credentials and 2FA can log in

Sources

after valid login:
- the login dialog closes
- source data is downloaded and source list is populated
- user is prompted for GPG key access
- submissions and replies are decrypted
- the source list is displayed but no sources are selected by default
- the conversation view is not populated
when a source is selected in source list:
- conversation view is populated with source conversation
- a source message containing HTML is displayed as unformatted text
- source submissions have an active Download button
- source submission compressed file size is displayed accurately
when the upper right 3-dot button is clicked:
- a menu is displayed with a delete source account option
- when delete source account is selected:
  - the source is deleted from the source list and the conversation view is blanked
  - the source is deleted from the server and not restored on next sync
  - source submissions and messages are removed from the client's data directory
when a source is starred in source list, and the client is closed and reopened in Online mode:
- ❌ the source is still starred in the source list

Starring behavior

The behavior I'm seeing as follows:

I star a bunch of sources.
I exit the client.
I restart the client and log back into online mode.
I see the starring state before 1), until the sync completes. Then all the stars update to their correct values.

(I'm not sure if that's a regression. It's possible I'm only noticing now because I'm testing with a large number of sources, hence slower syncs.)

sssoleileraaa · 2020-07-09T23:08:13Z

Starring behavior

The behavior I'm seeing as follows:

I star a bunch of sources.

I exit the client.

I restart the client and log back into online mode.

I see the starring state before 1), until the sync completes. Then all the stars update to their correct values.

(I'm not sure if that's a regression. It's possible I'm only noticing now because I'm testing with a large number of sources, hence slower syncs.)

Potential explanation:

If you shut down the client before the client database is updated as a result of the server responding with the updated sources, then when you open the client again it'll show whatever is in the client database until the first sync. Jen and I have discussed potentially waiting until the first sync to make sure the client database is up-to-date first, but there would be a long gap between the client starting and information showing up.

eloquence · 2020-07-09T23:47:47Z

(@creviera confirms that this starring behavior is present in master and it's both minor and a bit tricky so we can revisit separately.)

eloquence · 2020-07-10T00:54:44Z

emkll · 2020-07-10T14:40:53Z

Testing in progress, securedrop 0.2.0-dev-20200709

Status: Online mode testing complete. Offline, concurrent and large dataset testing TBD

Scenario: Online mode

Login

when SecureDrop desktop icon is double-clicked, preflight updater is displayed
After preflight updater runs, when user clicks Continue, login dialog is displayed
In login dialog:
- show/hide password functionality works
- incorrect password cannot log in
- 2FA token reuse cannot log in after password failure
- invalid 2FA token cannot log in
- valid credentials and 2FA can log in

Sources

after valid login:
- the login dialog closes
- source data is downloaded and source list is populated
- user is prompted for GPG key access
- submissions and replies are decrypted
- the source list is displayed but no sources are selected by default
- the conversation view is not populated
when a source is selected in source list:
- conversation view is populated with source conversation
- a source message containing HTML is displayed as unformatted text
- source submissions have an active Download button
- source submission compressed file size is displayed accurately
when the upper right 3-dot button is clicked:
- a menu is displayed with a delete source account option
- when delete source account is selected:
  - the source is deleted from the source list and the conversation view is blanked
  - the source is deleted from the server and not restored on next sync
  - source submissions and messages are removed from the client's data directory
when a source is starred in source list, and the client is closed and reopened in Online mode:
- ❌ the source is still starred in the source list when logging into offline mode (but does appear when I log in again)

Replies

when a source is selected in the source list:
- the reply panel is available for use and there is no message asking the user to sign in
- a reply can be added to the conversations
- a reply containing HTML is displayed as unformatted text
- a reply with a single string of characters longer than 100 chars is displayed correctly
- ❌ a reply with a line longer than 100 chars is displayed correctly Tracking upstream issue QTBUG-85498 #815
- two replies added immediately after each other are ordered correctly

Submissions

Preview

when Download is clicked on a submission:
- the submission is downloaded and decrypted
- the Download button is replaced with Print and Export options
- the submission filename is displayed.
For a DOC submission:
- when the submission filename is clicked, a disposable VM (dispVM) is started.
- after the dispVM starts, the submission is displayed in LibreOffice
- when LibreOffice is closed, the dispVM shuts down
For a PDF submission:
- when the submission filename is clicked, a dispVM is started.
- after the dispVM starts, the submission is displayed in evince
- when evince is closed, the dispVM shuts down
For a JPEG submission:
- when the submission filename is clicked, a dispVM is started.
- after the dispVM starts, the submission is displayed in Image Viewer
- when evince is closed, the dispVM shuts down

Export

When Export is first clicked on a submission:
- the "Preparing to export..." message is displayed
- the sd-devices VM is started
- the user is prompted to insert an Export USB
- On clicking Cancel, the prompt closes and the file is not exported
When Export is clicked on the submission again:
- the "Preparing to export..." message is displayed
- the user is prompted to insert an Export USB
- When the user inserts an invalid Export USB, attaches it to the sd-devices VM and clicks OK:
  - a message is displayed indicating that the Export USB is invalid and
    the user is prompted to insert a valid device
When Export is clicked on the submission again:
- the "Preparing to export..." message is displayed
- the user is prompted to insert an Export USB
- When the user inserts a valid Export USB, attaches it to the sd-devices VM, and clicks OK:
  - the user is prompted for the Export USB's password
- When the user enters an invalid Export USB password and clicks Submit:
  - a failure message is displayed and the user is prompted to enter the password again
- When the user enters an valid Export USB password and clicks Submit:
  - the file is saved to the Export USB
When the user detaches the Export USB and mounts it on another VM or computer:
- the decrypted submission is available in on the Export USB, in a directory sd-export-<timestamp>/export_data

Print

When the user clicks Print on a downloaded submission:
- a "Preparing to print..." message is displayed
- the sd-devicesVM is started
- the user is prompted to connect a supported printer
When the user connects a printer, attaches it to the sd-devices VM, and clicks Continue:
- a "Printing..." message is displayed
- the X Printer Panel dialog is displayed with the printer selected
When the user clicks Print in the X Printer Panel:
- the submission is printed successflly.

Closing the client

When the user clicks the main window close button:
- the client exits.

Scenario: Offline mode without existing data

Prerequisites:

server is available and contains source test data

test data includes at least one previously downloaded submission

test data includes at least one undownloaded submission

~/.securedrop_client/data in sd-app is empty, and ~/.securedrop_client/svs.sqlite does not exist (do not delete the entire ~/.securedrop_client directory)

the sd-devices VM is not running (shut down manually if necessary)

a supported printer is available, but not attached.

Offline to Online

Scenario: Offline mode with existing data

Prerequisites:

server is available and contains source test data

test data includes at least one previously downloaded submission

test data includes at least one undownloaded submission

client data directory has been synced with server in a previous login

the sd-devices VM is not running (shut down manually if necessary)

a supported printer is available, but not attached.

Offline to Online

Scenario: Client and Journalist Interface both in use

Note: this scenario requires access to the Journalist Interface (JI) via
Tor Browser. If the scenario is being tested on Qubes, the JI address can be found
in sd-whonix in /usr/local/etc/torrc.d/50_user.conf. See https://github.com/freedomofpress/securedrop-workstation/wiki/Developer-Tips#how-to-connect-to-the-journalist-interface-in-qubes for instructions on how to connect to the JI in a VM.

Prerequisites:

server is available and contains source test data

client data directory is empty

Login

Sources, replies, submissions

Scenario: Large dataset

Prerequisites:

server is available and contains large source test dataset (256 sources,
submission sizes ranging from 1-500MB)

client data directory is empty

Sources

after valid login:
- the login dialog closes
- all source data is downloaded and source list is populated
- user can scroll to bottom of source list
- user is prompted for GPG key access
- submissions and replies are decrypted
- the source list is displayed but no sources are selected by default
- the conversation view is not populated
when a source is selected in source list:
- conversation view is populated with source conversation
- a source message containing HTML is displayed as unformatted text
- source submissions have an active Download button
- source submission compressed file size is displayed accurately
when the upper right 3-dot button is clicked:
- a menu is displayed with a delete source account option
- when delete source account is selected:
  - the source is deleted from the source list and the converation view is blanked
  - the source is deleted from the server and not restored on next sync
  - source submissions and messages are removed from the client's data directory
when a source is starred in source list, and the client is closed and reopened in Online mode:
- the source is still starred in the source list

Replies

when a source is selected in the source list:
- the reply panel is available for use and there is no message asking the user to sign in
- a reply can be added to the conversations
- a reply containing HTML is displayed as unformatted text
- two replies added immediately after each other are ordered correctly

Submissions

Preview

when Download is clicked on a submission:
- the submission is downloaded and decrypted
- the Download button is replaced with Print and Export options
- the submission filename is displayed.
For a DOC submission:
- when the submission filename is clicked, a disposable VM (dispVM) is started.
- after the dispVM starts, the submission is displayed in LibreOffice
- when LibreOffice is closed, the dispVM shuts down
For a PDF submission:
- when the submission filename is clicked, a dispVM is started.
- after the dispVM starts, the submission is displayed in evince
- when evince is closed, the dispVM shuts down
For a JPEG submission:
- when the submission filename is clicked, a dispVM is started.
- after the dispVM starts, the submission is displayed in Image Viewer
- when evince is closed, the dispVM shuts down

eloquence · 2020-07-10T22:19:50Z

Stepping through

Scenario: Client and Journalist Interface both in use

today.

eloquence · 2020-07-10T23:34:32Z

eloquence · 2020-07-10T23:55:41Z

Scenario: Client and Journalist Interface both in use (continued)

Environment details: see #1125 (comment)

when a reply is deleted by a source:
- ❌ the reply is flagged as having being read in the client -- not implemented yet, see Implement "read by source" receipts for journalist replies #889
when an individual file submission is deleted in the JI:
- the submission is no longer listed in the conversation view
- the submission files are deleted from the client data directory
when an individual message is deleted in the JI:
- the message is no longer listed in the conversation view
- the messages are deleted from the client database
when a source is deleted in the JI:
- the source is no longer listed in the client after next sync
- files associated with the source are no longer present in the client data directory
when a source is deleted in the client:
- the source is no longer listed in the JI after a page reload

eloquence · 2020-07-13T23:41:12Z

Environment details: see #1125 (comment)

To test whether the fix for #1113 works in staging, I followed the original STR, modified as follows:

Added 150 new sources using qa_loader.py
Deleted two sources between syncs
Waited for the next sync

The two sources were deleted as expected within a single sync period. Separately, I also performed lots of deletions while a sync was underway without issues.

eloquence · 2020-07-14T00:11:19Z

Environment details: see #1125 (comment)

To test whether #975 works in staging, I deleted the database and data directory, and re-synced the client in debug mode with 200+ sources, waiting for several rounds of syncs to complete. I then used these commands to verify that reply and message download jobs were not added to the queue repeatedly:

$ grep "Added ReplyDow" client.log | cut -d "'" -f2 | tee 1.log | uniq > 2.log
$ grep "Added MessageDow" client.log | cut -d "'" -f2 | tee 1.log | uniq > 2.log

The cut command is used to extract the job UUIDs. AS expected, 1.log and 2.log were identical in both cases, meaning that no duplicate jobs were inserted. I verified this by spot-checking the logs and double-checking the behavior of the above command pipeline. I also observed that many DuplicateJob ... skipping lines could be found in the DEBUG level logs, as expected.

eloquence · 2020-07-14T23:42:22Z

Environment details: see #1125 (comment)

To test whether #1103 is working as expected, I inspected a highly populated client, in online and offline mode, in both 1368x768 and 1280x1024. As expected, the client renders normally at 1368, while the bubbles still overflow slightly in 1280.

eloquence · 2020-07-15T00:56:28Z

Environment details: see #1125 (comment)

To test whether #1116 was resolved, I performed these steps:

Open highly populated client (~250 sources)
Delete all data via Journalist Interface
Run QA loader to create 250 sources while client is running (with defaults, i.e. 50% with replies)
Wait for client to sync and observe that no crash occurs

The client did not crash and renders the same results as the Journalist Interface. However, this process somehow resulted in a bunch of disconnected replies, which the client.log records as follows (128 log lines like this for each sync):

2020-07-14 17:39:17,316 - securedrop_client.storage:318(update_replies) ERROR: No source found for reply e8a87dc1-0b74-4dab-ab6d-82ab01e60f7c

Near as I can tell, this is not a bug in the client (I would expect the logs to no longer appear with 1.5.0, as disconnected replies should no longer be returned by the API per freedomofpress/securedrop#5351), but it could be some broken behavior in the deletion logic or the interaction with QA Loader. My guess is that the QA Loader's lock messed everything up, per the error messages below.

eloquence · 2020-07-15T01:09:34Z

^ Rebooting the staging server resolved the broken session state above, after which the shredder process happily resumed its unfinished work. The ERROR rate in client.log subsequently decreased until it reached zero.

rmol · 2020-07-15T13:26:01Z

@eloquence Do you still have that log on the app server? I'm curious what the first exception was.

eloquence · 2020-07-21T05:41:24Z

@rmol I do. The first error I see in /var/log/syslog is:

Jul 14 16:48:09 sd-staging-app python[856]: 2020-07-14 16:48:09,531 INFO Error purging deleted sources: This Session's transaction has been rolled back due to a previous exception during flush. To begin a new transaction with this Session, first issue Session.rollback(). Original exception was: (sqlite3.OperationalError) database is locked
Jul 14 16:48:09 sd-staging-app python[856]: [SQL: DELETE FROM submissions WHERE submissions.id = ?]
Jul 14 16:48:09 sd-staging-app python[856]: [parameters: (390,)]
Jul 14 16:48:09 sd-staging-app python[856]: (Background on this error at: http://sqlalche.me/e/e3q8)

I think it's as with freedomofpress/securedrop#5278, qa_loader is causing a database lock that interferes with other operations.

eloquence · 2020-07-21T06:13:51Z

This was completed today. Prod environment updated successfully. Closing.

eloquence pinned this issue Jul 8, 2020

This was referenced Jul 16, 2020

SecureDrop Client 0.2.1 #1127

Merged

Release securedrop-client 0.2.1 freedomofpress/securedrop-builder#178

Merged

eloquence closed this as completed Jul 21, 2020

eloquence unpinned this issue Jul 31, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Release SecureDrop Client 0.2.1 #1125

Release SecureDrop Client 0.2.1 #1125

eloquence commented Jul 8, 2020

eloquence commented Jul 9, 2020

sssoleileraaa commented Jul 9, 2020 •

edited

Loading

Starring behavior

eloquence commented Jul 9, 2020

eloquence commented Jul 10, 2020

emkll commented Jul 10, 2020 •

edited

Loading

eloquence commented Jul 10, 2020 •

edited

Loading

eloquence commented Jul 10, 2020 •

edited

Loading

eloquence commented Jul 10, 2020

eloquence commented Jul 13, 2020

eloquence commented Jul 14, 2020 •

edited

Loading

eloquence commented Jul 14, 2020

eloquence commented Jul 15, 2020 •

edited

Loading

eloquence commented Jul 15, 2020 •

edited

Loading

rmol commented Jul 15, 2020

eloquence commented Jul 21, 2020

eloquence commented Jul 21, 2020

Release SecureDrop Client 0.2.1 #1125

Release SecureDrop Client 0.2.1 #1125

Comments

eloquence commented Jul 8, 2020

eloquence commented Jul 9, 2020

Scenario: Online mode

Login

Sources

Starring behavior

sssoleileraaa commented Jul 9, 2020 • edited Loading

Starring behavior

eloquence commented Jul 9, 2020

eloquence commented Jul 10, 2020

Replies

Submissions

Preview

emkll commented Jul 10, 2020 • edited Loading

Testing in progress, securedrop 0.2.0-dev-20200709

Scenario: Online mode

Login

Sources

Replies

Submissions

Preview

Export

Print

Closing the client

Scenario: Offline mode without existing data

Offline to Online

Scenario: Offline mode with existing data

Offline to Online

Scenario: Client and Journalist Interface both in use

Login

Sources, replies, submissions

Scenario: Large dataset

Sources

Replies

Submissions

Preview

eloquence commented Jul 10, 2020 • edited Loading

eloquence commented Jul 10, 2020 • edited Loading

Scenario: Client and Journalist Interface both in use

Login

Sources, replies, submissions

Deleted journalist details remain in the client database

eloquence commented Jul 10, 2020

Scenario: Client and Journalist Interface both in use (continued)

eloquence commented Jul 13, 2020

eloquence commented Jul 14, 2020 • edited Loading

eloquence commented Jul 14, 2020

eloquence commented Jul 15, 2020 • edited Loading

eloquence commented Jul 15, 2020 • edited Loading

rmol commented Jul 15, 2020

eloquence commented Jul 21, 2020

eloquence commented Jul 21, 2020

sssoleileraaa commented Jul 9, 2020 •

edited

Loading

emkll commented Jul 10, 2020 •

edited

Loading

eloquence commented Jul 10, 2020 •

edited

Loading

eloquence commented Jul 10, 2020 •

edited

Loading

eloquence commented Jul 14, 2020 •

edited

Loading

eloquence commented Jul 15, 2020 •

edited

Loading

eloquence commented Jul 15, 2020 •

edited

Loading