Shipped securedrop-client incompatible with latest journalist API

[deeplow]

Description

When installing the securedrop-workstation, following the README.md and testing it with the latest securedrop journalist server version, the securedrop-client is not able to view the documents ("waiting to refresh" indefinitely)

Steps to Reproduce

1. configure sercuredrop server (as detailed in the docs)
2. follow README.md
3. start securedrop-client in sd-svs
4. login with credentials

Expected Behavior

A successful login and to be able to see the files shared with journalist

(the problem is not is not with the server, as suggested on gitter. Image below shows a successful login on the web interface)

Actual Behavior

It looks like the login is successful because it shows the button log out, but a message shows "waiting to refresh" indefinitely.

Comments

I was able to pinpoint the specific issue to a "Malformed authorization header." sent by the client. (Not that this matters, because it must have been fixed in the meantime)

The problem seems to stop when we copy from the source code to sd-svs and run that version (but that process is not documented and it took me a while to figure out - rpc-policies, enable networking, removing --no-proxy flag, etc.).

I ended up taking a few days to complete the whole installation process and to figure out a solution. But to avoid that people interested in the project also need to figure it out too I would suggest a warning in the section about the securedrop client saying that it might not work and it should instead be tested in the sd-dev machine (that's the easy solution that shouldn't take away the devs' time)

Another other solution would involve either making sure the master of this repo passes on the test with the journalist api server version of the upstream version (which I don't think it is much worth it in terms of time at this stage)

[redshiftzero]

wow - big apologies for the confusion @deeplow. So this was issue freedomofpress/securedrop-client#232 / freedomofpress/securedrop#4064, an issue with the auth header as you correctly point out.

We should probably cut another release of securedrop-client so that this doesn't continue to bite people, but until then, want to make a PR to note in the README that developers from qubes should set up the development environment instead of using the latest release?

[conorsch]

Is the required fix merged into securedrop-client, @redshiftzero ? In other words, would a fresh build of the relevant deb package resolve? If so, happy to push that out to unblock folks using Qubes immediately (as it seems @deeplow is, judging by the report).

[heartsucker]

Yes, that was fixed with freedomofpress/securedrop-client#237

[conorsch]

Thanks for the clarity, @heartsucker. Given that we'd also have to cut a new release of the client code in order to unblock the package builds, let's indeed wait a bit, as @redshiftzero . Holler if you have any problems with getting the dev env set up, @deeplow !

[redshiftzero]

(which is in master but unreleased - i.e. the latest securedrop-client debian packages are broken with latest securedrop server, hence the behavior @deeplow observes)

[deeplow]

wow - big apologies for the confusion @deeplow

@redshiftzero No worries! It was a bit frustrating in the beginning, but it actually made me learn lot of
how how the project is structured and how tondebug certain parts. I am kind of glad I had this trouble.

want to make a PR to note in the README that developers from qubes should set up the
development environment instead of using the latest release?

@redshiftzero Sounds good. I'll do it.

[deeplow]

Holler if you have any problems with getting the dev env set up, @deeplow !

@conorsch Thank you! I'll drop by gitter if I run into other problems

[conorsch]

Let's make sure to remove any clarifications from #242 in the README once this issue is resolved.

[eloquence]

The plan to update the client, as discussed during today's sprint planning for the 4/17-5/1 sprint, breaks the steps out as follows:

• Update SDK dependency to 0.0.7
• Document PyPI mirror usage
• Update client dependencies (wheels)
• Create client release
• Create and upload Debian package

@kushaldas has offered to be point on this, with @creviera shadowing as much as possible to become familiar with the release process.

[sssoleileraaa]

Update

Ran through creating a debian package with kushal today for securedrop-proxy in preparation for creating a package for the client.

Here's what we did and what we'll do again for the client...

Debian packaging prep

1. Copy new package files to local machine

Copy new package files to localwheels directory using make build-wheels. You will need your yubikey to sign the updated sha256sums.txt file.

2. Sync changes with s3

Push the changes to an aws s3 bucket: dev-bin.ops.securedrop.org. You will need permission to write to this bucket.

3. Manually update index.html file

Manually add s3 urls that point to the new wheels in the index.html file.

4. Make a PR

Make a PR for the changes made to index.html so new dependencies can be pulled from the s3 bucket.

5. Update target project

Once PR from step 4 is merged, run make requirements to update the target project's requirements.txt file (the target project being the project you want to package).

Update target project

1. Make a PR

Now that there's an updated requirements.txt file in the target repo, make a PR for that.

2. Push new tag

After all PRs are merged from the preparation steps, push a new tag on the target project.

Build the Debian package

1. Create a source distribution

Run python3 setup.py sdist to create a source distribution.

2. Run make securderop-proxy to make the Debian package.

3. Create a changelog entry

Run dch command for a little changelog automation and leave a note in the file about what changed. Make a PR for that. This shouldn't be merged until the package is released to the apt servers.

Release the package to the apt servers

(Mickael is needed for this step. We didn't get to this today.)

Notes

• All the steps above are done from the securedrop-debian-packaging repo except for pushing the updated requirements.txt file and the new tag, which will be done from the target repo that contains the code you want to package.
• PKG_DIR environment variable needs to be set to the path of the target project in order for make commands to work.
• securedrop-debian-packaging repo has helper scripts to help us package the proxy and client for Qubes.
• Yes, changelogs for debian packages live inside the securedrop-debian-packaging repo.

[sssoleileraaa]

🎉 🎉 Version 0.0.7 of the client has been released to the apt servers, resolving this issue. 🎉 🎉

You can pull the latest debian package from the apt test server by updating thesd-svs-template, restarting it, and restarting sd-svs

[redshiftzero]

For those that have an install of securedrop-client 0.0.6 in their sd-svs AppVM, note that you should delete ~/.securedrop_client before running securedrop-client 0.0.7 (this is a one time requirement due to the squashing of all the pre-0.0.7 migrations, future releases will have one release per migration)