Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make gpg not check the trustdb as we do not use the WoT #6390

Merged
merged 2 commits into from
Apr 7, 2022

Conversation

eaon
Copy link
Contributor

@eaon eaon commented Apr 7, 2022

Status

Ready for review

Description of Changes

GPG automatically checks the trustdb on the fly, an operation that can take a long time with very large keyrings as they sometimes exist on large instances. We don't use the Web of Trust at all though so we want to tell GPG to skip these checks by default.

Fixes #6389

Testing

  • Source key generation
    • During make dev startup, run while true; do grep "--no-auto-check-trustdb"<<<$(ps a); sleep 1; done and confirm that --no-auto-check-trustdb is actually passed to gpg2
    • With a fresh install, run loaddata.py to generate keys for 10-20k sources
    • Repeatedly generate new sources manually and see response times stay approximately the same
  • Source deletion
    • Log into journalist interface
    • Run while true; do grep "--no-auto-check-trustdb"<<<$(ps a); done (no sleep as this operation is faster than generating a key)
    • Delete source account and ensure --no-auto-check-trustdb is in the the grep loop output

Checklist

  • Linting (make lint) and tests (make test) pass in the development container
  • I have written a test plan and validated it for this PR

GPG automatically checks the trustdb on the fly, an operation that can
take a long time with very large keyrings as they sometimes exist on
large instances. We don't use the Web of Trust at all though so we want to tell
GPG to skip these checks by default.
@eaon eaon requested a review from a team as a code owner April 7, 2022 15:00
@zenmonkeykstop zenmonkeykstop self-assigned this Apr 7, 2022
@codecov-commenter
Copy link

codecov-commenter commented Apr 7, 2022

Codecov Report

Merging #6390 (27a3301) into develop (27f17b1) will decrease coverage by 1.26%.
The diff coverage is 100.00%.

@@             Coverage Diff             @@
##           develop    #6390      +/-   ##
===========================================
- Coverage    83.98%   82.72%   -1.27%     
===========================================
  Files           61       40      -21     
  Lines         4302     3855     -447     
  Branches       523      503      -20     
===========================================
- Hits          3613     3189     -424     
+ Misses         565      537      -28     
- Partials       124      129       +5     
Impacted Files Coverage Δ
securedrop/encryption.py 90.22% <100.00%> (+0.79%) ⬆️
securedrop/i18n_tool.py 39.92% <0.00%> (-44.53%) ⬇️
securedrop/sdconfig.py 54.78% <0.00%> (-33.05%) ⬇️
securedrop/journalist_app/col.py 63.29% <0.00%> (-17.73%) ⬇️
securedrop/source_app/utils.py 75.86% <0.00%> (-17.25%) ⬇️
securedrop/secure_tempfile.py 85.45% <0.00%> (-14.55%) ⬇️
securedrop/passphrases.py 82.81% <0.00%> (-14.07%) ⬇️
securedrop/i18n.py 80.89% <0.00%> (-12.36%) ⬇️
securedrop/journalist_app/account.py 84.74% <0.00%> (-10.17%) ⬇️
securedrop/journalist_app/main.py 83.00% <0.00%> (-2.00%) ⬇️
... and 23 more

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

As source key deletion is a synchronous operation, let's ensure there's
no unintended long-running trustdb check.
@eaon
Copy link
Contributor Author

eaon commented Apr 7, 2022

I updated test plan above to reflect the new commit adding --no-auto-check-trustdb to the deletion invocation

Copy link
Contributor

@zenmonkeykstop zenmonkeykstop left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Source key generation:

  • During make dev startup, run while true; do grep "--no-auto-check-trustdb"<<<$(ps a); sleep 1; done and confirm that --no-auto-check-trustdb is actually passed to gpg2
  • With a fresh install, run loaddata.py often enough to generate keys for 10-20k sources
  • Repeatedly generate new sources manually and see response times stay approximately the same (tested on dev environment with ~3k sources and prod environment with ~20k sources with diff manually applied)

Source deletion:

  • Log into journalist interface
  • Run while true; do grep "--no-auto-check-trustdb"<<<$(ps a); done (no sleep as this operation is faster than generating a key)
  • Delete source account and ensure --no-auto-check-trustdb is in the the grep loop output

LGTM, no side-effects noted during exploratory testing on prod or dev either.

@zenmonkeykstop zenmonkeykstop merged commit 443a051 into develop Apr 7, 2022
@zenmonkeykstop zenmonkeykstop deleted the 6389-gpg-no-auto-check-trustdb branch April 7, 2022 21:33
@zenmonkeykstop zenmonkeykstop mentioned this pull request Apr 10, 2022
21 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Large gpg keyrings slow down application performance due to automatic Web of Trust checks
3 participants