SNICKER receiving function

[AdamISZ]

For what "SNICKER" is, see https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2019-September/017283.html and the blog post (optional, discursive, not up to date) and the gist (technical spec, currently draft) directly linked there. I'll call that spec "BIP-SNICKER" in the following.

Function

Since the changes are quite substantial (almost all additions), I'll list them below, but first, explaining of exactly what functionality is and isn't added by this PR:

I doubt anyone is crazy or patient enough but in any case, DO NOT USE ON MAINNET, NOT READY .

This PR currently only allows a user to run an independent script (snicker-receiver.py) from the scripts directory on a Joinmarket wallet to passively listen for new proposals dropped into a file (proposals.txt) (only proposals with public keys attached are currently supported). If you follow that link I hope it helps to appreciate just how simple this will look at the top layer: just poll a resource periodically and automatically co-sign and broadcast if income > threshold (or other configurable condition); that's it.

Note the very restricted implementation so far:

• Only proposals from a file (todo, ping a url instead)
• Only proposals with pubkeys attached (code supports search for a key but not yet tested)
• Acceptance filter on proposals: only simple default "income of coinjoin > x" with x=0 by default, can be configured as command line argument).
• Only automated co-sign and publish, no option to temporarily store and check the coinjoin before broadcast (although easy enough to add in simple form).
• the Joinmarket standard utxo type p2sh-p2wpkh was assumed for now, although again updating to add support for counterparties with other types may be fairly simple
• biggest limitation is important: coinjoin destination scripts are imported using Joinmarket's existing private key import function, stored in a "fake" branch at mixdepth 0. This means there is no scanning code to search for and recover those coins as of yet. However the code is written to ensure that that scanning is possible in future, using the algorithm described in BIP-SNICKER.

Setting aside the whole issue of how BIP-SNICKER "Proposer"s will function, the Receiver side will need only a little more work from this to be integrated into a Maker with no impact on the Maker workflow except the key recovery issue. We can just add the task.LoopingCall to the yg script, and twisted will slot that into the event loop.

Primitives

Two modules secp256k1_ecdh.py and secp256k1_ecies.py are added for providing the ECDH and ECIES algorithms for key sharing and encryption, respectively. The algos are exactly as in the current BIP-SNICKER. In case it wasn't already clear, the secp256k1* naming scheme for files here reflects the use of a binding to libsecp256k1. Note that the ECIES implementation uses AES and so now imports pyaes, and is using AES exactly as the wallet currently does for encryption in jmclient (see storage.py).

Apart from the snicker functionality, there is a very minor change in secp256k1_transaction to allow for export of reading bitcoin serializations.

PSBT: a simple/partial implementation of this in Python was taken from here (see the README in that package for credit); a couple of edits were needed for support of p2sh-p2wpkh. The implementation is incomplete and would need more work; I chose it for now to minimize time. I'm still vacillating between copying my implementation here wholesale over to Python as I think it's a lot more complete and tested, but otoh for this task we don't need it to be perfect, right now; the only thing that's really important is that the half-signed PSBT in flight is valid for compatibility.
Additionally there is a jm_psbt.py module added to jmbitcoin to give an interface layer above whatever PSBT implementation we use.

A further small primitive addition was BIP69 as specified in BIP-SNICKER for input/output ordering, this was added in secp256k1_transaction.py as the function bip69_sort(tx).

Wallet

Two minor convenience functions are added: get_all_utxos (this is relevant for SNICKER as the source utxo can be from any mixdepth) and sign_tx_at_index, simply making signing more convenient in cases where we know the input index. These additions hint towards future changes: we will in future (I expect, anyway) make both signing and interfacing with the blockchain much more encapsulated within the wallet. This is more difficult with coinjoin wallets than with ordinary wallets (especially when we do funky things like PoDLE or ECDH with keys etc etc, just to bear in mind), but is still a relevant point (mainly key access encapsulation).

SNICKER functions

SNICKER as an algorithm is implemented in snicker.py in the jmbitcoin package. This makes use of the PSBT, ECDH and ECIES primitives mentioned above. It offers functions to create and parse "proposals" as defined in BIP-SNICKER, as well as lower level functions to standardise the creation of 'tweaked keys' and to verify them (particular paranoia is appropriate for the checking of these tweaked keys as any error will result in permanent funds loss!).
In jmclient the snicker_receiver.py module simply offers a SNICKERReceiver object that can query proposals and then process them, checking their validity and apply a default or user-chosen filter as to which proposals to accept based on economics (income).
In scripts the already mentioned snicker-receiver.py user level script just instantiates a SNICKERReceiver and then polls it in a loop.

Tests

Tests have been added for BIP69, ECDH, ECIES, PSBT in jmclient/test/test_tx_creation.py (this needs more, but note the basic PSBT functions were already tested in the github repo linked above, and anyway more work is needed on that), and for SNICKER, test is in jmclient/test/test_snicker.py which follows the whole create and parse (via SNICKERReceiver) workflow of SNICKER.

This is only the basics of the Receiver function and is not something that affects normal Joinmarket usage (wallet, Taker, Maker) if not used.

Next steps: automated Proposals setup and a testnet infrastructure is probably needed to do some real world testing. Add some configuration to joinmarket.cfg for people to switch this on or off. Some code to poll a URL (preferably Tor HS) for proposals. Improving or changing the PSBT code. And other stuff.

Help appreciated.

[AdamISZ]

Closing this as the work has been replicated (better) in #536 .