Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EXPERIMENTAL: MPP send and receive support (lowlevel) #3309

Merged
merged 22 commits into from
Dec 12, 2019

Commits on Dec 11, 2019

  1. db: add partid, total_msat fields to payment entries.

    This is in preparation for partial payments.  For existing payments,
    partid is 0 (arbitrarity) and total_msat is msatoshi.
    
    Signed-off-by: Rusty Russell <[email protected]>
    rustyrussell committed Dec 11, 2019
    Configuration menu
    Copy the full SHA
    178892b View commit details
    Browse the repository at this point in the history
  2. db: add partid field to htlc_out.

    This is in preparation for partial payments.  For existing payments,
    partid is 0 (to match the corresponding payment).
    
    Signed-off-by: Rusty Russell <[email protected]>
    rustyrussell committed Dec 11, 2019
    Configuration menu
    Copy the full SHA
    ad4ed97 View commit details
    Browse the repository at this point in the history
  3. htlcs: remove origin_htlc_id from htlc_out.

    This is a transient field, so rework things so we don't leave it in
    struct htlc_out.  Instead, load htlc_in first and connect htlc_out to
    them as we go.
    
    This also changes one place where we use it instead of the am_origin
    flag.
    
    Signed-off-by: Rusty Russell <[email protected]>
    rustyrussell committed Dec 11, 2019
    Configuration menu
    Copy the full SHA
    2a03434 View commit details
    Browse the repository at this point in the history
  4. lightningd: share more code between sendpay and sendonion.

    In particular, we're about to do surgery on the detection-of-previous-payments
    logic, and we should not do this in two places.
    
    Signed-off-by: Rusty Russell <[email protected]>
    rustyrussell committed Dec 11, 2019
    Configuration menu
    Copy the full SHA
    c654478 View commit details
    Browse the repository at this point in the history
  5. lightningd: change amount-in-flight check to be more nuanced.

    We currently refuse a payment if one is already in flight.  For parallel
    payments, it's a bit more subtle: we want to refuse if it we already have
    the total-amount-of-invoice in flight.
    
    So we get all the current payments, and sum the pending ones.
    
    Signed-off-by: Rusty Russell <[email protected]>
    rustyrussell committed Dec 11, 2019
    Configuration menu
    Copy the full SHA
    04a46f0 View commit details
    Browse the repository at this point in the history
  6. sendpay/sendonion: add optional partid arg, finesse msatoshi argument.

    msatoshi was used to indicate the amount the invoice asked for, but
    for parallel sendpay it's required, as it allows our sanity check of
    limiting the total payments in flight, ie. it becomes
    'total_msat'.
    
    There's a special case for sendonion, which always tells us the value is 0.
    
    Signed-off-by: Rusty Russell <[email protected]>
    rustyrussell committed Dec 11, 2019
    Configuration menu
    Copy the full SHA
    c61227b View commit details
    Browse the repository at this point in the history
  7. configure: make partid payments only available with EXPERIMENTAL_FEAT…

    …URES and payment_secret
    
    Explicit #if EXPERIMENTAL_FEATURES check in case we enable them at different
    times, but it requires a payment_secret since we put them in the same field.
    
    This incidently stops it working on legacy nodes.
    
    Signed-off-by: Rusty Russell <[email protected]>
    rustyrussell committed Dec 11, 2019
    Configuration menu
    Copy the full SHA
    15fa972 View commit details
    Browse the repository at this point in the history
  8. waitsendpay: add partid arg.

    We need to be able to wait for a unique payment, now payment_hash is not
    always unique.
    
    Signed-off-by: Rusty Russell <[email protected]>
    rustyrussell committed Dec 11, 2019
    Configuration menu
    Copy the full SHA
    3bc4636 View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    94d3897 View commit details
    Browse the repository at this point in the history
  10. doc: update experimental bolt version quotes.

    Signed-off-by: Rusty Russell <[email protected]>
    rustyrussell committed Dec 11, 2019
    Configuration menu
    Copy the full SHA
    2e4416e View commit details
    Browse the repository at this point in the history
  11. lightningd: cleanup redundant args from handle_localpay

    The cltv_expiry and payment_hash are in hin, so no need to hand them
    in here.
    
    Signed-off-by: Rusty Russell <[email protected]>
    rustyrussell committed Dec 11, 2019
    Configuration menu
    Copy the full SHA
    d94ae31 View commit details
    Browse the repository at this point in the history
  12. lightningd: split invoice check into separate function.

    We now return the same error for various "does not match this
    invoice", so it makes sense to encapsulate these checks.  We'll also
    want to expose this for multi-part payments.
    
    Signed-off-by: Rusty Russell <[email protected]>
    rustyrussell committed Dec 11, 2019
    Configuration menu
    Copy the full SHA
    3c6e33a View commit details
    Browse the repository at this point in the history
  13. lightningd: implement htlc sets.

    This isn't plumbed in yet, but the idea is that every htlc gets put
    into a "set" and then we process them once the set is satisfied.  For
    the !EXPERIMENTAL_FEATURES, the set is simply always size 1.
    
    Signed-off-by: Rusty Russell <[email protected]>
    rustyrussell committed Dec 11, 2019
    Configuration menu
    Copy the full SHA
    555b217 View commit details
    Browse the repository at this point in the history
  14. lightningd: wrap htlc replay in a database transaction.

    AFAICT this only "worked" previously because replay htlc simply failed
    them all (no peers are currently connected).  With upcoming changes
    (foreshadowed by the comment) this is no longer true:
    
    Attempting to prepare a db_stmt outside of a transaction: wallet/invoices.c:373
    lightningd: FATAL SIGNAL 6 (version v0.7.3-188-g45b0af4-modded)
    0x55b475590a73 send_backtrace
    	common/daemon.c:41
    0x55b475590b1d crashdump
    	common/daemon.c:54
    0x7f16c557b46f ???
    	???:0
    0x7f16c557b3eb ???
    	???:0
    0x7f16c555a898 ???
    	???:0
    0x55b475564c8f fatal
    	lightningd/log.c:814
    0x55b4755c3ed5 db_prepare_v2_
    	wallet/db.c:605
    0x55b4755c76b5 invoices_find_unpaid
    	wallet/invoices.c:373
    0x55b4755ce91c wallet_invoice_find_unpaid
    	wallet/wallet.c:1990
    0x55b47555861f invoice_check_payment
    	lightningd/invoice.c:257
    0x55b475557a7c htlc_add_set
    	lightningd/htlc_set.c:112
    0x55b47557b294 handle_localpay
    	lightningd/peer_htlcs.c:332
    0x55b47557c63c htlc_accepted_hook_callback
    	lightningd/peer_htlcs.c:857
    0x55b475585573 plugin_hook_call_
    	lightningd/plugin_hook.c:118
    0x55b47557c747 plugin_hook_call_htlc_accepted
    	lightningd/peer_htlcs.c:882
    0x55b47557ca3e peer_accepted_htlc
    	lightningd/peer_htlcs.c:991
    0x55b47557ffb9 htlcs_resubmit
    	lightningd/peer_htlcs.c:2131
    0x55b4755620f7 main
    	lightningd/lightningd.c:801
    
    Signed-off-by: Rusty Russell <[email protected]>
    rustyrussell committed Dec 11, 2019
    Configuration menu
    Copy the full SHA
    73bf9e0 View commit details
    Browse the repository at this point in the history
  15. lightningd: sew in htlc set.

    The invoice_try_pay code now takes a set, rather than a single htlc, but
    it's basically the same thing.
    
    Signed-off-by: Rusty Russell <[email protected]>
    rustyrussell committed Dec 11, 2019
    Configuration menu
    Copy the full SHA
    1839483 View commit details
    Browse the repository at this point in the history
  16. plugins: listpays ignores pre-0.7.0 or manual sendpay payments w/ no …

    …bolt11.
    
    The pay plugin has been supplying the bolt11 string since 0.7.0, so only
    ancient "pay" commands would be omitted by this change.
    
    You can create a no-bolt11 "sendpay" manually, but then you'll find it
    in 'listsendpays'.
    
    Changelog-Removed: JSON: `listpays` won't shown payments made via sendpay without a bolt11 string, or before 0.7.0.
    Signed-off-by: Rusty Russell <[email protected]>
    rustyrussell committed Dec 11, 2019
    Configuration menu
    Copy the full SHA
    8cee375 View commit details
    Browse the repository at this point in the history
  17. plugins: listpays will now consolidate multi-part payments.

    This won't usually be visible to the end-user, since the pay plugin doesn't
    do multi-part yet (and mpp requires EXPERIMENTAL_FEATURES), but we're ready
    once it does.
    
    Signed-off-by: Rusty Russell <[email protected]>
    rustyrussell committed Dec 11, 2019
    Configuration menu
    Copy the full SHA
    84a2753 View commit details
    Browse the repository at this point in the history
  18. common: offer option_basic_mpp for EXPERIMENTAL_FEATURES.

    Signed-off-by: Rusty Russell <[email protected]>
    rustyrussell committed Dec 11, 2019
    Configuration menu
    Copy the full SHA
    c6bbb41 View commit details
    Browse the repository at this point in the history
  19. pytest: add more multi-part-payment tests.

    Signed-off-by: Rusty Russell <[email protected]>
    rustyrussell committed Dec 11, 2019
    Configuration menu
    Copy the full SHA
    cbfc84f View commit details
    Browse the repository at this point in the history
  20. lightningd: require payment_secret for MPP.

    It makes sense, and it's been proposed for addition to the spec to
    broad agreement:
    
    	lightning/bolts#712
    
    Signed-off-by: Rusty Russell <[email protected]>
    rustyrussell committed Dec 11, 2019
    Configuration menu
    Copy the full SHA
    2b4ca09 View commit details
    Browse the repository at this point in the history

Commits on Dec 12, 2019

  1. lightningd: fix spurious "more than twice final" error.

    Bastien TEINTURIER <[email protected]> writes:
    > It looks like the split on c-lightning side is quite limited at the moment:
    > the only option is to split a payment in exactly its two halves,
    > otherwise I get rejected because of the rule of overpaying more than
    > twice the amount?
    
    We only tested exactly two equal-size payments; indeed, our finalhop
    test was backwards.  We only complain if the final hop pays more than
    twice msat (technically, this test is still too loose for mpp: the
    spec says we should sum to the exact amount).
    
    Reported-by: @t-bast
    Signed-off-by: Rusty Russell <[email protected]>
    rustyrussell committed Dec 12, 2019
    Configuration menu
    Copy the full SHA
    207ae69 View commit details
    Browse the repository at this point in the history
  2. lightningd: fix failure message in waitsendpay with multi-part payments.

    Bastien TEINTURIER <[email protected]> writes:
    > One thing I noticed but didn't investigate much: after sending the two
    > payments, I tried using `waitsendpay` and it reported an error *208*
    > (*"Never attempted payment for
    > '98ee736d29d860948e436546a88b0cc84f267de8818531b0fdbe6ce3d080f22a'"*).
    > 
    > I was expecting the result to be something like: "payment succeeded for
    > that payment hash" (the HTLCs were correctly settled).
    
    Indeed, if you waitsendpay without specifying a partid, you are waiting
    for 0, which may not exist.  Clarify the error msg.
    
    Reported-by: @t-bast
    Signed-off-by: Rusty Russell <[email protected]>
    rustyrussell committed Dec 12, 2019
    Configuration menu
    Copy the full SHA
    e6edb76 View commit details
    Browse the repository at this point in the history