Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BOLT04: Atomic Multi-path Payments [feature 30/31] #658

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .aspell.en.pws
Original file line number Diff line number Diff line change
Expand Up @@ -362,3 +362,11 @@ tlvs
snprintf
GitHub
IRC
preimages
timelocks
th
atomicity
adaptively
preimages
griefing
decorrelation
198 changes: 198 additions & 0 deletions 04-onion-routing.md
Original file line number Diff line number Diff line change
Expand Up @@ -248,6 +248,13 @@ This is a more flexible format, which avoids the redundant `short_channel_id` fi
1. type: 6 (`short_channel_id`)
2. data:
* [`short_channel_id`:`short_channel_id`]
1. type: 10 (`option_amp`)
2. data:
* [`32*byte`:`payment_id`]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For recurring payments, the idea with AMP is that payment_id can be used as a type of 'deposit box', similar to a traditional bank account number.

But this field is also used to prevent multiple people trying to pay to the same invoice simultaneously from disturbing each others sets.

It seems that both uses of the same field are mutually exclusive?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, clarified offline. That is what set_id is for

* [`32*byte`:`stream_id`]
* [`32*byte`:`share`]
* [`u16`:`child_index`]
* [`u64`:`total_msat`]

### Requirements

Expand All @@ -261,6 +268,197 @@ The reader:

The requirements for the contents of these fields are specified [above](#legacy-hop_data-payload-format).

## Atomic Multi-path Payments

If the final node receives an onion packet with `option_amp` field,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think having both option_amp and option_mpp is very confusing (but hopefully the two proposals merge at least partially). I think that the main feature this proposal adds to rusty's MPP proposal is the spontaneous part (because option_mpp is currently also atomic - controlled by the recipient). Maybe the naming should reflect that (option_spontaneous_mpp)?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might even make sense that this PR mutates in a specification of spontaneous payments (not invoice-based), encompassing both the multi-part aspect and the non multi-part?

Copy link
Collaborator Author

@cfromknecht cfromknecht Aug 28, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that the main feature this proposal adds to rusty's MPP proposal is the spontaneous part

I disagree here, the primary goal here is not reuse payment hashes for better privacy:

  • intermediaries can't correlate subpayments
  • forwarding a payment doesn't leak anything about the invoice being paid. since all identifiable information is enclosed only for the receiver's eyes (if the invoice is public)
  • eliminates known probing vectors (of the payment hash) since it is never exposed directly over the network

The spontaneous + recurring pieces are useful side-effects of the sender generating the required randomness.

the payment MAY be an atomic multi-path payment. Such atomic multi-path payments
MAY use a _distinct_ payment hash for each path.
cfromknecht marked this conversation as resolved.
Show resolved Hide resolved

The `amt_to_forward` value will be the amount for this partial payment only. The
`option_amp` flag flag is a promise by the sender that the rest of the payment
cfromknecht marked this conversation as resolved.
Show resolved Hide resolved
will follow in succeeding HTLCs with the same `stream_id`; we call these HTLCs,
which that the same `stream_id`, an "HTLC set".
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found set_id more descriptive than stream_id.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that this confusion is mostly because of the payment_id being different from our usual invoice-based payment_id. I think that stream_id should be renamed payment_id (it identifies this particular payment, which is sent in multiple parts), and payment_id should be renamed something else (subscription_id or something that makes sense for identifying a recurring payment?).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found set_id more descriptive than stream_id.

I did too, reverted back to set_id for now.

I think that this confusion is mostly because of the payment_id being different from our usual invoice-based payment_id. I think that stream_id should be renamed payment_id (it identifies this particular payment, which is sent in multiple parts), and payment_id should be renamed something else (subscription_id or something that makes sense for identifying a recurring payment?).

The intention behind calling it payment_id was to keep some overlap with payment_hash, since they reuse the same field in the invoice. I'm not totally convinced on subscription_id, since that is only one of many use cases. Perhaps payment_addr is better than payment_id? In many ways it does behave more like an addr than an id

cfromknecht marked this conversation as resolved.
Show resolved Hide resolved

One key distinction with `option_amp` is that the sender generates all
preimages, and only reveals them to the final hop if all partial payments arrive
successfully. As such, the payer _will not_ learn a new preimage as it would in
the regular payment flow. For accounting purposes, however, `option_amp` can be
used to fulfill an invoice akin to the regular payment flow, and also enforce
additional constraints such as amounts and timelocks.

The writer:
- MUST NOT include `option_amp` for any non-final node.
- if the sender has an invoice and `option_amp` feature was not set in the invoice:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should specify how this is to be communicated after #656 is merged.

- MUST NOT include `option_amp` for the final node.
- otherwise:
- MAY include `option_amp` for the final node.
- if it does include `option_amp`:
- MUST generate a random `stream_id` to be used on all HTLCs in the set.
- MAY send more than one HTLC using the same `stream_id`.
- MUST set the `share` values of all HTLCs such that their xor is a random
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MUST ensure all share values are unique?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added SHOULD since it's not a hard requirement, and receiver will accept it all the same. I don't have a strong opinion tho, if others think we should use MUST

root seed `r`.
- SHOULD choose a unique child_index_i for each HTLC.
- MUST derive the `payment_hash` for an HTLC using `amp_child(r, child_index_i)`.
- if the invoice specifies a non-zero `amount`:
- MUST set `total_msat` to `amount`.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this allow over-payment (as specified for standard invoice payments)?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes probably, i think it should mirror whatever is finalized in #643

- otherwise:
- MUST set `total_msat` to the amount it wishes to pay.
- MUST ensure the total `amount_to_forward` in the HTLC set which arrives at
the payee is equal to `total_msat`.
- if the sender has an invoice:
- MUST set the `payment_id` of each HTLC to the `payment_hash` in the
invoice.
- otherwise:
- MUST set the `payment_id` of each HTLC to zero.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or define it as a separate tlv type? There may be some overlap with the random identifier generated by the receiver that was discussed before for regular payments and mpp. Both are ids generated by the receiver and both are not used to lock the htlc onto.


The reader:
- if `option_amp` is present:
- MUST fail the HTLC if it is not the final node.
- MUST fail the HTLC as it would otherwise fail a single HTLC of
`amt_to_forward`, `payment_hash`, and `cltv_expiry` without context of the
invoice.
- if the `payment_id` is non-zero:
- MUST fail the HTLC if an invoice for `payment_id` does not exist.
- MUST fail the HTLC if `total_msat` is less than the invoice's `amount`.
- MUST fail the HTLC if `cltv_expiry` does not satisfy the invoice's
`min_final_cltv_expiry`.
- otherwise:
- MUST fail the HTLC if `cltv_expiry` does not satisfy the node's default
`min_final_cltv_expiry`
- MUST fail the entire HTLC set if `total_msat` is not the same for all HTLCs
in the set.
- if the total `amount_to_forward` of the HTLC set is equal `total_msat`:
- MUST reconstructs `r` as the xor of all `share`s in the HTLC set.
- MUST compute `p_i, h_i = amp_child(r, child_index_i)` for all HTLCs in the
set.
- if any `i-th` HTLC's `payment_hash` differs from `h_i`:
- MUST fail the HTLC set.
- otherwise:
- MAY fulfill the `i-th` HTLC in the set using `p_i`.
- otherwise:
- MUST fail an HTLC in set if its `cltv_expiry` elapses.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't it be failed earlier than that? In case of an (amp) hodl invoice, an application assumes that when the invoice is marked as accepted (we know the root seed, but haven't pulled yet), the invoice cltv expiry condition holds.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wasn't sure exactly what you meant here, can you elaborate on "failed earlier"?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Example:

  • current height: 100
  • (hodl) invoice cltv delta: 40, amt: 100
  • first htlc for 50 sats comes in with expiry 140 -> accepted as partial payment
  • new block comes in
  • second htlc for 50 sats comes in with expiry 141 -> accepted as partial payment

The hodl invoice should now move to the accepted state, because enough has been paid. We'd send the accepted event to rpc subscribers. But at that point, there are only 39 blocks left before the first htlc expires. The subscriber will probably assume that when the event comes in, the final cltv delta of 40 is still met.

- MAY fail all HTLCs in the set after a reasonable timeout.

### Atomic Multi-path Payment Derivation

Let the _root seed_ `r` be a random 32-byte value. A unique _child preimage_ and
_child hash_ can be derived for a given `child_index` using the `amp_child`
function:

```golang
func amp_child(root_seed [32]byte, child_index uint16) ([32]byte, [32]byte) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I think this derivation is simple enough to be included as "crypto-math" instead of actual code.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i agree it is a little overkill, tho i haven't been able to come up with something that conveys the type information as well the pseudocode. open to suggestions tho :)

preimage := SHA256(root_seed || child_index)
hash := SHA256(preimage)
return preimage, hash
}
```
where `child_index` is serialized using big-endian byte order.
cfromknecht marked this conversation as resolved.
Show resolved Hide resolved

The sender will use `amp_child` to derive a child hash for each HTLC it sends
out, and includes the `child_index` used in the derivation in the final hop's
payload. The receiver will use `amp_child` to settle each HTLC with its
corresponding child preimage, and also to verify that the correct child hash was
set by the sender.

In order to provide cryptographic atomicity over the fulfillment of an HTLC set,
each partial payment `i` also transmits a 32-byte `share`. Each share `s_i`
represents an n-of-n secret sharing of `r`, such that:
cfromknecht marked this conversation as resolved.
Show resolved Hide resolved

```
r = s_1 ^ ... ^ s_n
```

If `n` is known upfront, satisfying this equation can be done simply by
generating all `s_i` randomly.

Otherwise, the sender can generate the shares _adaptively_ by first generating a
random `r`. For all but the last outgoing HTLC, a random `s_i` is generated and
included directly. The final HTLC then computes `s_n` as the xor of all other
shares and `r`:

```
s_n = s_1 ^ ... ^ s_n-1 ^ r
```

If a partial payment fails, this process can be applied recursively to generate
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Def an underrated feature of this scheme!

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a problem with the receiver reading the share value, but then canceling the htlc with for example an invalid onion key error? To the sender is looks like a failure that may even have been caused by the second last node, but in reality the receiver has already obtained the root seed.

smaller partial payments, at the same time guaranteeing that the xor of all
shares results in `r`.

This construction prevents the receiver from learning `r` until all `s_i` have
arrived. If `r` is successfully reconstructed, the receiver can verify the
correctness of child hashes used in the HTLC set, and settle use the child
preimages if they were offered correctly.

The diagram below depicts the relationship between shares, the root seed, child
preimages, and child hashes in the non-adaptive case. Lowercase variables are
used to signal independent variables, while capital letters are used to describe
dependent variables. All independent variables are chosen upfront by the sender.
```
s_1 s_2 s_3 SHARES OF ROOT SEED
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This ascii art is great, it very neatly summarizes the scheme! You should make a t-shirt of it ;)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you! time well spent 😁


| | |
└──┐ | ┌──┘ R = s_1 ^ s_2 ^ s_3
V V V

R ROOT SEED

| | |
┌──┘ | └──┐ P_i = SHA256(R || child_index_i)
V V V

P_1 P_2 P_3 CHILD PREIMAGES

| | |
| | | H_i = SHA256(P_i)
V V V

H_1 H_2 H_3 CHILD HASHES
cfromknecht marked this conversation as resolved.
Show resolved Hide resolved
```

### Rationale
cfromknecht marked this conversation as resolved.
Show resolved Hide resolved

The inclusion of a `payment_id` in `option_amp` allows the receiver to map an
incoming AMP payment to a particular invoice. The sender should set the
`payment_id` to the `payment_hash` of the invoice they are trying to pay,
permitting the receiver to enforce custom parameters, e.g. CLTV deltas, and
unify tracking of AMP payments with the existing invoicing system.

At the same time, AMP payments can be made spontaneously (without and invoice),
cfromknecht marked this conversation as resolved.
Show resolved Hide resolved
since the sender generates all of the necessary secrets. To do so, the sender
leaves the `payment_id` blank, which can be used to facilitate secure donations.

In the event that two payments are made with the same `payment_id`, either to
the same invoice for both are spontaneous, a second identifier is introduced
cfromknecht marked this conversation as resolved.
Show resolved Hide resolved
called the `stream_id`. The `stream_id` should be unique to each HTLC set sent
by the sender, and allows the receiver to distinguish concurrent payments that
collide on `payment_id`.

Both the `stream_id` and `payment_id` are only known to sender and receiver,
preventing intermediaries from introducing griefing via colliding payment
identifiers with high probability.

The `child_index` is included in the final payload so that the receiver can
gracefully tolerate reordering of the partial payments. When each `child_index`
is unique, this offers decorrelation of the partial payments, since they bear
different child payment hashes while traversing the network.

The `total_msat` field is used to determine when all partial payments have been
received. If the AMP is paying an invoice, this also allows the sender to
securely overpay an invoice, for instance, if the invoice's `amount` is
unspecified. If the AMP is spontaneous, this allows the sender to communicate
the exact value to be received in a end-to-end authenticated manner, preventing
cfromknecht marked this conversation as resolved.
Show resolved Hide resolved
certain classes of attacks where intermediaries can steal up to the overpaid
amount.

The entire payment is contingent on the receiver being able to reconstruct the
root seed `r`, which prevents the receiver from pulling any of the partial
payments until the entirety of the HTLC set arrives. This is enforced by the
n-of-n secret shares provided in the final hop payload of each arriving partial
payment.

None of the requirements enforce that more than one HTLC is sent, permitting the
base case of 1 HTLC to function as a standalone spontaneous payment.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👏 I think this is important and this is why I view this work more as a "spontaneous payments" overall feature


# Accepting and Forwarding a Payment

Once a node has decoded the payload it either accepts the payment locally, or forwards it to the peer indicated as the next hop in the payload.
Expand Down