Stateless BOLT 12 message verification #1989

jkczyz · 2023-01-26T05:06:05Z

An Offer may contain metadata which can be used to authenticate that the Offer was created from a known key. This allows for stateless verification that a received InvoiceRequest was in response to such an Offer since the InvoiceRequest contains all the TLV records from the Offer.

To achieve this, the metadata includes a nonce and an HMAC of the remaining Offer data (excluding the signing_pubkey). A new key in ExpandedKey is used for constructing the HMAC and the nonce is used as the initialization vector. Additionally, the nonce is used to derive a transient signing key from the new key.

Similarly, a Refund and an InvoiceRequest contain payer metadata, which can similarly be constructed for statelessly verifying that a received Invoice was in response to either. The payer_id is also derived similarly as a signing_pubkey.

Based on ~~#1972~~ #1977.

@TheBlueMatt @valentinewallace Looking for early feedback on the HEAD commit. This is simpler than the approach used in inbound_payments since the InvoiceRequest contains all the Offer's fields. However, I'm not sure if this is sufficient.

The signing pubkey should probably be ephemeral and stateless as well, as it is needed to sign an Invoice created from the InvoiceRequest. So it seems the corresponding private key will need to be generated using an EntropySource and encrypted in the metadata. Does that sound right or do you have an alternative in mind?

codecov-commenter · 2023-01-26T05:22:39Z

Codecov Report

Patch coverage: 91.43% and project coverage change: +0.01 🎉

Comparison is base (a332bfc) 91.34% compared to head (5f8ad79) 91.36%.

❗ Current head 5f8ad79 differs from pull request most recent head 8afe694. Consider uploading reports for the commit 8afe694 to get more accurate results

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1989      +/-   ##
==========================================
+ Coverage   91.34%   91.36%   +0.01%     
==========================================
  Files         102      104       +2     
  Lines       50492    51090     +598     
  Branches    50492    51090     +598     
==========================================
+ Hits        46124    46676     +552     
- Misses       4368     4414      +46

Impacted Files	Coverage Δ
lightning/src/offers/parse.rs	`93.33% <ø> (ø)`
lightning/src/offers/payer.rs	`66.66% <66.66%> (-33.34%)`	⬇️
lightning/src/offers/signer.rs	`82.95% <82.95%> (ø)`
lightning/src/offers/invoice.rs	`90.36% <85.49%> (+0.39%)`	⬆️
lightning/src/ln/inbound_payment.rs	`91.54% <87.50%> (-0.96%)`	⬇️
lightning/src/offers/offer.rs	`89.89% <92.53%> (+0.43%)`	⬆️
lightning/src/offers/refund.rs	`94.36% <92.80%> (-0.07%)`	⬇️
lightning/src/offers/invoice_request.rs	`93.40% <95.76%> (+0.55%)`	⬆️
lightning/src/offers/test_utils.rs	`96.72% <96.72%> (ø)`
lightning/src/offers/merkle.rs	`99.40% <100.00%> (+0.10%)`	⬆️
... and 1 more

... and 3 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

TheBlueMatt · 2023-01-26T19:51:18Z

Discussed very briefly offline - for metadata verification we, I think, need to be able to sign with both our node and and an auto-derived one. That means, I think, that we should do the verification of the metadata without the signing key at all, but then also derive the signing key from that metadata if we're not using our node_id. That metadata should probably be some kind of HMAC over the rest of the offer data plus a nonce.

jkczyz · 2023-01-26T23:39:50Z

Need to check my understanding here...

Discussed very briefly offline - for metadata verification we, I think, need to be able to sign with both our node and and an auto-derived one.

How should we represent this in the OfferBuilder interface? Currently, OfferBuilder::new requires a PublicKey since OfferContents::signing_pubkey is not an Option and would need to be initialized to something.

We could make OfferContents::signing_pubkey an Option and live with unwrapping internally. Then we could have OfferBuilder::new take an enum like:

enum SigningPubkey {
    Explicit(PublicKey),
    Derived,
}

Which at very least could be used in the builder to determine if signing_pubkey should be set to None and derived when calling build. Storing the enum directly in OfferContents instead would have implications for OfferContents::signing_pubkey method, OfferContents::as_tvl_stream, writing, and parsing, so I'd want to avoid that at least in favor of Option<PublicKey>.

That means, I think, that we should do the verification of the metadata without the signing key at all, but then also derive the signing key from that metadata if we're not using our node_id.

If a user gives an explicit pubkey and their own metadata, I assume we can't verify then? I guess this is expected and would mean they couldn't use our onion message handler to respond to the InvoiceRequest. So, a user MUST call metadata_derived if using SigningPubkey::Derived and otherwise MAY call metadata_derived, it seems.

That metadata should probably be some kind of HMAC over the rest of the offer data plus a nonce.

Do you mean metadata = HMAC(rest of offer) + nonce? In which case, when responding, the nonce is used to derive the signing key after verifying the HMAC using the ExpandedKey and the appropriate offer TLV records, I assume? Let me know if I'm missing something.

jkczyz · 2023-01-28T00:50:21Z

I was able to make the above sketched out change without using an Option for OfferContents::signing_pubkey by changing the Derived variant to Derived(&'a ExpandedKey, Nonce). This let's us derive the signing_pubkey immediately and add the nonce later to the metadata. This also means OfferBuilder::metadata_derived doesn't need to be called when using this variant.

Looking for feedback on the signing_pubkey derivation, though, and earlier questions related to it. The latest push has it implemented.

TheBlueMatt · 2023-01-30T20:18:32Z

I guess I'm confused - do we want the OfferBuilder to be tied to the LDK-specific offer metadata, or do we want to do that at a higher level? I had recalled previously that you said you wanted them to be separate.

TheBlueMatt · 2023-01-30T20:24:13Z

lightning/src/ln/inbound_payment.rs

+	///
+	/// [`Offer::metadata`]: crate::offers::offer::Offer::metadata
+	pub(crate) fn hmac_for_offer(&self, nonce: Nonce) -> HmacEngine<Sha256> {
+		let mut hmac = HmacEngine::<Sha256>::new(&self.ldk_pmt_hash_key);


Obviously needs a new key for this.

lightning/src/ln/inbound_payment.rs

TheBlueMatt · 2023-01-30T20:30:00Z

lightning/src/offers/offer.rs

@@ -102,13 +120,21 @@ impl OfferBuilder {
 	/// while the offer is valid.
 	///
 	/// Use a different pubkey per offer to avoid correlating offers.
-	pub fn new(description: String, signing_pubkey: PublicKey) -> Self {
+	pub fn new(description: String, signing_pubkey: SigningPubkey) -> Self {


I guess I'm confused why we need to take the two variants here - they're handled the same internally?

When using the Explicit variant, the metadata will only be generated if metadata_derived is called. Two use cases:

LDK-independent usage (must call metadata)

LDK usage with public node id (must call metadata_derived)

When using Derived, the metadata will be generated without needing to call anything. See tests.

Note that, as written, if a user gives SigningPubkey::Explicit and calls OfferBuilder::metadata_derived with an LDK-independent ExpandedKey, then InvoiceRequest::verify should work correctly. So it could be useful leaving this exposed for general usage.

I guess I'm confused why add the complexity when the LDK internal usage can just always call metadata_derived? I'm not sure I understand why we need two ways to call metadata_derived.

You'd have to ensure that the nonce used to create the signing pubkey is the same one passed to OfferBuilder::metadata_derived. With the current approach the builder ensures that it is the same.

What if we instead move the signing_pubkey argument and the metadata type argument to the final build method? Then we could make it all one argument - either explicit for both or implicit for both.

We'd have to make OfferContents::signing_pubkey an Option if we did so.

Can we use a BuildingOffer struct or so? The current API just seems really confusing - users have to provide an enum variant at OfferBuilder constructor which can be explicit, in which case they can call the manual metadata function, or it can be derived, in which case they cant (or dont need to) call either metadata function, and if they provide derived they cant call the derived-metadata function (unless its with the same data) or they'll get the wrong pubkey at the end?

What if we instead move the signing_pubkey argument and the metadata type argument to the final build method? Then we could make it all one argument - either explicit for both or implicit for both.

Note that metadata is optional, so it isn't required when given an explicit signing_pubkey. Also, that approach doesn't support verification for when an explicit signing_pubkey is given (node id or otherwise), FWIW.

Can we use a BuildingOffer struct or so?

Meh, not sure if I really want to repeat the fields and need to do something similar in all the other types, which may have nested contents. Seems error-prone.

The current API just seems really confusing - users have to provide an enum variant at OfferBuilder constructor which can be explicit, in which case they can call the manual metadata function, or it can be derived, in which case they cant (or dont need to) call either metadata function, and if they provide derived they cant call the derived-metadata function (unless its with the same data) or they'll get the wrong pubkey at the end?

Hmmm... if we think it will be confusing, then I'm more inclined to keep the constructor as it was and add a pub(crate) constructor for the derived variant taking &ExpandedKey and Nonce.

Then the question would be whether or not we want to support verification externally. If no, then OfferBuilder::metadata_derived is only needed for when used internally with a node id. So we can either make it pub(crate) or replace it with another pub(crate) constructor.

My feeling is that LDK-independent users likely would just want to set the signing_pubkey and metadata directly and won't care about the derived versions or verification. If true, I'd lean towards the pub(crate) approached outlined above.

Hmmm... if we think it will be confusing, then I'm more inclined to keep the constructor as it was and add a pub(crate) constructor for the derived variant taking &ExpandedKey and Nonce.

I'm fine with this.

Then the question would be whether or not we want to support verification externally. If no, then OfferBuilder::metadata_derived is only needed for when used internally with a node id. So we can either make it pub(crate) or replace it with another pub(crate) constructor.

Also fine with it not being exposed.

My feeling is that LDK-independent users likely would just want to set the signing_pubkey and metadata directly and won't care about the derived versions or verification. If true, I'd lean towards the pub(crate) approached outlined above.

Yep, would agree.

lightning/src/ln/inbound_payment.rs

jkczyz · 2023-01-30T21:11:41Z

I guess I'm confused - do we want the OfferBuilder to be tied to the LDK-specific offer metadata, or do we want to do that at a higher level? I had recalled previously that you said you wanted them to be separate.

Turns out it is much cleaner to incorporate it into the builder. But it still allows users to set it explicitly by giving SigningPubkey::Explicit and calling OfferBuilder::metadata. We could make dedicated methods and use pub(crate) where applicable, of course, if we want to hide it from the public interface.

jkczyz · 2023-01-30T21:51:36Z

FYI, pushed some commits with clean-up and also using SigningPubkey for payer metadata in InvoiceRequest.

jkczyz · 2023-02-15T19:30:57Z

While implementing message handling base on this PR, I came across a few decisions that need to be made. @TheBlueMatt @valentinewallace Let me know if you have any thoughts on the following.

`ExpandedKey` ownership

ExpandedKey is needed at both creation and handling time for constructing and verifying the metadata, respectively. ExpandedKey is constructed from KeyMaterial retrieved from a NodeSigner. Currently, ChannelManager constructs and owns the ExpandedKey.

For builder creation, we can add methods to ChannelManager so that it retains ExpandedKey ownership. Since ChannelManager also has an EntropySource, it can create a Nonce and construct a DerivedPubkey for the builder.

pub fn create_offer_builder(&self, description: String) -> OfferBuilder {
    let nonce = inbound_payment::Nonce::from_entropy_source(&*self.entropy_source);
    let signing_pubkey = DerivedPubkey::new(&self.inbound_payment_key, nonce);
    OfferBuilder::deriving_signing_pubkey(description, signing_pubkey)
 }

But for verification, either OnionMessenger or its internal handler will need the ExpandedKey. A few options come to mind:

reconstructed the ExpandedKey such that OnionMessenger or its internal handler has a copy
have a shared reference, moving the creation of ExpandedKey out of ChannelManager entirely
give OnionMessenger a reference to the ChannelManager for verification
delegate verification to the user via an event instead, requiring them to call ChannelManager for verification

Message handling flow

Onion messages containing either an InvoiceRequest or an Invoice need to be handled by sending an Invoice or making a payment, respectively, or by sending back an InvoiceError.

For InvoiceRequest, it can theoretically be verified and have an Invoice sent as a response automatically given we don't support creating offers with Amount::Currency.

For Invoice, the payment could be similarly sent without any user interaction. However, if the invoice is for an offer using Amount::Currency without an explicit amount set in the InvoiceRequest, then we may want to surface an InvoiceReceived event to the user to allow them to confirm the amount before making payment.

Note, also that handling will require constructing blinded reply paths or blinded payment paths, which indicates the need for a ChannelManager with a Router. Also, access to the ExpandedKey when constructing an Invoice and a manner to sign the Invoice using a DerivedPubkey. With that in mind, we may need to provide indication to the user of failure to create an Invoice as well (e.g., if a blinded path could not be formed). Could possibly use an event if automating?

The question here is then somewhat related to the ExpandedKey ownership, since it is needed when constructing an Invoice, too. Should we have the flow be event-centric, with user calling methods on ChannelManager / OnionMessenger when handling? OnionMessenger would then need to be an EventsProvider, IIUC.

valentinewallace · 2023-02-15T21:30:15Z

reconstructed the ExpandedKey such that OnionMessenger or its internal handler has a copy

This seems fine to me since OnionMessenger has a node_signer already.

However, if the invoice is for an offer using Amount::Currency without an explicit amount set in the InvoiceRequest, then we may want to surface an InvoiceReceived event to the user to allow them to confirm the amount before making payment.

This sounds like a stopgap before real support for Amount::Currency, at which point we could remove it? It seems preferable to just refuse to pay such offers for the time being 🤔

With that in mind, we may need to provide indication to the user of failure to create an Invoice as well (e.g., if a blinded path could not be formed). Could possibly use an event if automating?

Maybe we could expand PaymentFailed to cover this?

TheBlueMatt · 2023-02-15T23:24:04Z

Somehow I was thinking ChannelManager would be that "internal handler" - that OnionMessenger would have a Bolt12MessageHandler trait, which ChannelManager would implement. Because invoices have to find their way to the ChannelManager for payment anyway this makes sense IMO - if we have an offer we want to pay, tell ChannelManager, it will generate and send the invoice_request as an OnionMessageHandler and then when it gets the invoice it already knows about it and can pay it.

If we didn't want to do this we'd probably change the Bolt12OnionMessageHandler to only include invoice messages and handle invoice_requests via some other struct, but it seems simpler to just do it all in one?

jkczyz · 2023-02-16T16:25:58Z

Somehow I was thinking ChannelManager would be that "internal handler" - that OnionMessenger would have a Bolt12MessageHandler trait, which ChannelManager would implement. Because invoices have to find their way to the ChannelManager for payment anyway this makes sense IMO - if we have an offer we want to pay, tell ChannelManager, it will generate and send the invoice_request as an OnionMessageHandler and then when it gets the invoice it already knows about it and can pay it.

Yeah, this seems like the right approach. Then the user can choose to implement their own handling if they aren't using the stateless verification. I have this implemented in #2039 for reference. The handling method will need to either take an &OnionMessenger so it can call send_onion_message in response or return a response for OnionMessenger to send for them, IIUC.

Still an open question is if we want to expose an InvoiceReceived event to the users for the Currency::Amount case, or fail to pay for now as @valentinewallace suggested.

jkczyz · 2023-02-28T23:40:42Z

@valentinewallace @TheBlueMatt Have some questions / comments mostly related to the follow-up #2039, which now has the most recent push.

Message signing

I added a sign_bolt12_message method to NodeSigner and additional parameters to the sign methods in the offers module, including the metadata. This is need to extract the Nonce in order to sign an Invoice for a particular Offer, for instance. NodeSigner has the get_inbound_payment_key_material method used to create the ExpandedKey, so KeysManager will need to recreate the ExpandedKey when signing (or store it on creation). That means we still need the ExpandedKey in two places.

Blinded paths

I have utility methods for creating offers, etc. in ChannelManager, which will need to set blinded paths. Additionally, the OffersMessageHandler implementation for ChannelManager also will need to do so when creating an Invoice when handling an InvoiceRequest and will need BlindedPayInfo for each path.

Since blinded paths need to be at least two hops, we'll need a NetworkGraph at very least to construct these. This likely means ChannelManager needs another parameter. 😕 Or should we add methods to the Router trait? This isn't for finding a full path, only the blinded portion.

valentinewallace · 2023-03-01T17:33:51Z

Since blinded paths need to be at least two hops, we'll need a NetworkGraph at very least to construct these. This likely means ChannelManager needs another parameter. 😕 Or should we add methods to the Router trait? This isn't for finding a full path, only the blinded portion.

Conceptually, it would make sense to me to add method(s) to the Router trait.

jkczyz · 2023-03-02T18:27:45Z

Rebased

TheBlueMatt · 2023-03-06T20:38:59Z

If we're including a unique signing pk, cant we just use that as the metadata? ie take all the fields in the offer, hash them with the hmac, then treat that as the signing secret key and check that the signing pk in the offer matches. If it does, we have authenticated without needing the extra bytes in the offer/qr code.

jkczyz · 2023-03-06T22:21:53Z

If we're including a unique signing pk, cant we just use that as the metadata? ie take all the fields in the offer, hash them with the hmac, then treat that as the signing secret key and check that the signing pk in the offer matches. If it does, we have authenticated without needing the extra bytes in the offer/qr code.

IIUC, then we'd still need the nonce in the metadata but could drop the Sha256Hash. Or am I misunderstanding?

Also, FWIW, if we can't form a blinded path, we'd need to use the node pubkey instead of unique signing key. So in that case we couldn't do the check. That is, when using OfferBuilder::new and OfferBuilder::metadata_derived instead of only OfferBuilder::deriving_signing_pubkey.

TheBlueMatt

Two comments, otherwise looks good!

TheBlueMatt · 2023-04-19T01:18:56Z

lightning/src/offers/offer.rs

+				let tlv_stream = TlvStream::new(bytes).range(OFFER_TYPES).filter(|record| {
+					match record.r#type {
+						OFFER_METADATA_TYPE => false,
+						OFFER_NODE_ID_TYPE => !self.metadata.as_ref().unwrap().derives_keys(),


Isn't this always false because we were deserialized so our metadata is Bytes? (and in invoice_request/refund/invoice)

Metadata::derives_keys was updated in the first commit that introduced verification to check the length of the metadata.

lightning/src/offers/signer.rs

TheBlueMatt

In a followup we should add the verify calls to the fuzzers, though I'm not too worried about it here specifically.

valentinewallace

LGTM, feel free to address nits in follow-up

lightning/src/offers/signer.rs

lightning/src/offers/parse.rs

Add support for deriving a transient signing pubkey for each Offer from an ExpandedKey and a nonce. This facilitates recipient privacy by not tying any Offer to any other nor to the recipient's node id. Additionally, support stateless Offer verification by setting its metadata using an HMAC over the nonce and the remaining TLV records, which will be later verified when receiving an InvoiceRequest.

Add an iterator that yields TlvRecords over a range of a TlvStream. Useful for verifying that, e.g., an InvoiceRequest was sent in response to an Offer constructed by the intended recipient.

Verify that an InvoiceRequest was produced from an Offer constructed by the recipient using the Offer metadata reflected in the InvoiceRequest. The Offer metadata consists of a 128-bit encrypted nonce and possibly a 256-bit HMAC over the nonce and Offer TLV records (excluding the signing pubkey) using an ExpandedKey. Thus, the HMAC can be reproduced from the offer bytes using the nonce and the original ExpandedKey, and then checked against the metadata. If metadata does not contain an HMAC, then the reproduced HMAC was used to form the signing keys, and thus can be checked against the signing pubkey.

InvoiceRequestBuilder has a field containing InvoiceRequestContents. When deriving the payer_id from the remaining fields, a struct is needed without payer_id as it not optional. Refactor InvoiceRequestContents to have an inner struct without the payer_id such that InvoiceRequestBuilder can use it instead.

Add support for deriving a transient payer id for each InvoiceRequest from an ExpandedKey and a nonce. This facilitates payer privacy by not tying any InvoiceRequest to any other nor to the payer's node id. Additionally, support stateless Invoice verification by setting payer metadata using an HMAC over the nonce and the remaining TLV records, which will be later verified when receiving an Invoice response.

Verify that an Invoice was produced from an InvoiceRequest constructed by the payer using the payer metadata reflected in the Invoice. The payer metadata consists of a 128-bit encrypted nonce and possibly a 256-bit HMAC over the nonce and InvoiceRequest TLV records (excluding the payer id) using an ExpandedKey. Thus, the HMAC can be reproduced from the invoice request bytes using the nonce and the original ExpandedKey, and then checked against the metadata. If metadata does not contain an HMAC, then the reproduced HMAC was used to form the signing keys, and thus can be checked against the payer id.

Add support for deriving a transient payer id for each Refund from an ExpandedKey and a nonce. This facilitates payer privacy by not tying any Refund to any other nor to the payer's node id. Additionally, support stateless Invoice verification by setting payer metadata using an HMAC over the nonce and the remaining TLV records, which will be later verified when receiving an Invoice response.

Stateless verification of Invoice for Offer Verify that an Invoice was produced from a Refund constructed by the payer using the payer metadata reflected in the Invoice. The payer metadata consists of a 128-bit encrypted nonce and possibly a 256-bit HMAC over the nonce and Refund TLV records (excluding the payer id) using an ExpandedKey. Thus, the HMAC can be reproduced from the refund bytes using the nonce and the original ExpandedKey, and then checked against the metadata. If metadata does not contain an HMAC, then the reproduced HMAC was used to form the signing keys, and thus can be checked against the payer id.

For offers where the signing pubkey is derived, the keys need to be extracted from the Offer::metadata in order to sign an invoice. Parameterize InvoiceBuilder such that a build_and_sign method is available for this situation.

0.0.115 - Apr 24, 2023 - "Rebroadcast the Bugfixes" API Updates =========== * The MSRV of the main LDK crates has been increased to 1.48 (lightningdevkit#2107). * Attempting to claim an un-expired payment on a channel which has closed no longer fails. The expiry time of payments is exposed via `PaymentClaimable::claim_deadline` (lightningdevkit#2148). * `payment_metadata` is now supported in `Invoice` deserialization, sending, and receiving (via a new `RecipientOnionFields` struct) (lightningdevkit#2139, lightningdevkit#2127). * `Event::PaymentFailed` now exposes a failure reason (lightningdevkit#2142). * BOLT12 messages now support stateless generation and validation (lightningdevkit#1989). * The `NetworkGraph` is now pruned of stale data after RGS processing (lightningdevkit#2161). * Max inbound HTLCs in-flight can be changed in the handshake config (lightningdevkit#2138). * `lightning-transaction-sync` feature `esplora-async-https` was added (lightningdevkit#2085). * A `ChannelPending` event is now emitted after the initial handshake (lightningdevkit#2098). * `PaymentForwarded::outbound_amount_forwarded_msat` was added (lightningdevkit#2136). * `ChannelManager::list_channels_by_counterparty` was added (lightningdevkit#2079). * `ChannelDetails::feerate_sat_per_1000_weight` was added (lightningdevkit#2094). * `Invoice::fallback_addresses` was added to fetch `bitcoin` types (lightningdevkit#2023). * The offer/refund description is now exposed in `Invoice{,Request}` (lightningdevkit#2206). Backwards Compatibility ======================= * Payments sent with the legacy `*_with_route` methods on LDK 0.0.115+ will no longer be retryable via the LDK 0.0.114- `retry_payment` method (lightningdevkit#2139). * `Event::PaymentPathFailed::retry` was removed and will always be `None` for payments initiated on 0.0.115 which fail on an earlier version (lightningdevkit#2063). * `Route`s and `PaymentParameters` with blinded path information will not be readable on prior versions of LDK. Such objects are not currently constructed by LDK, but may be when processing BOLT12 data in a coming release (lightningdevkit#2146). * Providing `ChannelMonitorUpdate`s generated by LDK 0.0.115 to a `ChannelMonitor` on 0.0.114 or before may panic (lightningdevkit#2059). Note that this is in general unsupported, and included here only for completeness. Bug Fixes ========= * Fixed a case where `process_events_async` may `poll` a `Future` which has already completed (lightningdevkit#2081). * Fixed deserialization of `u16` arrays. This bug may have previously corrupted the historical buckets in a `ProbabilisticScorer`. Users relying on the historical buckets may wish to wipe their scorer on upgrade to remove corrupt data rather than waiting on it to decay (lightningdevkit#2191). * The `process_events_async` task is now `Send` and can thus be polled on a multi-threaded runtime (lightningdevkit#2199). * Fixed a missing macro export causing `impl_writeable_tlv_based_enum{,_upgradable}` calls to not compile (lightningdevkit#2091). * Fixed compilation of `lightning-invoice` with both `no-std` and serde (lightningdevkit#2187) * Fix an issue where the `background-processor` would not wake when a `ChannelMonitorUpdate` completed asynchronously, causing delays (lightningdevkit#2090). * Fix an issue where `process_events_async` would exit immediately (lightningdevkit#2145). * `Router` calls from the `ChannelManager` now call `find_route_with_id` rather than `find_route`, as was intended and described in the API (lightningdevkit#2092). * Ensure `process_events_async` always exits if any sleep future returns true, not just if all sleep futures repeatedly return true (lightningdevkit#2145). * `channel_update` messages no longer set the disable bit unless the peer has been disconnected for some time. This should resolve cases where channels are disabled for extended periods of time (lightningdevkit#2198). * We no longer remove CLN nodes from the network graph for violating the BOLT spec in some cases after failing to pay through them (lightningdevkit#2220). * Fixed a debug assertion which may panic under heavy load (lightningdevkit#2172). * `CounterpartyForceClosed::peer_msg` is now wrapped in UntrustedString (lightningdevkit#2114) * Fixed a potential deadlock in `funding_transaction_generated` (lightningdevkit#2158). Security ======== * Transaction re-broadcasting is now substantially more aggressive, including a new regular rebroadcast feature called on a timer from the `background-processor` or from `ChainMonitor::rebroadcast_pending_claims`. This should substantially increase transaction confirmation reliability without relying on downstream `TransactionBroadcaster` implementations for rebroadcasting (lightningdevkit#2203, lightningdevkit#2205, lightningdevkit#2208). * Implemented the changes from BOLT PRs lightningdevkit#1031, lightningdevkit#1032, and lightningdevkit#1040 which resolve a privacy vulnerability which allows an intermediate node on the path to discover the final destination for a payment (lightningdevkit#2062). In total, this release features 110 files changed, 11928 insertions, 6368 deletions in 215 commits from 21 authors, in alphabetical order: * Advait * Alan Cohen * Alec Chen * Allan Douglas R. de Oliveira * Arik Sosman * Elias Rohrer * Evan Feenstra * Jeffrey Czyz * John Cantrell * Lucas Soriano del Pino * Marc Tyndel * Matt Corallo * Paul Miller * Steven * Steven Williamson * Steven Zhao * Tony Giorgio * Valentine Wallace * Wilmer Paulino * benthecarman * munjesi

jkczyz force-pushed the 2023-01-stateless-offers branch from 830a2e2 to 2630fc8 Compare January 28, 2023 00:42

TheBlueMatt reviewed Jan 30, 2023

View reviewed changes

lightning/src/ln/inbound_payment.rs Outdated Show resolved Hide resolved

jkczyz force-pushed the 2023-01-stateless-offers branch from 2630fc8 to adcfbd6 Compare January 30, 2023 21:45

jkczyz force-pushed the 2023-01-stateless-offers branch from adcfbd6 to 9e36915 Compare February 9, 2023 20:39

jkczyz changed the title ~~Stateless BOLT 12 Offer verification~~ Stateless BOLT 12 message verification Feb 9, 2023

jkczyz force-pushed the 2023-01-stateless-offers branch from 9e36915 to de86b2c Compare February 9, 2023 21:27

jkczyz marked this pull request as ready for review February 11, 2023 20:41

jkczyz force-pushed the 2023-01-stateless-offers branch from de86b2c to a9a69a0 Compare February 14, 2023 04:39

valentinewallace added the blocked on dependent pr label Feb 14, 2023

jkczyz mentioned this pull request Feb 16, 2023

BOLT 12 Offers message flow #2039

Merged

jkczyz force-pushed the 2023-01-stateless-offers branch from a9a69a0 to 0c52188 Compare February 28, 2023 23:20

jkczyz force-pushed the 2023-01-stateless-offers branch from 0c52188 to 673541e Compare March 2, 2023 18:27

valentinewallace removed the blocked on dependent pr label Mar 6, 2023

jkczyz force-pushed the 2023-01-stateless-offers branch 2 times, most recently from 0d7f06b to f44ba89 Compare April 18, 2023 23:43

TheBlueMatt reviewed Apr 19, 2023

View reviewed changes

jkczyz force-pushed the 2023-01-stateless-offers branch from f44ba89 to 5f8ad79 Compare April 19, 2023 03:17

TheBlueMatt previously approved these changes Apr 19, 2023

View reviewed changes

TheBlueMatt assigned valentinewallace Apr 19, 2023

valentinewallace previously approved these changes Apr 19, 2023

View reviewed changes

lightning/src/offers/signer.rs Show resolved Hide resolved

lightning/src/offers/signer.rs Outdated Show resolved Hide resolved

lightning/src/offers/parse.rs Outdated Show resolved Hide resolved

jkczyz added 13 commits April 19, 2023 21:06

TlvStream range iterator

b2e87ff

Add an iterator that yields TlvRecords over a range of a TlvStream. Useful for verifying that, e.g., an InvoiceRequest was sent in response to an Offer constructed by the intended recipient.

Fix builder docs in offers module

9bd43e0

DRY up verification of invreq TLV records

259aa9a

Extract keys from Offer::metadata to sign Invoice

e1a6bc3

For offers where the signing pubkey is derived, the keys need to be extracted from the Offer::metadata in order to sign an invoice. Parameterize InvoiceBuilder such that a build_and_sign method is available for this situation.

Support responding to refunds with transient keys

c8a847a

DRY up InvoiceFields construction

8afe694

jkczyz dismissed stale reviews from valentinewallace and TheBlueMatt via 8afe694 April 20, 2023 02:31

jkczyz force-pushed the 2023-01-stateless-offers branch from 5f8ad79 to 8afe694 Compare April 20, 2023 02:31

TheBlueMatt approved these changes Apr 20, 2023

View reviewed changes

valentinewallace approved these changes Apr 20, 2023

View reviewed changes

TheBlueMatt merged commit b8ed4d2 into lightningdevkit:main Apr 20, 2023

jkczyz mentioned this pull request May 10, 2023

BOLT12 Offers Tracking Issue #1970

Open

60 tasks

jkczyz mentioned this pull request Jul 25, 2023

Support signing BOLT 12 messages in NodeSigner #2432

Merged

valentinewallace mentioned this pull request Aug 8, 2023

BOLT 12 outbound PaymentId #2468

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Stateless BOLT 12 message verification #1989

Stateless BOLT 12 message verification #1989

jkczyz commented Jan 26, 2023 •

edited

Loading

codecov-commenter commented Jan 26, 2023 •

edited

Loading

TheBlueMatt commented Jan 26, 2023 •

edited

Loading

jkczyz commented Jan 26, 2023

jkczyz commented Jan 28, 2023

TheBlueMatt commented Jan 30, 2023

TheBlueMatt Jan 30, 2023

TheBlueMatt Jan 30, 2023

jkczyz Jan 30, 2023

jkczyz Jan 30, 2023

TheBlueMatt Jan 30, 2023

jkczyz Jan 30, 2023

TheBlueMatt Jan 31, 2023 •

edited

Loading

jkczyz Jan 31, 2023 •

edited

Loading

TheBlueMatt Jan 31, 2023

jkczyz Jan 31, 2023

TheBlueMatt Jan 31, 2023

jkczyz commented Jan 30, 2023

jkczyz commented Jan 30, 2023

jkczyz commented Feb 15, 2023

valentinewallace commented Feb 15, 2023

TheBlueMatt commented Feb 15, 2023

jkczyz commented Feb 16, 2023

jkczyz commented Feb 28, 2023

valentinewallace commented Mar 1, 2023

jkczyz commented Mar 2, 2023

TheBlueMatt commented Mar 6, 2023

jkczyz commented Mar 6, 2023

TheBlueMatt left a comment

TheBlueMatt Apr 19, 2023

jkczyz Apr 19, 2023

TheBlueMatt left a comment

valentinewallace left a comment

Stateless BOLT 12 message verification #1989

Stateless BOLT 12 message verification #1989

Conversation

jkczyz commented Jan 26, 2023 • edited Loading

codecov-commenter commented Jan 26, 2023 • edited Loading

Codecov Report

TheBlueMatt commented Jan 26, 2023 • edited Loading

jkczyz commented Jan 26, 2023

jkczyz commented Jan 28, 2023

TheBlueMatt commented Jan 30, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TheBlueMatt Jan 31, 2023 • edited Loading

Choose a reason for hiding this comment

jkczyz Jan 31, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jkczyz commented Jan 30, 2023

jkczyz commented Jan 30, 2023

jkczyz commented Feb 15, 2023

ExpandedKey ownership

Message handling flow

valentinewallace commented Feb 15, 2023

TheBlueMatt commented Feb 15, 2023

jkczyz commented Feb 16, 2023

jkczyz commented Feb 28, 2023

Message signing

Blinded paths

valentinewallace commented Mar 1, 2023

jkczyz commented Mar 2, 2023

TheBlueMatt commented Mar 6, 2023

jkczyz commented Mar 6, 2023

TheBlueMatt left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TheBlueMatt left a comment

Choose a reason for hiding this comment

valentinewallace left a comment

Choose a reason for hiding this comment

jkczyz commented Jan 26, 2023 •

edited

Loading

codecov-commenter commented Jan 26, 2023 •

edited

Loading

TheBlueMatt commented Jan 26, 2023 •

edited

Loading

TheBlueMatt Jan 31, 2023 •

edited

Loading

jkczyz Jan 31, 2023 •

edited

Loading

`ExpandedKey` ownership