-
Notifications
You must be signed in to change notification settings - Fork 364
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add OutputSweeper
utility persisting and sweeping spendable outputs
#2825
Add OutputSweeper
utility persisting and sweeping spendable outputs
#2825
Conversation
Note Reviews PausedUse the following commands to manage reviews:
WalkthroughThe codebase introduces an Changes
Poem
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configration File (
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Review Status
Actionable comments generated: 0
Configuration used: CodeRabbit UI
Files selected for processing (3)
- lightning/src/util/mod.rs (1 hunks)
- lightning/src/util/persist.rs (1 hunks)
- lightning/src/util/sweep.rs (1 hunks)
Additional comments: 15
lightning/src/util/mod.rs (1)
- 24-24: The addition of the
sweep
module is noted and appears to be correctly placed within theutil
module structure.lightning/src/util/sweep.rs (13)
7-10: The module documentation for
OutputSweeper
clearly explains its purpose and functionality.11-23: The imports and dependencies for the
OutputSweeper
are correctly listed and seem appropriate for the functionality described.30-31: The constant
DEFAULT_REGENERATE_SPEND_THRESHOLD
is well-documented and its purpose is clear.33-68: The
TrackedSpendableOutput
struct is well-defined with clear documentation for each field. It encapsulates the state of a spendable output.70-108: The methods
to_watched_output
andis_spent_in
withinTrackedSpendableOutput
are logically sound and well-implemented.110-119: The implementation of
impl_writeable_tlv_based!
forTrackedSpendableOutput
is correct and ensures that the struct can be serialized and deserialized.121-288: The
OutputSweeper
struct and its constructor methods are well-documented and structured. The use of generics and trait bounds is appropriate, and the code is clean and maintainable.291-341: The method
track_spendable_outputs
is logically sound, with clear handling of theexclude_static_outputs
flag. Error handling with logging is also correctly implemented.343-346: The method
tracked_spendable_outputs
provides a simple and effective way to retrieve the current state of tracked outputs.348-428: The method
rebroadcast_if_necessary
contains logic to rebroadcast transactions as needed. The logic is sound, but it's important to ensure that the error handling in the persistence calls does not inadvertently swallow errors that should cause a failure in the calling code.Verification successful
The
persist_info
method in theOutputSweeper
utility logs errors encountered during the persistence process and then returns those errors to the caller. This means that the method does not swallow errors; instead, it allows the calling code to handle the error as deemed appropriate. This addresses the concern raised in the review comment regarding error handling.* 430-485: The method `prune_confirmed_outputs` and `persist_info` are well-implemented with appropriate error handling and logging.Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Verify that the error handling in the persistence calls of rebroadcast_if_necessary does not swallow critical errors. # This script is a placeholder for the actual verification process, which would involve a deeper review of the error handling strategy. echo "Review the error handling strategy for persistence calls in rebroadcast_if_necessary."Length of output: 180
Script:
#!/bin/bash # Examine the implementation of `self.persist_info` to understand how errors are handled. ast-grep --lang rust --pattern 'impl OutputSweeper { $$$ fn persist_info($_: &$_) -> $_ { $$$ } $$$ }'Length of output: 116
Script:
#!/bin/bash # Use rg to search for the persist_info method within the OutputSweeper implementation. rg "fn persist_info"Length of output: 134
Script:
#!/bin/bash # Extract the full implementation of the persist_info method from the sweep.rs file. rg -A 10 "fn persist_info" lightning/src/util/sweep.rsLength of output: 399
Script:
#!/bin/bash # Extend the rg command to capture more lines following the persist_info method to see the complete error handling logic. rg -A 20 "fn persist_info" lightning/src/util/sweep.rsLength of output: 636
488-589: The implementations of the
Listen
andConfirm
traits forOutputSweeper
are correct and handle block connection/disconnection and transaction confirmation/unconfirmation appropriately.613-651: The function
read_spendable_outputs
and the helperid_to_hex_string
are correctly implemented and handle errors appropriately.lightning/src/util/persist.rs (1)
- 68-77: The addition of the constants
SPENDABLE_OUTPUT_INFO_PERSISTENCE_PRIMARY_NAMESPACE
andSPENDABLE_OUTPUT_INFO_PERSISTENCE_SECONDARY_NAMESPACE
is consistent with the existing pattern for defining namespaces for persistence. These constants are used to namespace the persistence ofTrackedSpendableOutput
s by theOutputSweeper
. The primary namespace is set to"spendable_outputs"
, and the secondary namespace is an empty string, which is a valid and common pattern in this codebase.
0df5bcc
to
da27965
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Review Status
Actionable comments generated: 0
Configuration used: CodeRabbit UI
Files selected for processing (3)
- lightning/src/util/mod.rs (1 hunks)
- lightning/src/util/persist.rs (1 hunks)
- lightning/src/util/sweep.rs (1 hunks)
Files skipped from review as they are similar to previous changes (3)
- lightning/src/util/mod.rs
- lightning/src/util/persist.rs
- lightning/src/util/sweep.rs
da27965
to
ed9f8bb
Compare
@coderabbitai pause |
Codecov ReportAttention: Patch coverage is
❗ Your organization needs to install the Codecov GitHub app to enable full functionality. Additional details and impacted files@@ Coverage Diff @@
## main #2825 +/- ##
==========================================
- Coverage 89.42% 89.18% -0.25%
==========================================
Files 117 118 +1
Lines 96290 96920 +630
Branches 96290 96920 +630
==========================================
+ Hits 86109 86437 +328
- Misses 7962 8240 +278
- Partials 2219 2243 +24 ☔ View full report in Codecov by Sentry. |
dc0340e
to
c5052e2
Compare
Added first basic testcase in the |
29b4a7a
to
0626f34
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Concept ACK
and proposed to review club lightningdevkit/ldk-review-club#28
0626f34
to
ca91946
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Basically LGTM, I assume this is already in ldk-node so there's no use commenting on particular storage format?
lightning/src/util/sweep.rs
Outdated
self.outputs.lock().unwrap().clone() | ||
} | ||
|
||
fn rebroadcast_if_necessary(&self) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we make the broadcasting optional here so that users can use the utility to just spend when they need to spend? Would require just letting the user skip this and then refactoring the spend logic to take an extra (few) output(s) when called manually.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not quite sure what you're suggesting. What would be the benefit of skipping the broadcast, and how would it then be different from KeyManager::spend_spendable_output
? Given that broadcasting is always fallible without us getting feedback until we see a confirmation, what would be a scenario where rebroadcasting every block isn't desired? Also, what do you mean with 'extra few outputs'?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
how would it then be different from KeyManager::spend_spendable_output?
Mostly just that there would be a utility to handle the storage for you. I agree its not a big difference and certainly anyone managing their own storage can just do that.
Given that broadcasting is always fallible without us getting feedback until we see a confirmation, what would be a scenario where rebroadcasting every block isn't desired? Also, what do you mean with 'extra few outputs'?
I didn't mean not rebroadcasting, only that it'd be nice to let users delay broadcasting until they want to do some other onchain transaction and can just include the sweeping then.
lightning/src/util/sweep.rs
Outdated
} | ||
|
||
if let Some(latest_broadcast_height) = output_info.latest_broadcast_height { | ||
// Re-generate spending tx after regenerate_spend_threshold, rebroadcast |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we end up with some output that got claimed another way, won't this mean all our future spends are gonna get stuck?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Intermittently yes, but we'll still pick up on any confirmed spends from which point on we wouldn't try to include the outputs in the spending transaction regeneration. (Note that in Confirm::transactions_confirmed
here we don't match on the txid but check for any confirmed transaction whether it spends one of our tracked outputs).
We'd then also prune the 'stuck' outputs once the (potentially third-party) spend is sufficiently confirmed.
If that's not acceptable we could of course fall back to individually retry to spend & broadcast outputs that previously failed, however, I'd rather not go that route if it's not really necessary as generating one spend per output is really wasteful fee-wise.
Yes, essentially. We could of course always find some migration steps if it would be really necessary though. Is there anything in particular that stands out to you? |
ca91946
to
f418234
Compare
Rebased on main to resolve minor conflict. |
f418234
to
3db8480
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Needs another rebase, sorry about that. Basically LGTM, I was just wondering if I should spend time trying to puzzle over a simpler (read: less stateful) design or if this is kinda fixed and we should just ship it as long as its good enough (it more than is).
Alright. Actually I had originally started out with keeping more state and tracking in a proper STM, which had some benefits w.r.t. explicitness of transitions and would give the user access to complimentary information, but that was reduced during the review on LDK Node. If there's no major concern it's probably best to keep it as is (read: not unnecessarily break compat with LDK Node's sweeper). |
3db8480
to
eff8db9
Compare
Rebased on |
f107d77
to
dd09907
Compare
) -> Result<Self, DecodeError> { | ||
let (broadcaster, chain_data_source, kv_store, logger, spend_outputs_callback) = args; | ||
let state = SweeperState::read(reader)?; | ||
let best_block = state.best_block; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wonder if we should enforce this instead by defining a trait for returning the BestBlock
(or even add a best_block
method to chain::Listen
/ chain::Confirm
) such that our utilities query it rather than relying on the user providing it paired with the given type. Returning it as part of Readable
doesn't prevent the user from doing the wrong thing.
if i == num_blocks { | ||
node.node.best_block_updated(&header, height); | ||
node.chain_monitor.best_block_updated(&header, height); | ||
node.sweeper.best_block_updated(&header, height); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are we only testing this using lightning-background-processor
because our functional tests utilities don't allow additional chain::Listen
/chain::Confirm
implementations?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is one reason, but also as we require a 'real' KeysManager
allowing to spend the outputs. Essentially we require most of the core components of a Lightning node, and our BP tests seems to be the only place where all come together?
986bfcd
to
2c8fffc
Compare
Now updated to
|
3ef6e21
to
4dfcfc0
Compare
lightning/src/util/sweep.rs
Outdated
state_lock.best_block = BestBlock::new(header.block_hash(), height); | ||
self.prune_confirmed_outputs(&mut *state_lock); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe not worth it, but could use the result of these to determine if persistence can be skipped. Or do we always need to persist when the best block is updated?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, yeah, previously I left this one alone as I assumed we'd want to persist the BestBlock
. Whether we could do that lazily might be debatable. Note it would only safe a single persist call on each block connect if we don't have any outputs that are currently broadcasted, as otherwise we'd always update latest_broadcast_height
anyways.
If we want to do it lazily, we might need to consider introducing an interval in which we force persisting the best block, as otherwise long-running nodes with low traffic might not persist the BestBlock
for a long time, and we'd then force them to (unnecessarily) resync a lot of chain data on restart.
4dfcfc0
to
e362dac
Compare
Addressed feedback and rebased to resolve minor conflicts. |
/// Will be `None` if no `channel_id` was given to [`OutputSweeper::track_spendable_outputs`] | ||
pub channel_id: Option<ChannelId>, | ||
/// The current status of the output spend. | ||
pub status: OutputSpendStatus, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, so I can't say I'm a huge fan of all the internal state being publicly exposed, it means we don't really do any validation at all of what the user tells us the state of an output is and all the debug assertions in this file are reachable if the user simply provides some garbage.
If we want to expose them all so that users can inspect using tracked_spendable_outputs
maybe we should make from_outputs
#[doc(hidden)]
so that users are discouraged from passing invalid state that we can't verify?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, so I can't say I'm a huge fan of all the internal state being publicly exposed, it means we don't really do any validation at all of what the user tells us the state of an output is and all the debug assertions in this file are reachable if the user simply provides some garbage.
If we want to expose them all so that users can inspect using
tracked_spendable_outputs
maybe we should makefrom_outputs
#[doc(hidden)]
so that users are discouraged from passing invalid state that we can't verify?
I think we def. want to expose the TrackedSpendableOutput
s to allow users insight into what state their funds are in. Currently (and otherwise going forward), the funds just disappear and reappear at some magic time in the user's on-chain wallet if everything goes according to plan. To make this more transparent, I think it's important to give full insight into what's happening (and even allow them to grab the Transaction
and broadcast it out-of-band if something goes wrong). This is one reason why I previously was hesitant to make too many assumptions about what status a transaction is in, if we don't have explicit data on it.
That said, if you're worried users would feed us bogus data, we can probably drop from_outputs
from the API now that we implemented ReadableArgs
on OutputSweeper
. Now added a fixup doing so.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That said, if you're worried users would feed us bogus data, we can probably drop from_outputs from the API now that we implemented ReadableArgs on OutputSweeper. Now added a fixup doing so.
Don't you need it for ldk-node?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't you need it for ldk-node?
By now I kind of gave up on the idea of reconstructing the new data types from the old persisted state and would probably just re-add the persisted descriptors without any prior history, which I think it should work fine as a one-time migration step and seems cleaner than having a hidden API call lying around here.
} | ||
|
||
/// Returns whether the output is spent in the given transaction. | ||
pub fn is_spent_in(&self, tx: &Transaction) -> bool { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What's the reason for this to be pub?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For one it it may be a useful util to have. For example, I used it in the tests in BP. If you'd rather not have it pub
, happy to either copy over the code or do a cfg(test)
/cfg(not(test))
dance.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess? Just seems like a kind of random utility to expose.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess? Just seems like a kind of random utility to expose.
True, happy to go either way really. Really not sure what I prefer here (keeping it pub
, copying code, adding cfg
boilerplate). 🤷♂️
e362dac
to
71b3225
Compare
LGTM, feel free to squash. |
.. we move `spend_spendable_outputs` to a new trait which we can easily reuse in `OutputSweeper` later.
.. which users should implement on their on-chain wallet to allow us to retrieve a new change destination script.
71b3225
to
9a24220
Compare
Squashed without further changes. |
We add a `OutputSweeper` utility that allows to track the state of spendable output descriptors as emitted by `Event::SpendableOutputs`. To this end, the `OutputSweeper` persists the necessary information in our `KVStore` and regularly tries to sweep the spendable outputs, removing them after reaching threshold confirmations, i.e., `ANTI_REORG_DELAY`.
.. which enables users to batch output spends.
.. we simply check that the `OutputSweeper` generates a spending tx and that the `TrackedSpendableOutput` is pruned once it reaches `ANTI_REORG_DELAY`.
9a24220
to
be8c0d0
Compare
Force pushed without any net changes moving the |
v0.0.123 - May 08, 2024 - "BOLT12 Dust Sweeping" API Updates =========== * To reduce risk of force-closures and improve HTLC reliability the default dust exposure limit has been increased to `MaxDustHTLCExposure::FeeRateMultiplier(10_000)`. Users with existing channels might want to consider using `ChannelManager::update_channel_config` to apply the new default (lightningdevkit#3045). * `ChainMonitor::archive_fully_resolved_channel_monitors` is now provided to remove from memory `ChannelMonitor`s that have been fully resolved on-chain and are now not needed. It uses the new `Persist::archive_persisted_channel` to inform the storage layer that such a monitor should be archived (lightningdevkit#2964). * An `OutputSweeper` is now provided which will automatically sweep `SpendableOutputDescriptor`s, retrying until the sweep confirms (lightningdevkit#2825). * After initiating an outbound channel, a peer disconnection no longer results in immediate channel closure. Rather, if the peer is reconnected before the channel times out LDK will automatically retry opening it (lightningdevkit#2725). * `PaymentPurpose` now has separate variants for BOLT12 payments, which include fields from the `invoice_request` as well as the `OfferId` (lightningdevkit#2970). * `ChannelDetails` now includes a list of in-flight HTLCs (lightningdevkit#2442). * `Event::PaymentForwarded` now includes `skimmed_fee_msat` (lightningdevkit#2858). * The `hashbrown` dependency has been upgraded and the use of `ahash` as the no-std hash table hash function has been removed. As a consequence, LDK's `Hash{Map,Set}`s no longer feature several constructors when LDK is built with no-std; see the `util::hash_tables` module instead. On platforms that `getrandom` supports, setting the `possiblyrandom/getrandom` feature flag will ensure hash tables are resistant to HashDoS attacks, though the `possiblyrandom` crate should detect most common platforms (lightningdevkit#2810, lightningdevkit#2891). * `ChannelMonitor`-originated requests to the `ChannelSigner` can now fail and be retried using `ChannelMonitor::signer_unblocked` (lightningdevkit#2816). * `SpendableOutputDescriptor::to_psbt_input` now includes the `witness_script` where available as well as new proprietary data which can be used to re-derive some spending keys from the base key (lightningdevkit#2761, lightningdevkit#3004). * `OutPoint::to_channel_id` has been removed in favor of `ChannelId::v1_from_funding_outpoint` in preparation for v2 channels with a different `ChannelId` derivation scheme (lightningdevkit#2797). * `PeerManager::get_peer_node_ids` has been replaced with `list_peers` and `peer_by_node_id`, which provide more details (lightningdevkit#2905). * `Bolt11Invoice::get_payee_pub_key` is now provided (lightningdevkit#2909). * `Default[Message]Router` now take an `entropy_source` argument (lightningdevkit#2847). * `ClosureReason::HTLCsTimedOut` has been separated out from `ClosureReason::HolderForceClosed` as it is the most common case (lightningdevkit#2887). * `ClosureReason::CooperativeClosure` is now split into `{Counterparty,Locally}Initiated` variants (lightningdevkit#2863). * `Event::ChannelPending::channel_type` is now provided (lightningdevkit#2872). * `PaymentForwarded::{prev,next}_user_channel_id` are now provided (lightningdevkit#2924). * Channel init messages have been refactored towards V2 channels (lightningdevkit#2871). * `BumpTransactionEvent` now contains the channel and counterparty (lightningdevkit#2873). * `util::scid_utils` is now public, with some trivial utilities to examine short channel ids (lightningdevkit#2694). * `DirectedChannelInfo::{source,target}` are now public (lightningdevkit#2870). * Bounds in `lightning-background-processor` were simplified by using `AChannelManager` (lightningdevkit#2963). * The `Persist` impl for `KVStore` no longer requires `Sized`, allowing for the use of `dyn KVStore` as `Persist` (lightningdevkit#2883, lightningdevkit#2976). * `From<PaymentPreimage>` is now implemented for `PaymentHash` (lightningdevkit#2918). * `NodeId::from_slice` is now provided (lightningdevkit#2942). * `ChannelManager` deserialization may now fail with `DangerousValue` when LDK's persistence API was violated (lightningdevkit#2974). Bug Fixes ========= * Excess fees on counterparty commitment transactions are now included in the dust exposure calculation. This lines behavior up with some cases where transaction fees can be burnt, making them effectively dust exposure (lightningdevkit#3045). * `Future`s used as an `std::...::Future` could grow in size unbounded if it was never woken. For those not using async persistence and using the async `lightning-background-processor`, this could cause a memory leak in the `ChainMonitor` (lightningdevkit#2894). * Inbound channel requests that fail in `ChannelManager::accept_inbound_channel` would previously have stalled from the peer's perspective as no `error` message was sent (lightningdevkit#2953). * Blinded path construction has been tuned to select paths more likely to succeed, improving BOLT12 payment reliability (lightningdevkit#2911, lightningdevkit#2912). * After a reorg, `lightning-transaction-sync` could have failed to follow a transaction that LDK needed information about (lightningdevkit#2946). * `RecipientOnionFields`' `custom_tlvs` are now propagated to recipients when paying with blinded paths (lightningdevkit#2975). * `Event::ChannelClosed` is now properly generated and peers are properly notified for all channels that as a part of a batch channel open fail to be funded (lightningdevkit#3029). * In cases where user event processing is substantially delayed such that we complete multiple round-trips with our peers before a `PaymentSent` event is handled and then restart without persisting the `ChannelManager` after having persisted a `ChannelMonitor[Update]`, on startup we may have `Err`d trying to deserialize the `ChannelManager` (lightningdevkit#3021). * If a peer has relatively high latency, `PeerManager` may have failed to establish a connection (lightningdevkit#2993). * `ChannelUpdate` messages broadcasted for our own channel closures are now slightly more robust (lightningdevkit#2731). * Deserializing malformed BOLT11 invoices may have resulted in an integer overflow panic in debug builds (lightningdevkit#3032). * In exceedingly rare cases (no cases of this are known), LDK may have created an invalid serialization for a `ChannelManager` (lightningdevkit#2998). * Message processing latency handling BOLT12 payments has been reduced (lightningdevkit#2881). * Latency in processing `Event::SpendableOutputs` may be reduced (lightningdevkit#3033). Node Compatibility ================== * LDK's blinded paths were inconsistent with other implementations in several ways, which have been addressed (lightningdevkit#2856, lightningdevkit#2936, lightningdevkit#2945). * LDK's messaging blinded paths now support the latest features which some nodes may begin relying on soon (lightningdevkit#2961). * LDK's BOLT12 structs have been updated to support some last-minute changes to the spec (lightningdevkit#3017, lightningdevkit#3018). * CLN v24.02 requires the `gossip_queries` feature for all peers, however LDK by default does not set it for those not using a `P2PGossipSync` (e.g. those using RGS). This change was reverted in CLN v24.02.2 however for now LDK always sets the `gossip_queries` feature. This change is expected to be reverted in a future LDK release (lightningdevkit#2959). Security ======== 0.0.123 fixes a denial-of-service vulnerability which we believe to be reachable from untrusted input when parsing invalid BOLT11 invoices containing non-ASCII characters. * BOLT11 invoices with non-ASCII characters in the human-readable-part may cause an out-of-bounds read attempt leading to a panic (lightningdevkit#3054). Note that all BOLT11 invoices containing non-ASCII characters are invalid. In total, this release features 150 files changed, 19307 insertions, 6306 deletions in 360 commits since 0.0.121 from 17 authors, in alphabetical order: * Arik Sosman * Duncan Dean * Elias Rohrer * Evan Feenstra * Jeffrey Czyz * Keyue Bao * Matt Corallo * Orbital * Sergi Delgado Segura * Valentine Wallace * Willem Van Lint * Wilmer Paulino * benthecarman * jbesraa * olegkubrakov * optout * shaavan
We add a
OutputSweeper
utility that allows to track the state of spendable output descriptors as emitted byEvent::SpendableOutputs
.To this end, the
OutputSweeper
persists the necessary information in ourKVStore
and regularly tries to sweep the spendable outputs, removing them after reaching threshold confirmations, i.e.,ANTI_REORG_DELAY
.This is a generalized version of the module previously added to LDK Node: lightningdevkit/ldk-node#152.
TODOs: