Expose onion messages in public API

[valentinewallace]

This completes the final anti-DoS steps needed for us to be able to make the onion messages API public and integrate the OnionMessenger with the PeerManager. See commit messages for more info.

TODO: add tests for the last commit after it's conceptually ACK'd

Blocked on #1604 and #1648

[valentinewallace]

Now based on #1604

[TheBlueMatt]

Needs rebase.

[codecov-commenter]

Codecov Report

Merging #1650 (447ba29) into main (0624cf9) will increase coverage by 0.13%.
The diff coverage is 90.58%.

❗ Current head 447ba29 differs from pull request most recent head a8ea0bd. Consider uploading reports for the commit a8ea0bd to get more accurate results

@@            Coverage Diff             @@
##             main    #1650      +/-   ##
==========================================
+ Coverage   90.85%   90.98%   +0.13%     
==========================================
  Files          85       85              
  Lines       45948    46681     +733     
  Branches    45948    46681     +733     
==========================================
+ Hits        41746    42474     +728     
- Misses       4202     4207       +5     

Impacted Files	Coverage Δ
lightning/src/lib.rs	100.00% <ø> (ø)
lightning/src/ln/features.rs	99.46% <ø> (ø)
lightning/src/ln/msgs.rs	86.24% <ø> (+0.49%)	⬆️
lightning/src/onion_message/packet.rs	72.52% <50.00%> (+7.69%)	⬆️
lightning/src/onion_message/messenger.rs	89.55% <87.27%> (+0.01%)	⬆️
lightning/src/ln/peer_handler.rs	57.02% <100.00%> (+0.25%)	⬆️
lightning/src/onion_message/functional_tests.rs	96.33% <100.00%> (+0.45%)	⬆️
lightning/src/routing/gossip.rs	91.43% <0.00%> (-0.24%)	⬇️
lightning/src/ln/functional_tests.rs	96.96% <0.00%> (-0.15%)	⬇️
lightning/src/routing/scoring.rs	96.14% <0.00%> (+0.01%)	⬆️
... and 7 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[valentinewallace]

@TheBlueMatt should we only be connecting peers to OnionMessenger if and when we have an open channel? Or does this new approach make that less important?

[TheBlueMatt]

We should consider some kind of only-channel-peers thing in the future, I think, but just as a way to give those peers larger buffers and give other peers almost no buffer space. That said, we need to limit the buffer in OnionMessager to some size. Given we're in OM-specific stuff, let's do it as some number of bytes, not a number of messages (ie we have to walk the pending messages map before deciding to insert). Should be super trivial but needs to happen here I think.

Picking a random number for the buffer limit, let's go with 256KiB for now. Default open files limit on my local machine in front of me is 1024, which means we'd max out at 256MiB in buffer size.

[TheBlueMatt]

Looks like the fuzzing seeds need updating.

[jkczyz]

No major concerns.

[jkczyz]

Could we optimize this by implmenting serialized_length on OnionMessage and Packet? Seems a bit wasteful to serialize all pending messages any time an onion message is sent.

[TheBlueMatt]

I'd put good money on it being free. In the past I've investigated this for a few things and LLVM is pretty good at dealing with the simple cases. As the comment for serialized_length says "Note that LLVM optimizes this away in most cases! Check that it isn't before you override!"

[jkczyz]

Ah, right. Was just looking at the implementation and noticed that comment. Feel free to disregard!