-
Notifications
You must be signed in to change notification settings - Fork 492
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Peer backup storage (feature 40/41/42/43) #881
base: master
Are you sure you want to change the base?
Conversation
Thank you for writing this up! On first read I had assumed that the backup blobs are per-peer but then realised they are actually per-channel. One thing I would like to note is the usecase where, after restoring from seed, wallet software might be able to find some of their channel peers but not the channels themselves. i.e. Alice's wallet might know they have some channels with Bob but not know any of the channel ids (funding outpoints). In such a case Alice can establish a transport with Bob, and then wait for Bob to send one or more One thing I am not clear is the (a)symmetry of this feature. I think this needs to be cleared up. Let's say there is a light client who wants to store backups with their peer (Carol); and let's say the light client does not want to store backups for the other party (or does it need to? it could I guess). Would the light client set I am also wondering if the feature can even be used in a symmetric way. |
Thanks @SomberNight this is very good feedback. I've started with the minimal approach we use in Phoenix, which is simpler than the general case because Phoenix is always fundee and initiates reconnections, but this PR is a good opportunity to offer a backup mechanism that supports more scenarios. I think it would make sense to have two distinct types of backups, which would address the scenario described in your first paragraph:
We could store in the node backup the list of our peers and the channel IDs we have with each peer (encrypted of course). That node backup could be sent to all of our peers: this way we only need to remember and reconnect to one of them to discover the others and the list of our channels. We would need a new message to ask our peers to store an updated version of our Regarding the asymmetry of this feature, I think it is both desired and representative of real-world scenarios. You're completely right to highlight that the case where both peers store each other's backups doesn't work: if one of them loses data, that will force the other to close channels. I don't think this feature is meant to be used between routing nodes running on servers. This type of node should implement DB redundancy internally to ensure they never lose data (instead of relying on their peers for that). I think this feature should only be used for mobile wallets (or more generally light clients) connecting to "service providers". But your comment made me realize that if two service providers connect to each other they will both have the |
Yes, that sounds good. I think it might not be strictly needed, as a workaround could be the light client always waiting for the remote sending
I think one feature bit is sufficient. It could behave as follows, given a transport between Alice and Bob:
One thing to note here is that light clients might want to "require" this feature but there isn't really a way to communicate that fact. Maybe that is fine as they could simply not peer with nodes that do not set it. Another point is about deleting backups. If Alice ("light client") and Bob ("service provider") have a single channel, and Alice stores backups with Bob, and Bob e.g. force-closes the channel, would Bob now delete Alice's backup? I think Bob could keep the backups for some time or maybe even indefinitely (on a best effort basis) - as even then the service seems hard to DOS: opening channels is probably costly enough even if they end up being closed. I suggest the spec include a recommendation about what to do if there no longer are any open channels with a peer but the peer has stored backups with us. One higher level point re use cases: I am wondering what the assumptions are when using this for "resumable channels". e.g. restoring from seed and discovering your existing channels, keeping them open and keep using them. Phoenix does this of course; but in that case the clients probably already have some trust in the ACINQ node... What do you think would be the implications of using it with arbitrary nodes? I think it might only work well if there is at least some trust. Unfortunately I don't think the game theory works out well: if the channel funder is the light client (user), there is virtually no risk for the remote for sending an old blob to the client on reestablish. They only risk missing out on future forwarding fees... their potential gain compared to that is huge :/ |
Yes, I think so as well, adding a second one wouldn't be a very clean "fix".
Exactly, I think that instead of requiring it, light clients would just scan
This is a good point, the spec should mention that storage providers should (best effort) keep the latest backups stored even after closing the channel. That's what eclair currently does, as it doesn't cost much space (and as you mention, it's not subject to DoS since the size of backups is limited by the message size and there is an economic cost in the channel creation). I'll add a section mentioning that.
I agree with you, this backup mechanism isn't completely bullet-proof. You can check that your peer behaves correctly at each I'll work on adding a commit to include the |
@SomberNight I add the |
I like this! But I think we should downgrade it to strictly "best effort", at least for peer_backup.
The way I would use this feature is to store the commitment number in every peer_backup, and update at least three peers on every send of revoke_and_ack. That is the critical piece of information so we don't accidentally go backwards on restore. Then have a rule on restart that insists on a threshold of init msgs before sending reestablish messages. I also think it would be informative (perhaps in a new doc) to indicate the format implementations used: a TLV seems to fit pretty well here, IMHO, but details of encryption matter too. |
Features clash with #759 BTW! |
You're right, I made it best effort in 3129979
It's worth discussing, if we think it makes sense we should include it now in the backup tlv (to avoid adding a new tlv later for it).
I agree, bLIPs are likely a good place for that.
Fixed in 42e68e0 I realized that my calculation for the maximum size for backups is completely wrong... When that happens, there isn't much space available in the Should we still define an arbitrary maximum size for backup data (e.g. 32768 bytes, powers of two are nice) and add a requirement that the sender must not include the backup if that would cause the message to be bigger than the 65535 bytes message size limit? Or should we completely drop the maximum size for backup data and let it be bounded by the message size limit (65535 bytes)? |
It seems to me this is actually the most important part at least for as long as we need to store revoked commit history for each channel. That history can't be attached to channel messages TLV and restoring and then keep using channels without it is obviously dangerous, they better be closed ASAP. OTOH an encrypted list of peers is small and wallet user won't need to remember all peers they had channels with on wallet recovery, which is very useful. |
Don't add TLVs everywhere, simply use a separate message? That increases the limit (to its natural amount), as well. |
Nodes can offer to altruistically store small amounts of data on behalf of their channel peers. It complements
option_data_loss_protect
and can let nodes that lost data fully recover their channels without any on-chain transaction.The contents of the
peer_backup
is left unspecified to offer wide flexibility for implementations. It can for example contain an encrypted version of your internal channel state, or a mix of important data from several channels, it's really up to the implementations to go #reckless.Please note that a drawback is that this uses a lot of space in the tlv stream of existing messages, which means there won't be a lot of additional space available if we want to enrich these messages with new tlv fields. We should choose the maximum size of backups carefully if we don't want to paint ourselves into a corner.