Epic: Solving message sending/management painpoints

[raulk]

This meta-issue collects reports related to message management footguns, as well as the work taking place to resolve those issues.

Context

Most of these issues have their origin in a few simple points:

1. The base fee is a dynamically changing value and it directly determines whether pending messages will be included or not.
2. The max gas fee cap is a statically configured value in Lotus configuration, and by default Lotus ships with a value that might be considered too low in a large number of cases.
3. Messages from a given sender are ordered by nonce. A failure to include pending message 1 will prevent pending messages 2, 3, etc. from being included. This is the head-of-line blocking problem.

Related reports, issues, and feature requests

• [EPIC] Local Mpool Management: [EPIC] Local Mpool Management #5333
• Set message fees interactively: Set messages' fees interactively #5422
• mpool: messages stuck if have incorrect/stalled fee configuration: mpool: messages stuck if have incorrect/stalled fee configuration #5192
• Log message execution error: Log message execution error #5423
• Mpool monitoring tool for checking message selection: Mpool monitoring tool for checking message selection #5424
• GasPrice and GasLimit have global policies/configs: GasPrice and GasLimit have global policies/configs #2759

Solution

This is a summary of the solution being implemented:

1. Introduce a node health check API to detect when the Lotus node is not in a healthy state and prevent message sending.
   • NodeStatus API Endpoint for checking node health #5762
2. Introduce an mpool-level healthcheck API to evaluate inclusion-ability of a give message, and to detect when pending messages from a given sender are stuck, and therefore new messages would be affected by head-of-line blocking.
   • MpoolHealth API Endpoint for checking health of given sender #5763
3. An interactive UI for message sending that outputs the check results (including errors), and allows the user to adjust the base fee dynamically, getting real-time feedback on message inclusion probability.
4. Detection of whether TTY is interactive or non-interactive.
   • If interactive, the CLI will open the interactive mode automatically.
   • If non-interactive, the CLI will behave like it behaves today (just submit the message).
5. Inversion of control so that all CLI operations that send messages (e.g. send, msig, deal making, etc.) call back to the UI for fee adjustment.
   • Invert control of message sending #5765
6. Interactive message management UI to visualise all pending messages from a sender, getting healthcheck status, and opening the interactive message sending UI to enable replace-by-fee.
7. CLI flag to override the max fee cap in non-interactive mode.
   • issue pending
8. Enabling external signing (aka Bring Your Own Signature).
   • TBD if this is being solved here too.
   • Enable messages to be constructed without being signed/published #5122

Technical notes

https://gist.github.com/raulk/368429797072295f339e0dbe67168098

NOTE: given that this project introduces JSON-RPC API changes (such that message-sending methods now return a message prototype instead of blindly sending the message -- this is to enable inversion of control), it depends on Lotus API versioning.

Work

• dependency: API struct codegen API proxy struct codegen #5854
• NodeStatus API Endpoint for checking node health NodeStatus API Endpoint for checking node health #5762
• MpoolHealth API Endpoint for checking health of given sender MpoolHealth API Endpoint for checking health of given sender #5763
• dependency: clean design for versioned JSON-RPC API layer in Lotus
• v1 API: V1 API staging branch #5685
• MpoolCheckReplaceMessages API: MpoolCheckReplaceMessages API #5863
• Staging branch: Robust message management  #5822

[steven004]

@raulk In Venus' implementation, we're having a separated mpool (or Messenger we called) service to support the features you listed.
And it is in progress, please see more in https://github.com/ipfs-force-community/venus-messager

I would think it would be best to have this service to be independent, and be able to support whatever implementations a miner is using, which just requires the well defined API. If you think it is fine, we can move Venus-Messager to filecoin-project and rename it to go-messenger, thus we do not need to redundant work.

[raulk]

@steven004 looks interesting. I briefly peeked at the code. Would you mind adding a few notes to the README on current functionality, and what's coming? Then we can coordinate. Is this a re-implementation of the lotus mpool?

[steven004]

A draft of readme is added: https://github.com/ipfs-force-community/venus-messager/blob/main/README.md, which will be polished later.

[raulk]

Looping in @Kubuxu to evaluate what are the best next steps re: go-messenger / venus-messenger and s shared component.

[Fatman13]

Hello, @raulk, venus is preparing a proposal regarding @steven004 's suggestion. Please feel free to chime in. Thank you! https://github.com/ipfs-force-community/venus-messager/issues/40