[Proposal] Merge venus-messager to go-messenger

[Fatman13]

As proposed by @steven004 in this comment. venus-messager could potentially be an alternative solution to various message related problems mentioned in #5869. In order to move this initiative forward, we may need to map out existing features venus-messager already offers, future roadmap, how venus-messager could be interacting with Lotus and etc in this thread before making a proposal to Lotus team. Hope to get everyone's valuable input on this.

Context

Due to volatilities in gas, sending/managing messages became problematic for miners/operators. As summarized by @raulk, current Lotus message management falls short on the following...

(a) The base fee is a dynamically changing value and it directly determines whether pending messages will be included or not.
(b) The max gas fee cap is a statically configured value in Lotus configuration, and by default Lotus ships with a value that might be considered too low in a large number of cases.
(c) Messages from a given sender are ordered by nonce. A failure to include pending message 1 will prevent pending messages 2, 3, etc. from being included. This is the head-of-line blocking problem.

On the one hand, Lotus team is tight in resources to work on message optimizations. On the other hand, miners like @f8-ptrk shows concerns that changing existing logic might break current status quo and turn out to be unfavorable. Given the above, venus-messager can be the perfect ad-hoc solution which could be deployed side by side with Lotus while giving the flexibility of allowing miners/operators reverting back to Lotus' way of managing messages any time they want.

Related Issues

• [EPIC] Local Mpool Management: #5333
• Set message fees interactively: #5422
• mpool: messages stuck if have incorrect/stalled fee configuration: #5192
• Log message execution error: #5423
• Mpool monitoring tool for checking message selection: #5424
• GasPrice and GasLimit have global policies/configs: #2759

Compiled by @raulk

Solution

messager tries its best to mitigate problem (a) and (c) mentioned in the above by delaying gas estimates and doing preliminary check before pushing messages to mpool. Instead of getting gas estimates at the time of constructing the message, messager get the estimates of a small batch of messages which are ready to be pushed to mpool. Thus, increase the accuracy of the estimates and reduce failure rate. For example, when a miner tries to push 1000 messages, messager would choose, say, 10 of them and have them go through a round of preliminary check. If there are any errors, the erroneous message will have to wait for the next round of "message packing". And only then the 10 messages will get the gas estimates and being pushed to mpool.

Comments on #5869

Introduce a node health check API to detect when the Lotus node is not in a healthy state and prevent message sending.
#5762

messager can compensate for some degree of off sync with the chain.

Introduce an mpool-level healthcheck API to evaluate inclusion-ability of a give message, and to detect when pending messages from a given sender are stuck, and therefore new messages would be affected by head-of-line blocking.
#5763

On the current roadmap of messager.

Detection of whether TTY is interactive or non-interactive.
If interactive, the CLI will open the interactive mode automatically.
If non-interactive, the CLI will behave like it behaves today (just submit the message).

The team don't feel as strongly about a interactive CLI for message sending. Most of the process should be automated and customized by config file to save manual efforts.

Inversion of control so that all CLI operations that send messages (e.g. send, msig, deal making, etc.) call back to the UI for fee adjustment.
#5765

Iterative message construction can be achieved by using messager. Interaction through web UI is in the roadmap.

Enabling external signing (aka Bring Your Own Signature).
TBD if this is being solved here too.
#5122

Done!

References

https://github.com/ipfs-force-community/venus-messager/issues/40
#5869

Work

• Map out existing venus-messager features
• Document on how venus-messager is addressing some of the current issues
• Roadmap on addressing remaining issues
• How Lotus can interact with venus-messager
• Performance test
• Merge venus-messager to go-messenger

[Fatman13]

Hello, @raulk, @Kubuxu and @steven004 , please take a look at this proposal and look forward to your comments on it. Thank you!

[f8-ptrk]

it would be much easier to implement 2 things:

• config flags to accept a certain baseFee for messages (in addition or alternative to the max gas cost flag)
• a way to change the values via API calls while the daemon/miner is running

[jennijuju]

@Fatman13 thanks for creating the issue - but this does look like something that should live in one of the Venus's repo. I dont see any action item in lotus yet or its already captured in #5869 (comment).

@raulk @Kubuxu I will leave you two to decide if we should keep this ticket here or just use #5869.

[rjan90]

It seems like this proposal is now living in a Venus-repo: https://github.com/filecoin-project/venus-messager/issues/40. So this can probably be closed now? #rengjøring