refactor(x/auth/middleware)!: tx middleware to support pluggable feemarket module

[yihuang]

Closes: #11415

Description

• Create FeeMarket interface, and move exiting static fee logic into StaticFeeMarket implementation.
• Merged MempoolFeeMiddleware and TxPriorityMiddleware into DeductFeeMiddleware, so we can deduct fee based on the check result.
• Support extension options in Tx.AuthInfo, so feemarket module can extend the tx fields. Keep in TxBody
• No Ledger support in v0.46

---

Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

I have...

• included the correct type prefix in the PR title
• added ! to the type prefix if API or client breaking change
• targeted the correct branch (see PR Targeting)
• provided a link to the relevant issue or specification
• followed the guidelines for building modules
• included the necessary unit and integration tests
• added a changelog entry to CHANGELOG.md
• included comments for documenting Go code
• updated the relevant documentation or specification
• reviewed "Files changed" and left comments if necessary
• confirmed all CI checks have passed

Reviewers Checklist

All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.

I have...

• confirmed the correct type prefix in the PR title
• confirmed ! in the type prefix if API or client breaking change
• confirmed all author checklist items have been addressed
• reviewed state machine logic
• reviewed API design and naming
• reviewed documentation is accurate
• reviewed tests and test coverage
• manually tested (if applicable)

[amaury1093]

blocking for now, would prefer to include this in 0.47, to be able to have ledger support

[codecov]

Codecov Report

Merging #11413 (a74ca39) into master (3cf11e1) will decrease coverage by 0.09%.
The diff coverage is 70.94%.

❗ Current head a74ca39 differs from pull request most recent head 3b76647. Consider uploading reports for the commit 3b76647 to get more accurate results

@@            Coverage Diff             @@
##           master   #11413      +/-   ##
==========================================
- Coverage   65.96%   65.87%   -0.10%     
==========================================
  Files         675      678       +3     
  Lines       69832    69843      +11     
==========================================
- Hits        46065    46009      -56     
- Misses      21073    21130      +57     
- Partials     2694     2704      +10     

Impacted Files	Coverage Δ
types/tx/types.go	0.00% <0.00%> (ø)
x/auth/middleware/auth_ext.go	38.23% <38.23%> (ø)
x/auth/middleware/fee.go	75.55% <80.64%> (-4.45%)	⬇️
x/auth/middleware/validator_tx_fee.go	87.17% <87.17%> (ø)
x/auth/middleware/middleware.go	78.57% <100.00%> (+1.29%)	⬆️
x/auth/tx/legacy_amino_json.go	88.23% <100.00%> (+1.00%)	⬆️
x/group/codec.go	44.68% <0.00%> (-55.32%)	⬇️
x/group/module/abci.go	33.33% <0.00%> (-16.67%)	⬇️
x/group/proposal.go	28.57% <0.00%> (-14.29%)	⬇️
x/bank/keeper/grpc_query.go	47.88% <0.00%> (-5.06%)	⬇️
... and 26 more

[robert-zaremba]

If we agree to put extensions in Fee then I would advocate to include this in 0.46 for the following reasons:

• update the protocol and make a signal for tools to research a support for this
• all fee mechanisms we are discussing for 1+ year already will require this field.
• updating a structure now will give more time to ecosystem to adopt it. The next update with all the work planned will probably take more than 6 months from now to release (I assume that it will take 1month to finalize the 0.46 release).

[yihuang]

So according to my understanding, we have here choices here:

1. Use the extension_options in TxBody
   • which will be rejected in legacy amino signing mode if not empty.
2. Add extension_options in Fee
   • marshaled to ad-hoc JSON string in legacy amino encoding, could be supported in the legacy amino mode?
3. Add extension_options in AuthInfo
   • also reject it in legacy amino signing mode if not empty.

So the consequences of 1 and 3 are the same if we add the reject condition in signModeLegacyAminoJSONHandler.
If legacy amino support is not a big deal, then option 3 looks good?

[amaury1093]

Implementation LGTM, added a couple of few nits here and there

[robert-zaremba]

Let's go back to analysis and discuss pros/cons of including adding new extensions to AuthInfo or Fee.

Tier selection is related to the tx priority and tx fee level (higher tires have higher fees - but this is not necessary a case).
So if we want to be very pedantic, then @aaronc suggestion of NOT including it in Fee makes sense.

Based on the AuthInfo doc, following the pedantic approach, then the tier / priority selection should also not be a part of the AuthInfo:

// AuthInfo describes the fee and signer modes that are used to sign a
// transaction.

So we go back to re-use the TxBody.extension_options.

Supporting Leger seams to be a very valid use case, specially that some ledger transactions will like to get executed immediately. What's the chance that a transaction won't be processed in next 2-3 blocks in a busy chains like Terra or Crypto.com?

Can we add TxBody.extension_options to StdSignDoc?

[amaury1093]

Adding extensions to Authinfo has all the pros, and @yihuang's solution 3 in #11413 (comment) makes sure there's no malleability issue.

The only con is Ledger support. We can add ledger support for extension_options in both TxBody and AuthInfo, but it'll take some time. The current solution can be merged for v0.46, we just make it clear amino signing is not supported. And ledger support will come in 0.47

[tac0turtle]

Not blocking but we should not sneak in such a large breaking change right before the release of 0.46-beta without a deprecation path for the old middleware

now I see in the comment that the old behaviour is in the staticfeemarket but there is not changelog entry for this. Why do we need to break the api here?

[yihuang]

@robert-zaremba and @yihuang What do you think of merging this PR today?

I still didn't get answer from Zondax team. I pinged them again, but also if no response from them, I'll try to understand that code, and do some testing on Ledger myself. We can add ExtensionOptions to StdSignDoc in another PR.

Sounds good to me.

[amaury1093]

Got a response from ledger that unknown fields are simply ignored by the ledger app. IMO, this means that we cannot add TxBodyExtensions to StdSignDoc, or else there will be malleability issues. So ledger support will come in v0.47.

We can merge this PR as-is.

[robert-zaremba]

Since this is an important decision for transaction processing, I would like to wait for a final ack from @aaronc . He will be back on Monday.

[aaronc]

Just a few small things I would like to see addressed. It would be nice to make the language about "fee markets" a little clearer and either come up with a different term or explain it better. Generally this looks like a good approach 👍

[yihuang]

Added one more thing: Unpack the Anys in extension options, I think it's necessary for user to add fields into the extension options?

how it's gonna be used: https://github.com/yihuang/cosmos-sdk/blob/a61283b5aefaf1ce30177666442ecf4ec6311aff/x/tieredfee/types/codec.go.

[robert-zaremba]

Thanks for the update

[amaury1093]

LGTM. Let's put automerge as soon as the test is fixed