Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove bech32 PubKey support #7447

Closed
5 of 9 tasks
aaronc opened this issue Oct 2, 2020 · 27 comments · Fixed by #7597 or #7477
Closed
5 of 9 tasks

Remove bech32 PubKey support #7447

aaronc opened this issue Oct 2, 2020 · 27 comments · Fixed by #7597 or #7477
Assignees
Labels
Milestone

Comments

@aaronc
Copy link
Member

aaronc commented Oct 2, 2020

Summary

Let's remove support for bech32 pubkeys.

Problem Definition

Bech32 PubKey's depend on amino encoding and as I understand it were never a good idea in the first place. Can you say more on this @alexanderbez ?

Proposal

We can just use google.protobuf.Any for representing PubKeys where needed (for instance in x/staking and x/slashing).

Work

Followup tasks


For Admin Use

  • Not duplicate issue
  • Appropriate labels applied
  • Appropriate contributors tagged
  • Contributor assigned/self-assigned
@aaronc aaronc added this to the v0.40.1 milestone Oct 2, 2020
@sunnya97
Copy link
Member

sunnya97 commented Oct 4, 2020

Why do we want to remove bech32 pubkeys? Bech32 is independent of amino encoding. It's just about how to encode a pubkey that's represented in bytes, regardless of amino or not.

Also, I'm okay with removing amino encoding of pubkeys, but it must be noted that one of its benefits is that the amino encoding (and thus its bech32 representation as well) implicitly include what type of signature algorithm it is. Any replacement should try to keep this property.

@alexanderbez
Copy link
Contributor

alexanderbez commented Oct 5, 2020

Public keys are not meant to be bech32 encoded. @sunnya97 amino is directly tied to the bech32 encoding of public keys. How is it independent if the bytes come from Amino? Not to mention multi-sigs aren't even supported and will panic when you try to decode because of length restrictions. I have zero clue why this was done initially @aaronc.

@sunnya97
Copy link
Member

sunnya97 commented Oct 5, 2020

Bech32 can be used to encode any series of bytes. I'm saying that we can still use bech32 for human-readable pubkeys, even if its not amino pubkeys that are being bech32-ed.

@robert-zaremba
Copy link
Collaborator

Bech32 can be used to encode any series of bytes. I'm saying that we can still use bech32 for human-readable pubkeys, even if its not amino pubkeys that are being bech32-ed.

We have addresses for that. PubKeys are long (usually 32 bytes) and with Bech32 (which makes them even longer) are not meant to be human readable.

@alexanderbez
Copy link
Contributor

alexanderbez commented Oct 6, 2020

Right, but if you try to bech32-decode that blob @sunnya97, most binaries will panic because of the hard-capped 90 char limit. This is why we can't bech32 encode pubkeys in JSON output in the SDK right now, because the standard bech32 library panics when the string > 90 chars.

@tarcieri
Copy link

tarcieri commented Oct 13, 2020

Note that parsing Bech32 that's longer than the 90 character limit specified in BIP-0173 is already a part of the BOLT11 Encoding used by the Lightning Network. So there are quite a few Bech32 parsers/serializers which support exceeding this limit in order to support BOLT11.

That said, it seems like another option here is to retain Bech32 encoding for public keys, but remove the Amino prefixes so that the Bech32 encoding is less than 90 characters.

@alexanderbez
Copy link
Contributor

I don't think the amino prefixes is what makes it more than 90 chars. If you have a multisig with >2-3 keypairs, you'll get an encoding that results in more than 90 characters.

@tarcieri
Copy link

tarcieri commented Oct 13, 2020

Aah, well in that case, the BOLT11-style encoding should suffice (really it's just BIP-0173 without the 90-char limit)

@ethanfrey
Copy link
Contributor

There was a long discussion about this (which I cannot find) on the launchpad migration which hit bech32 key issues with multsigs.

There was a nice long discussion here as well: #6886 (which should be good background reading).

@iramiller made some good arguments about base58 being a much more accepted standard for public keys. Just one comment on github #6886 (comment) but I know there were some nice discord discussions. Maybe you want to add your comments here?

In general, if a spec and many libraries enforce a 90 char limit, it seems best not to abuse it. Sure, you can find a go library that doesn't enforce the limit. But what about JS clients, or Rust clients, or Dart clients? Everyone has to go through the same issues the first time they encounter a multisig address, which is not in the general happy path testing. Let's just use an encoding compatible with all major libraries.

@iramiller
Copy link
Contributor

The base 58 encoding (and base58check implementation) is a useful option for public key encoding for all of the reasons highlighted in the bitcoin source. The encoding was specifically created for the purpose of encoding public keys which is a large part of why I offered it up originally as a solution.

https://github.com/bitcoin/bitcoin/blob/master/src/base58.h#L7-L12

@iramiller
Copy link
Contributor

@ethanfrey mentioned a Discord discussion above. Below are the relevant quotes from my conversation with Ethan I believe:

Ira [07/24/2020]: why not standardize on public key encoding using base58 like BIP32 specifies?

Ira [07/24/2020]: xpub6DpKcTRrVcqYS6rXyicDg52FveMhMLr3SSbBA5iFhZPLdVdAMnC597Y9HmV157MqRJJgnDA3tAvX7SjrKmgatuLUctnwNz4rnQYTSLoW9hy
for example ... it has more space that base 32, is well known and supported... and even supports the concept of HRP

has more space that base 32 should read: is more space efficient than base 32

Ira [07/24/2020]: bech32 is useful for addresses per spec ... the base58 approach was specified explictily for encoding those HD wallet extended keys ... would be well understood by the industry outside of cosmos and likely has a fair amount of documentation that could be quickly pulled into standardize this

And the mentioned specification is here:
https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki#Serialization_format

@alexanderbez
Copy link
Contributor

ACK on Base58 👍

@amaury1093
Copy link
Contributor

amaury1093 commented Oct 20, 2020

In #7597 we're working to use protobuf Any for pubkeys throughout the SDK, which should remove the usage of custom pubkey encoding altogether. I am having trouble seeing where the Base58 format would come in, once that PR is merged.

@aaronc
Copy link
Member Author

aaronc commented Oct 20, 2020

PubKey encoding could be reconsidered post Stargate before we settle on v1 proto files but for now our plan is to stick with the decision to use Any.

One thing that we would need to resolve if we use something like base58 is how we define PubKey prefixes. Is there a registry of key prefixes that we maintain and a standard way to prepend the prefix before converting to base58? Using Any while definitely not as space efficient (because of the type URL) provides us with a convenient way to do this that is consistent with the rest of the SDK's usage of Any.

@robert-zaremba
Copy link
Collaborator

I'm also for sticking to protobuf / Any. All clients have to support protobuf anyway. Adding more encoding options for few structures won't help. It will only makes things more complex / twisted.

@ethanfrey
Copy link
Contributor

Why do we want a prefix for the pubkey encoding?

If we just use any and encode them as normal byte field - base64 bytes in json/toml - that seems fine with me.

If we add some custom encoding, I think something like xpub (which is base58 with a prefix and checksum) will give us a prefix

@robert-zaremba
Copy link
Collaborator

Why do we want a prefix for the pubkey encoding?

With Any we will go with the protobuf any encoding without custom prefix. Note: Any uses a type url.

@amaury1093 amaury1093 reopened this Oct 23, 2020
@amaury1093
Copy link
Contributor

Re-opening. Let's wait on #7477 before closing this (as per original post).

@robert-zaremba
Copy link
Collaborator

Yes, it shouldn't be closed. The tracking PRs are listed in the OP. (Work section)

@aaronc aaronc added the backlog label Oct 26, 2020
@aaronc
Copy link
Member Author

aaronc commented Oct 26, 2020

Just so everyone is on the same page here. Our team is working on removing bech32 pubkey support and replacing it with Any which we are using for public keys elsewhere.

If there is desire for a different format in the future, can we address that post-stargate? Currently we are not considering some other format like base58.

@clevinson
Copy link
Contributor

clevinson commented Nov 16, 2020

Just flagging that we still have bech32 pubkeys expected on the slashing queries for validator SigningInfo and SigningInfos (see here). Proto defs here.

@robert-zaremba is this already on your radar?

I see this as higher priority than keyring, as it's client facing.

@amaury1093
Copy link
Contributor

Aren't these addresses? afaiu we're keeping them as bech32.

@robert-zaremba
Copy link
Collaborator

robert-zaremba commented Nov 16, 2020

gRPC queries don't use bech32 pubkey.

REST pubkey queries will stay in v0.40 and are marked deprecated. We are planning to remove them in v0.41

@clevinson
Copy link
Contributor

Yep these are addresses. My bad!

@robert-zaremba
Copy link
Collaborator

While working on this task I encounter few problems from the DevX perspective. In #7477 I added few helper functions and tests on the PubKey serialization. For other things, I've updated the description of this task with list of followup tasks.
They will be blocked until #7477 get merged (after Stargate release?).

@robert-zaremba
Copy link
Collaborator

Question:
I found that we also use bech32 pubkey prefixes in config. If we delete it it may break some client code who is still relaying on this serialization format. For the moment I added a TODO for that in the legacybech32 package. WDYT?

@clevinson
Copy link
Contributor

Once ADR028 and secp256r1 work is merged, this may get looked at again by @robert-zaremba, but we're considering it a nice-to-have for v0.42

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.