crypto/ed25519: Implement Ed25519ph

[titanous]

Update, Apr 6 2022: The proposed API is in #31804 (comment).

---

The Ed25519ph variant specified in RFC 8032 allows signing/verifying a message that has already been hashed with SHA-512 without risking the collision-resistant properties of "PureEdDSA" when using the same keys for messages signed using both schemes.

This is useful in at least two scenarios:

1. When the private key is isolated to another piece of hardware and passing the entire message to be signed is not possible, for example when using a HSM and signing messages larger than a few KB.
2. When working with large messages that are too large to be reasonably buffered for the current one-shot API.

This variant can be implemented minimally using the existing crypto.Signer API plus an additional verification function, without encouraging unsafe use by providing easy access to an API that takes an io.Reader or io.Writer.

Due to the additional internal hash initialization, there is no way to implement this without forking the package or upstreaming an implementation patch.

I will send a CL with a proposed implementation.

Relevant: #31727

/cc @zx2c4 @FiloSottile

[gopherbot]

Change https://golang.org/cl/174941 mentions this issue: ed25519: Implement Ed25519ph

[x30n]

+1

[Hades32]

Thanks @titanous , this is just what we needed. Confirmed to behave as the reference implementation (libsodium). 👍

Not sure if this is still in time for 1.13... @FiloSottile

[FiloSottile]

Too late for Go 1.13, targeting Go 1.14. (crypto/ed25519 is now in the standard library.)

[Yawning]

While I understand that the crypto.Signer interface is fixed in stone and can't be changed, if this is going to happen, it would be nice if it supported the full Ed25519ph algorithm as described in the RFC.

As it stands right now, the proposed implementation does not support a domain separation context.

Since there already is a Sign method in the package, and the PR adds VerifyHashed, this could be done by adding SignHashed(privateKey PrivateKey, context, message []byte) []byte and changing the proposed VerifyHashed to take another byte slice.

[FiloSottile]

RFC 8032 defined context independently of pre-hashing, so to support the whole spec we'd also have to support pure Ed25519 with custom context.

I am really not a fan of extending the API surface of a standard library package that should be what we point users to for basic public key signatures. How about this alternative API, which is more extensible and still makes it very opt-in to select the variants?

// Options can be used with PrivateKey.Sign or VerifyWithOptions
// to select Ed25519 variants.
type Options struct {
    // Hash can be zero for regular Ed25519, or crypto.SHA512 for Ed25519ph.
    Hash    crypto.Hash
    Context string
}

func (*Options) HashFunc() crypto.Hash

func VerifyWithOptions(publicKey PublicKey, message, sig []byte, opts *Options) bool

[Yawning]

If this proposed API is accepted, then VerifyHashed as proposed in the PR will go away right? I would be in favor of this, since it seems like a cleaner way to support the functionality.

Nitpicking: Is there any particular reason why Context is a string over []byte? I understand they are fairly interchangeable (and string may be more const friendly). I personally would make it a []byte to make it clear that it is an arbitrary "octet string of at most 255 octets" (and include the size limit in a doc string comment).

[FiloSottile]

If this proposed API is accepted, then VerifyHashed as proposed in the PR will go away right? I would be in favor of this, since it seems like a cleaner way to support the functionality.

Yep.

Nitpicking: Is there any particular reason why Context is a string over []byte? I understand they are fairly interchangeable (and string may be more const friendly). I personally would make it a []byte to make it clear that it is an arbitrary "octet string of at most 255 octets" (and include the size limit in a doc string comment).

Mostly consistency, we've used string for contexts elsewhere, in some cases to make it a different type from the message itself (which is not relevant here). I personally think it fits better the nature of the value, because it's usually fixed or at least immutable, often human-readable, and as you say can be a const.

We should definitely document the max length, thank you.

[Yawning]

I went and implemented this in a package I maintain for dayjob (because dayjob needs ph-with-context support), and have more feedback.

What should happen when Hash is crypto.Hash(0) and Context is ""?

1. Ed25519ctx with a 0 octet context ("The context input SHOULD NOT be empty.").
2. Ed25519pure

I went with option 2 as option 1 is somewhat nonsensical and recommended against, though I will happlily change the package to match what the runtime library does. If Context were a byte, this could be disambiguated by nil vs []byte{}, but it's not clear to me if that justifies the loss of consistency and const friendliness, just for the sake of completeness.

Minor: I used opts crypto.SignerOpts for VerifyWithOptions so that it is possible to pass crypto.SHA512 when the context is not required (following PublicKey.Sign). The type naming is somewhat unfortunate, but the ease of use for what I suspect is a common case probably wins out.

[FiloSottile]

What should happen when Hash is crypto.Hash(0) and Context is ""?

1. Ed25519ctx with a 0 octet context ("The context input SHOULD NOT be empty.").
2. Ed25519pure

I went with option 2 as option 1 is somewhat nonsensical and recommended against, though I will happlily change the package to match what the runtime library does. If Context were a byte, this could be disambiguated by nil vs []byte{}, but it's not clear to me if that justifies the loss of consistency and const friendliness, just for the sake of completeness.

Yeah, definitely pure if Context is "", the alternative is confusing and not something people should do anyway, but we should document it. I am also not a fan of semantic differences between nil and []byte{} in general.

Minor: I used opts crypto.SignerOpts for VerifyWithOptions so that it is possible to pass crypto.SHA512 when the context is not required (following PublicKey.Sign). The type naming is somewhat unfortunate, but the ease of use for what I suspect is a common case probably wins out.

I thought about that, but I think I'd rather use the concrete type for a few reasons:

• you can't actually semantically use any other crypto.SignerOpts here, the reason Sign has it is to match an interface which needs the abstraction, VerifyWithOptions has no semantic justification
• Ed25519ph is not something we need to encourage and make easy, so having to explicitly use Options seems fine
• interface boxing still causes heap escapes, I think

[odeke-em]

Thank you for mailing CL 174941 @titanous, unfortunately that didn't make it into the cut before the Go1.14 freeze, but please rebase from master and we'll hopefully get this in for Go1.15. Apologies for lack of eyes on it during Go1.14.

[armfazh]

I think this is wrong,

If opts.HashFunc() is crypto.SHA512, the pre-hashed variant Ed25519ph
is used and message is expected to be a SHA-512 hash,

the prehashed mode must do the job internally, i.e. explicitly hashing the (likely to be large) message with SHA-512.
Otherwise, there is no guarantee that the hashed message was generated by SHA-512, it could be generated with another hash function that also outputs the same number of bytes.

[titanous]

the prehashed mode must do the job internally, i.e. explicitly hashing the (likely to be large) message with SHA-512.
Otherwise, there is no guarantee that the hashed message was generated by SHA-512, it could be generated with another hash function that also outputs the same number of bytes.

That is not how the crypto.Signer API works. You are expected to hash the message before calling Sign if a hash function option is specified. For example, look at how the ecdsa package does it.

[armfazh]

That is not how the crypto.Signer API works.
You are expected to hash the message before calling Sign if a hash function option is specified. For example, look at how the ecdsa package does it.

Of couse doesn't work like that because crypto.Signer doesn't take into account the use of prehashed signature schemes. Ed25519Ph is one of them, as it prehashes the message internally using always SHA-512.

I acknowledge the lack of a Signer Go interface that handles the new signature schemes for example those in RFC-8032, which enable prehashing and receive domain separation strings as input.

[rsc]

Based on the discussion above, this proposal seems like a likely accept.
— rsc for the proposal review group

[rsc]

No change in consensus, so accepted. 🎉
This issue now tracks the work of implementing the proposal.
— rsc for the proposal review group

[ianlancetaylor]

CC @golang/security

Looks like this didn't make 1.19. Moving to backlog. Please recategorize as appropriate.