HKDF instantiations

[schanzen]

Hello,

according to the (new) documentation for HKDF "HKDF-SHA512 is present for consistency and compatibility with existing protocols, but it doesn’t have practical security benefits over HKDF-SHA256."
I would like to discuss this in the context of https://datatracker.ietf.org/doc/html/rfc5869 and https://eprint.iacr.org/2010/264.pdf especially since the former basically outsources all security considerations to the latter.

In particular, in the paper Section 6 and Appendix D (Point 4.) the "Truncated NMAC" construction is discussed which gives a statistical extractor (as opposed to a computational extractor).
The gist is that you use HKDF-Extract with SHA512 and truncate the output PRK to 32 bytes.
Then HKDF-Expand is used with SHA256 and the truncated PRK.
We use this construction in GNUnet: https://git.gnunet.org/gnunet.git/tree/src/lib/util/crypto_kdf.c#n46.
Rather: We thought we did, but we actually have an error in the implementation and use the 64 byte PRK directly as key for the HMAC in the Expand phase :/. This erroneous implementation cannot even be done with the new HKDF API of libsodium, only by using the HMAC API directly and manually implement HKDF-Expand.

Now, the fact that RFC5869 does not define any SHA512 test vectors or test vectors that mix hash functions has always been itching me. But since libsodium does implement HKDF with SHA512, the "Truncated NMAC" instantiation is an actual (possibly more secure in a theoretical sense) alternative to either SHA256 or SHA512 HKDF. And it can be done by simply mixing the SHA512 Extract API with the SHA256 Expand API in libsodium.

I would like to know what the community thinks about this as I am not sure about the viability myself.
But maybe it makes sense to document this possibility and not basically discard the use of SHA512 in the documentation outright and simply keeping it for "compatibility" reasons?

Best

[jedisct1]

No one is using NMAC, and you're going to struggle to find an implementation. It doesn't use standard hash functions, but requires changing their IV, so this is not something that can be implemented on top of existing cryptographic libraries either.

That paper was written in 2010. Since then, not only the security of HMAC has been better understood, but we are also confident that relaxing the statistical closeness property is fine.

crypto_hkdf as a whole is only there for compatibility reasons. Libsodium has crypto_kdf_derive_from_key() as a high level function for key derivation.

[schanzen]

NMAC is the scheme underlying HMAC, and in the paper the instantiation outlined above is just called "Truncated NMAC" but of course applied to HKDF (or rather, its use of HMAC). See "NMAC as extractor" on page 16.

Do you have any specific reference regarding this better understanding of HMAC (in combination with HKDF)? I am sure that any instantiation in scope of this discussion is secure in practice, but I would like to know on what basis we choose.
In my mind the only reasons why you would want to use SHA256-only or SHA512-only instantiations is if for some reason one or the other is faster or you want to design a protocol that does not require implementers to implement/use two different hash functions.

I also understand that 2010 is a long time ago, but crypto_kdf_derive_from_key itself is only useful for protocol design when the underlying primitive is standardized or sufficiently specified and widely available.
Of course that is true for Blake2b at least in terms of specification, but HMAC and/or HKDF are much more widely available.
What was the motivation for implementing HKDF at all if only for compatibility? HKDF is implemented with a couple of lines around HMAC, after all.

[jedisct1]

https://crypto.stackexchange.com/ may be a better place for discussions that are not directly related to this project, that you don't even seem to be using.

[schanzen]

Sure we can stop this here. But there seems to be a misunderstanding. We do use libsodium and we are migrating our code from gcrypt to libsodium where possible. And for HKDF we wanted to use the new API, but we couldn't. That is/was not the fault of libsodium, but it made me think about our use of HKDF and general considerations on how to use it.
Eventually, we implemented HKDF using the libsodium hmac API anyway as for example Debian stable does not ship 1.0.19, yet.

[schanzen]

In particular, the implementation of HKDF in libsodium is quite opinionated. For example, HKDF-Expand (and HMAC for that matter) support the use of keys with length >= 64 bytes which the current libsodium API does not (even though the HMAC API itself does).

[bellebaum]

I apologize if I get some concepts of libsodium wrong, as I have seldomly used this library, but I believe there is a misunderstanding about what a "Key Derivation Function" is.

For libsodium, KDF seems to be a pseudorandom function. I.e. it takes as input a uniformly random and secret key, as well as further inputs, to produce streams of random looking bits. In HKDF terms, this maps to the functionality of HKDF-Expand.
The API is designed to always derive random keys before handing them to the user. E.g. the equivalent of HKDF-Extract, a random oracle or randomness extractor, is built into e.g. the key exchange functions directly (and happens to be a second BLAKE2b, going via the documentation, which is fine for ECDH).

For gnunet, a KDF is something more abstract which takes as input a value of some computational entropy or "some shared secret", which need not be a secure symmetric key, and outputs random looking strings of bits. It is used in several places for many different purposes and defining any hash function to serve all purposes likely requires modelling the KDF as a random oracle, which is why HKDF seems appealing, especially since parts of gnunet aim to be published as RFCs.

Hope this helps to bring the discussion forward