Issues recovering 24-word launchpad mnemonics

[michaelsproul]

Description

Our recovery from 24-word mnemonics seems to be broken. Take this mnemonic that I derived from the eth2.0-deposit-cli just now:

spider hand master liar awkward silver name upper lucky page later lake hold inhale venue occur match carry fortune mansion accident reflect head waste

The deposit CLI says that the 0th voting key for this mnemonic is 0x9671e8c1ccc469fd63d81d0eefef185374f5447da479e5327a38e8cb9e4d61bab78449d6e8b0fa3a9f20c45bba770da1, but lighthouse account validator recover yields 0x95327ade0bfab496c26f0af24c49a9c45dfdb0c133dee2d49712d6a369ecc814cb15d95df61e93dd16d33fd016f7b127. The path seems to be correct -- m/12381/3600/0/0/0 in both cases. We also derive the same key by recovering the wallet and then creating a validator.

Thanks to @FeelsGoodMan on Discord for raising this.

Version

Lighthouse v0.2.9-e5fc6bab4
BLS Library: blst

[michaelsproul]

Tested with Milagro just to be sure it isn't a BLS issue, and the same key is derived.

[realbigsean]

I checked out the wordlists each are using and they are identical.

https://github.com/ethereum/eth2.0-deposit-cli/blob/master/eth2deposit/key_handling/key_derivation/word_lists/english.txt

https://github.com/maciejhirsz/tiny-bip39/blob/master/src/langs/english.txt

[thomas-pegot]

Lighthouse uses scrypt as kdf whereas Prysm uses pbkdf2.
eth2.0-deposit-cli for the chain medalla might have bpkdf2 encryption in settings.

[realbigsean]

There were some changes to the EIP-2333 spec made a couple months ago that don't seem to be implemented in Lighthouse. I think this change specifically is causing the difference:

I hard-coded this test vector from the eth2.0-deposit-cli and it went from failing to passing after I made those updates ^

#[test]
fn test_seed_to_parent_secret(){
    let test_vector_seed = "c55257c360c07c72029aebc1b53c05ed0362ada38ead3e3e9efa3708e53495531f09a6987599d18264c1e1c92f2cf141630c7a3c4ab7c81b2f001698e7463b04";
    let test_vector_parent_sk = "5399117110774477986698372024995405256382522670366369834617409486544348441851";

    let master = DerivedKey::from_seed(hex::decode(test_vector_seed).unwrap().as_slice()).map_err(|()| Error::EmptyPassword).unwrap();
    let result = BigInt::from_bytes_be(Sign::Plus, master.secret()).to_string();

    assert_eq!(test_vector_parent_sk, result);
}

[paulhauner]

Lighthouse uses scrypt as kdf whereas Prysm uses pbkdf2.
eth2.0-deposit-cli for the chain medalla might have bpkdf2 encryption in settings.

As per EIP-2335 we support both of these formats, so I don't think this is the issue. Thanks, though :)

[thomas-pegot]

@paulhauner Sorry what I meant Lighthouse json output where scrypt kdf while prysm was pbkdf2 kdf for the same mnemo. Anyway even salt, checksum, and everything were different.

[realbigsean]

My comment is a little out of date because EIP-2333 was updated again yesterday. This the the PR to implement the newest changes in the eth2-deposit-cli: ethereum/staking-deposit-cli#108

[paulhauner]

Resolved in #1633

[michaelsproul]

I just opened #1665 to track generating 24-word mnemonics by default