(ADP-257) Add Shelley migration endpoints and Byron (haskell impl) support of migrateWallet

[paweljakubas]

Context

As an API Client,
I want to be able to empty a wallet and send all funds to a list of addresses,
So that I can easily move funds around and have the wallet dealing with fees for me.

Decision

We are adding migration endpoints as done in #1658 in Shelley

Acceptance Criteria

1. Shelley has migration sendAll endpoints and cost estimation endpoint for this.
2. We have Shelley migration post data unit tests.
3. Corresponding changes in swagger are done.
4. Migration tests are added to cardano-node's byron integration tests.

Comment: as transaction are not available for shelley we cannot check it at this moment in integration test mode.

---

Development

QA

1. migrateWallet was enabled for byron (haskell impl) and several integration tests were added in lib/byron.
2. The subtle bug was found in Transaction layer of byron (haskell impl) and TransactionSpec was adjusted.
3. migrateWallet was enabled also for shelley (haskell impl), but at this moment it was not impossible to test it as this lib misses transaction layer impl (although it should go very soon as Add Shelley TransactionLayer #1673 - so it would be worth testing this (maybe copy integration tests from byron lib concerning migration)

[KtorZ]

When reviewing #1658 I noticed that the current migration endpoints are actually associated with a /migrations namespace / resource. So, perhaps it makes sense to keep the separation between transactions / migrations in our API types as well? @paweljakubas

[piotr-iohk]

I have updated swagger spec -> #1687.
It would be nice to also address this -> #1682.

[piotr-iohk]

I'll leave it in QA for now because Shelley migrations are not quite testable currently without transactions.

[piotr-iohk]

I encounter a 500 response when running migrations from Byron wallet on F&F testnet (for Shelley wallets migrations seem to work fine).

Steps to reproduce:

1. Run migration on Byron wallet having funds and as a result you'll have 500 response:

POST http://localhost:8090/v2/byron-wallets/94c0af1034914f4455b7eb795ebea74392deafe9/migrations

The response was:
Code = 500,
Message = {"code":"created_invalid_transaction","message":"That's embarrassing. It looks like I've created an invalid transaction that could not be parsed by the node. Here's an error message that may help with debugging: ApplyTxError [LedgerFailure (UtxowFailure (MissingVKeyWitnessesUTXOW {pfUTXOmissingWitnesses = fromList [KeyHash 03170a2e7597b7b7e3d84c05391d139a62b157e78786d8c082f29dcf4c111314]}))]"}

In the wallet.log there is:

[cardano-wallet.api-server:Info:630] [2020-06-01 11:44:52.51 UTC] [RequestId 99] [POST] /v2/byron-wallets/94c0af1034914f4455b7eb795ebea74392deafe9/migrations
[cardano-wallet.network:Info:18] [2020-06-01 11:44:52.61 UTC] Send MsgSubmitTx Tx' {_body' = TxBody' {_inputs' = fromList [TxIn (TxId {_TxId = 1594e75578ceb053ac3bfa6d9da3fa2de4e2f89df1c269984a6fcc4b935cb8a1}) 0], _outputs' = StrictSeq {getSeq = fromList [TxOut (AddrBootstrap (Address {addrRoot = cf9aac64ca146b7812c623d40d1a6b429f92684e9c801078fc4f149d, addrAttributes = Attributes { data: AddrAttributes {aaVKDerivationPath = Just (HDAddressPayload {getHDAddressPayload = "s\212\ta\138\197\196\231J(}\146d\150\191\v6\232J\223\255\218\150\251\210@\DLE\211"}), aaNetworkMagic = NetworkTestnet 42} }, addrType = ATVerKey})) (Coin 999834719)]}, _certs' = StrictSeq {getSeq = fromList []}, _wdrls' = Wdrl {unWdrl = fromList []}, _txfee' = Coin 165281, _ttl' = SlotNo {unSlotNo = 686680}, _txUpdate' = SNothing, _mdHash' = SNothing, bodyBytes = "\164\NUL\129\130X \NAK\148\231Ux\206\176S\172;\250m\157\163\250-\228\226\248\157\241\194i\152Jo\204K\147\\\184\161\NUL\SOH\129\130XP\130\216\CANXF\131X\FS\207\154\172d\202\DC4kx\DC2\198#\212\r\SUBkB\159\146hN\156\128\DLEx\252O\DC4\157\162\SOHX\RSX\FSs\212\ta\138\197\196\231J(}\146d\150\191\v6\232J\223\255\218\150\251\210@\DLE\211\STXB\CAN*\NUL\SUB\NUL\196\250\143\SUB;\152D_\STX\SUB\NUL\STX\133\161\ETX\SUB\NUL\nzX", extraSize = 131}, _witnessVKeySet' = fromList [WitVKey' {wvkKey' = VKey (VerKeyEd25519DSIGN (PublicKey "\b\171\238\224\175\154\207\221M3S\SO\224\253\132\ETB|\210\214\251d(\DC4\200\202c\191\172`\238\206\170")), wvkSig' = SignedDSIGN (SigEd25519DSIGN (Signature "\243\241\133\254\186\150G\191\161\133\189\176\246L\ACK\DC4\167\254%T\192\241,\213b\146?\185%\231v@C\221U\143\199\185\233e\188\229\EMo\130\216\202@\184+2\241\191~\242\DEL\233\CAN\209c\f\245\152\ACK")), wvkBytes = "\130X \b\171\238\224\175\154\207\221M3S\SO\224\253\132\ETB|\210\214\251d(\DC4\200\202c\191\172`\238\206\170X@\243\241\133\254\186\150G\191\161\133\189\176\246L\ACK\DC4\167\254%T\192\241,\213b\146?\185%\231v@C\221U\143\199\185\233e\188\229\EMo\130\216\202@\184+2\241\191~\242\DEL\233\CAN\209c\f\245\152\ACK"}], _witnessMSigMap' = fromList [], _metadata' = SNothing, txWitsBytes = "\161\NUL\129\130X \b\171\238\224\175\154\207\221M3S\SO\224\253\132\ETB|\210\214\251d(\DC4\200\202c\191\172`\238\206\170X@\243\241\133\254\186\150G\191\161\133\189\176\246L\ACK\DC4\167\254%T\192\241,\213b\146?\185%\231v@C\221U\143\199\185\233e\188\229\EMo\130\216\202@\184+2\241\191~\242\DEL\233\CAN\209c\f\245\152\ACK", txFullBytes = "\131\164\NUL\129\130X \NAK\148\231Ux\206\176S\172;\250m\157\163\250-\228\226\248\157\241\194i\152Jo\204K\147\\\184\161\NUL\SOH\129\130XP\130\216\CANXF\131X\FS\207\154\172d\202\DC4kx\DC2\198#\212\r\SUBkB\159\146hN\156\128\DLEx\252O\DC4\157\162\SOHX\RSX\FSs\212\ta\138\197\196\231J(}\146d\150\191\v6\232J\223\255\218\150\251\210@\DLE\211\STXB\CAN*\NUL\SUB\NUL\196\250\143\SUB;\152D_\STX\SUB\NUL\STX\133\161\ETX\SUB\NUL\nzX\161\NUL\129\130X \b\171\238\224\175\154\207\221M3S\SO\224\253\132\ETB|\210\214\251d(\DC4\200\202c\191\172`\238\206\170X@\243\241\133\254\186\150G\191\161\133\189\176\246L\ACK\DC4\167\254%T\192\241,\213b\146?\185%\231v@C\221U\143\199\185\233e\188\229\EMo\130\216\202@\184+2\241\191~\242\DEL\233\CAN\209c\f\245\152\ACK\246"}
[cardano-wallet.network:Info:18] [2020-06-01 11:44:52.64 UTC] Recv MsgRejectTx (ApplyTxError [LedgerFailure (UtxowFailure (MissingVKeyWitnessesUTXOW {pfUTXOmissingWitnesses = fromList [KeyHash 03170a2e7597b7b7e3d84c05391d139a62b157e78786d8c082f29dcf4c111314]}))])
[cardano-wallet.api-server:Error:630] [2020-06-01 11:44:52.64 UTC] [RequestId 99] 500 Internal Server Error in 0.128202956s

Byron wallet on F&F testnet with funds to reproduce ->

purchase carbon forest frog robot actual used news broken start plunge family

[piotr-iohk]

Shelley migrations are also lacking any integration tests at this moment.

[piotr-iohk]

For the record this is still the case on cardano-node 1.14.1 -> #1675 (comment)

[KtorZ]

@piotr-iohk we can't yet do transactions from Byron addresses. We need to acknowledge for Byron witnesses in transactions, which only landed recently on cardano-node. We can keep this ticket open perhaps until we solve this.

[paweljakubas]

@piotr-iohk now this can be re-looked

[paweljakubas]

this one is in QA #1844 and related

[piotr-iohk]

lgtm, closing. Migration tests enabled in #1900.