Foundational work to get a cluster of pools for integration testing #1731

KtorZ · 2020-06-08T08:31:24Z

Issue Number

N/A

Overview

5a263bc
📍 move content of 'Test.Utils.Ports' to 'Cardano.Wallet.Network.Ports'
6ed9efc
📍 rework the shelley launcher code so that we can start a cluster of nodes
using 'withCluster ... ... 0' will result in the same behavior as before. Right now, pools would
exists as process but not on chain. In order to exist on chain, pools need to be registered, which
will be done in a separate commit.
1a8b720
📍 fix database path
6745ea9
📍 Regenerate nix
6b7b947
📍 assign sensible default ordering to 'listAllTransactions'

Comments

This should make it possible to run a cluster of pools as simply as:

        let numberOfPools = 3
        withCluster tr Info numberOfPools $ \socketPath block0 (gp,vData) -> ...

So far, the pools won't have any stake and won't therefore be any useful.. I'll give them some stake in another PR.

using 'withCluster ... ... 0' will result in the same behavior as before. Right now, pools would exists as process but not on chain. In order to exist on chain, pools need to be registered, which will be done in a separate commit.

paweljakubas · 2020-06-08T09:27:41Z

lib/shelley/src/Cardano/Wallet/Shelley/Launch.hs

+    -> IO (Either ProcessHasExited a)
+withCluster tr severity n action = do
+    ports <- randomUnusedTCPPorts (n + 1)
+    print $ permutations ports


probably we want to use our logging infrastructure someday in the future - if not doing this now maybe add TO_DO to not miss it later

What do you mean?

I mean, we have now print - and maybe use our logging here?

Also, something more descriptive than

[(32403,[])]

would be nice

Ah sorry. I was really confused by your comment at first and didn't get why. This line is actually an oversight. I removed it 👍

paweljakubas

looks like sound scaffolding for what is coming next

paweljakubas · 2020-06-08T09:30:22Z

lib/shelley/src/Cardano/Wallet/Shelley/Launch.hs

+    --
+    -- >>> permutations [1,2,3]
+    -- [(1,[2,3]), (2, [1,3]), (3, [1,2])]
+    permutations :: Eq a => [a] -> [(a, [a])]


maybe reuse https://hackage.haskell.org/package/base-4.14.0.0/docs/src/Data.OldList.html#permutations

and then fmap with f(x:xs)=[x,[xs]] ?

It won't work because it's does not actually compute all permutations (unlike Data.List.permutations). Using Data.List.permutations on the example above, we would also get [(1, [3,2]), (2, [3,1]), (3, [2,1])]. So, we should also nub + sort the second element which is arguably simpler than the current function 🤔 ..

Changed for:

rotate :: Ord a => [a] -> [(a, [a])] rotate = nub . fmap (\(x:xs) -> (x, sort xs)) . permutations

Anviking · 2020-06-08T10:10:28Z

lib/shelley/src/Cardano/Wallet/Shelley/Launch.hs

+
+        forM_ (init $ permutations ports) $ \(port, peers) -> do
+            withAsync
+                (withStakePool tr severity (port, peers) $ readChan doneGroup)


Why not put the n-th withStakePool in the n-1-th's action?

Will all nodes stop if on stake pool goes down?

Why not put the n-th withStakePool in the n-1-th's action?

What?

Will all nodes stop if on stake pool goes down?

No. If one pools doesn't start for some reason, others will be cancelled. But once they've all started, if one dies, the cluster will keep running.

What?

Something like this:

withStakePool args $ do withStakePool args $ do withStakePool args $ do . . n times .

Slower to start, but maybe simpler, and should stop all if one goes down.

Anviking

looks neat

KtorZ · 2020-06-08T12:01:25Z

bors r+

1731: Foundational work to get a cluster of pools for integration testing r=KtorZ a=KtorZ # Issue Number  N/A # Overview  - 5a263bc 📍 **move content of 'Test.Utils.Ports' to 'Cardano.Wallet.Network.Ports'** - 6ed9efc 📍 **rework the shelley launcher code so that we can start a cluster of nodes** using 'withCluster ... ... 0' will result in the same behavior as before. Right now, pools would exists as process but not on chain. In order to exist on chain, pools need to be registered, which will be done in a separate commit. - 1a8b720 📍 **fix database path** - 6745ea9 📍 **Regenerate nix** - 6b7b947 📍 **assign sensible default ordering to 'listAllTransactions'** # Comments  This should make it possible to run a cluster of pools as simply as: ```hs let numberOfPools = 3 withCluster tr Info numberOfPools $ \socketPath block0 (gp,vData) -> ... ``` So far, the pools won't have any _stake_ and won't therefore be any useful.. I'll give them some stake in another PR.  Co-authored-by: KtorZ <[email protected]> Co-authored-by: IOHK <[email protected]>

iohk-bors · 2020-06-08T12:19:57Z

Build failed

ci/hydra-build:required

KtorZ · 2020-06-08T17:40:53Z

bors r+

1731: Foundational work to get a cluster of pools for integration testing r=KtorZ a=KtorZ # Issue Number  N/A # Overview  - 5a263bc 📍 **move content of 'Test.Utils.Ports' to 'Cardano.Wallet.Network.Ports'** - 6ed9efc 📍 **rework the shelley launcher code so that we can start a cluster of nodes** using 'withCluster ... ... 0' will result in the same behavior as before. Right now, pools would exists as process but not on chain. In order to exist on chain, pools need to be registered, which will be done in a separate commit. - 1a8b720 📍 **fix database path** - 6745ea9 📍 **Regenerate nix** - 6b7b947 📍 **assign sensible default ordering to 'listAllTransactions'** # Comments  This should make it possible to run a cluster of pools as simply as: ```hs let numberOfPools = 3 withCluster tr Info numberOfPools $ \socketPath block0 (gp,vData) -> ... ``` So far, the pools won't have any _stake_ and won't therefore be any useful.. I'll give them some stake in another PR.  Co-authored-by: KtorZ <[email protected]> Co-authored-by: IOHK <[email protected]>

iohk-bors · 2020-06-08T17:56:41Z

Build failed

ci/hydra-build:required

rvl · 2020-06-09T04:46:05Z

lib/shelley/src/Cardano/Wallet/Shelley/Launch.hs

+        group <- waitAll
+        if length (catMaybes group) /= n then do
+            cancelAll
+            throwIO $ ProcessHasExited


This looks like it is mixing thrown exceptions with IO (Either e a). If the bft node fails, it will return Left. If the others fail to start, they will throw.

Yes, I've changed that later actually while continuing working on this, but this isn't visible from this PR yet

rvl · 2020-06-09T04:46:15Z

lib/shelley/test/data/cardano-node-shelley/genesis.yaml

 staking:
 slotsPerKESPeriod: 86400
-slotLength: 8
+slotLength: 0.2


Wow, that's a short slot. So if the tests get paused for 100x0.2=20secs (for whatever reason), they will not be able to continue.

I've tweaked according to what the ledger / consensus teams are using for testing as well; it seems fairly okay though. I don't get why the tests wouldn't be able to continue on if they pause for 20s 🤔 ?

The epoch length is 20s and I think if an entire epoch's worth of slots are missed then it will not be able to elect slot leaders for the next epoch. But it should be ok anyway. Nobody's going to press ctrl-z during integration tests and expect them to pass.

The cluster is made of one BFT leader + 3 pools though, so there's always the BFT leader capable of producing blocks. I am not sure how that influence the slot leader election however 🤔

Nobody's going to press ctrl-z during integration tests and expect them to pass.

I guess that's also a fair assumption :)

rvl · 2020-06-09T04:49:21Z

lib/core/src/Cardano/Wallet/Network/Ports.hs

+    , findPort
+    , randomUnusedTCPPorts


Ideally we wouldn't have test util functions in the core library.
But I think you couldn't put it into the core-integration library due to a package cycle. Is that correct?

Actually no. I needed to share them and Cardano.Wallet.Network.Ports felt like a logical place to put them. I see your point about the "non production" aspect of these two. I believe they could fit in core-integration as well 🤔

rvl · 2020-06-09T04:51:25Z

The error from bors does look like a legitimate test failure:

Failures:

  src/Test/Integration/Scenario/API/Shelley/Transactions.hs:887:5:
  1) API Specifications TRANS_LIST_02,03x - Can limit/order results with start, end and order
       uncaught exception: IOException of type UserError
       user error (expected a successful response but got an error: DecodeFailure "Error in $: parsing [] failed, expected Array, but encountered Object: Response {responseStatus = Status {statusCode = 400, statusMessage = \"Bad Request\"}, responseVersion = HTTP/1.1, responseHeaders = [(\"Transfer-Encoding\",\"chunked\"),(\"Date\",\"Mon, 08 Jun 2020 12:15:56 GMT\"),(\"Server\",\"Warp/3.3.5\")], responseBody = \"{\\\"code\\\":\\\"start_time_later_than_end_time\\\",\\\"message\\\":\\\"The specified start time '2020-06-08T12:15:56.471958Z' is later than the specified end time '2020-06-08T12:15:55.871958Z'.\\\"}\", responseCookieJar = CJ {expose = []}, responseClose' = ResponseClose}")

  To rerun use: --match "/API Specifications/TRANS_LIST_02,03x - Can limit/order results with start, end and order/"

Randomized with seed 1856737488

Finished in 223.8642 seconds
549 examples, 1 failure, 11 pending
builder for '/nix/store/6f339cm8hhd0k0xrdxmwbldkv8awg4yg-cardano-wallet-shelley-test-integration-2020.6.5-check.drv' failed with exit code 1

KtorZ · 2020-06-09T07:05:00Z

@rvl Yes it does, although I thought I fixed it in 6b7b947 but I obviously didn't. I only re-run once locally after this commit and it was probably luck that made it pass :/

One second is actually pretty large and we ended up in cases where, adding one second to the time of the first transaction would make it later than the second transaction! Causing some tests to fail for a wrong reason. This delta needs to be at most the slot length, so I've made it 100ms.

KtorZ · 2020-06-09T13:44:06Z

bors r+

iohk-bors · 2020-06-09T14:18:25Z

Build succeeded

KtorZ added 3 commits June 8, 2020 09:17

move content of 'Test.Utils.Ports' to 'Cardano.Wallet.Network.Ports'

5a263bc

fix database path

1a8b720

KtorZ requested a review from paweljakubas June 8, 2020 08:31

KtorZ self-assigned this Jun 8, 2020

KtorZ requested a review from Anviking June 8, 2020 08:31

IOHK and others added 2 commits June 8, 2020 08:40

Regenerate nix

6745ea9

assign sensible default ordering to 'listAllTransactions'

6b7b947

paweljakubas reviewed Jun 8, 2020

View reviewed changes

paweljakubas approved these changes Jun 8, 2020

View reviewed changes

Anviking reviewed Jun 8, 2020

View reviewed changes

Anviking approved these changes Jun 8, 2020

View reviewed changes

rewrite 'permutations' using built-in composition from Data.List

152ae3f

fix hlint suggestions

771b90f

KtorZ force-pushed the KtorZ/shelley-stake-pool-setup branch from 19559c4 to 771b90f Compare June 8, 2020 17:40

rvl approved these changes Jun 9, 2020

View reviewed changes

iohk-bors bot merged commit 0233f53 into master Jun 9, 2020

iohk-bors bot deleted the KtorZ/shelley-stake-pool-setup branch June 9, 2020 14:18

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Foundational work to get a cluster of pools for integration testing #1731

Foundational work to get a cluster of pools for integration testing #1731

KtorZ commented Jun 8, 2020 •

edited

Loading

paweljakubas Jun 8, 2020

KtorZ Jun 8, 2020

paweljakubas Jun 8, 2020

Anviking Jun 8, 2020

KtorZ Jun 8, 2020

paweljakubas left a comment

paweljakubas Jun 8, 2020

paweljakubas Jun 8, 2020 •

edited

Loading

KtorZ Jun 8, 2020

Anviking Jun 8, 2020

KtorZ Jun 8, 2020 •

edited

Loading

Anviking Jun 8, 2020

Anviking left a comment

KtorZ commented Jun 8, 2020

iohk-bors bot commented Jun 8, 2020

KtorZ commented Jun 8, 2020

iohk-bors bot commented Jun 8, 2020

rvl Jun 9, 2020

KtorZ Jun 9, 2020

rvl Jun 9, 2020

KtorZ Jun 9, 2020

rvl Jun 9, 2020

KtorZ Jun 9, 2020

KtorZ Jun 9, 2020

rvl Jun 9, 2020

KtorZ Jun 9, 2020

rvl commented Jun 9, 2020

KtorZ commented Jun 9, 2020

KtorZ commented Jun 9, 2020

iohk-bors bot commented Jun 9, 2020

Foundational work to get a cluster of pools for integration testing #1731

Foundational work to get a cluster of pools for integration testing #1731

Conversation

KtorZ commented Jun 8, 2020 • edited Loading

Issue Number

Overview

Comments

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

paweljakubas left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

paweljakubas Jun 8, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

KtorZ Jun 8, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Anviking left a comment

Choose a reason for hiding this comment

KtorZ commented Jun 8, 2020

iohk-bors bot commented Jun 8, 2020

Build failed

KtorZ commented Jun 8, 2020

iohk-bors bot commented Jun 8, 2020

Build failed

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rvl commented Jun 9, 2020

KtorZ commented Jun 9, 2020

KtorZ commented Jun 9, 2020

iohk-bors bot commented Jun 9, 2020

Build succeeded

KtorZ commented Jun 8, 2020 •

edited

Loading

paweljakubas Jun 8, 2020 •

edited

Loading

KtorZ Jun 8, 2020 •

edited

Loading