[DEVOPS-121] Cardano sl faster build #49

deepfire · 2017-05-20T16:36:00Z

Besides partially addressing https://issues.serokell.io/issue/DEVOPS-36#comment=96-3716, this also sets up NIX_PATH, so that a number of tools needn't have -I specified -- and so this pulls towards unification of nixpkgs specification (don't remember the issue).

cc @domenkozar, @jmitchell

TODO:

decide on testing being a part of all Hydra builds
decide on whether building all 13 executables is worth it for all Hydra builds

domenkozar · 2017-05-21T09:40:46Z

default.nix

@@ -11,6 +11,13 @@ let
  });
  socket-io-src = pkgs.fetchgit (removeAttrs (lib.importJSON ./pkgs/engine-io.json) ["date"]);
  cardano-sl-src = pkgs.fetchgit (removeAttrs (lib.importJSON ./pkgs/cardano-sl.json) ["date"]);
+  cores   = 4;


Best to have threads and cores should be part of the function inputs for default.nix

domenkozar · 2017-05-21T09:42:42Z

default.nix

+    buildTarget = subset;
+  });
+  buildSpeedupFlagsMarlowScaling = drv : appendConfigureFlag drv
+    "--ghc-option=-rtsopts --ghc-option=+RTS --ghc-option=-N${toString threads} --ghc-option=-A128m --ghc-option=-n2m --ghc-option=-qb0 --ghc-option=-qn${toString cores} --ghc-option=-RTS";


-N is already set from cabal in cardano-sl

The problem here is.. we need to set both -N and -qn, in the 2:1 ratio, as per Simon Marlow's suggestion (see https://ghc.haskell.org/trac/ghc/ticket/9221#comment:72).

Once we accept №1, a need comes to have an authoritative Nix-evaluation-time source of information on the amount of cores on the system.

I couldn't find such a source, hence the hard-coding.

Since we hard-code -qn, I guess it's kinda debatable if leaving -N free is a good thing. It would certainly somewhat obscure the Marlow ratio between cores and threads.

In fact, @domenkozar, I can't find where cabal sets -N -- at least it doesn't in https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/haskell-modules/generic-builder.nix

Cabal does set -j, but that's ghc --make-level parallelism.

domenkozar · 2017-05-21T10:52:03Z

For -static we should skip tests and build only targets we need for deployments. cardano-sl should still be a full build with tests etc.

I'd like to see time differences here before/after this PR and with current VM and 16 core VM.

deepfire-pusher · 2017-05-21T14:15:15Z

@domenkozar, regarding the testing setup:

the 16-core VM, should it be a new one, or the currently-provisioned i3.2xlarge is sufficient?
Also, your spec implies a 2x2 test matrix with 4 results, right?

deepfire-pusher · 2017-05-21T14:17:42Z

@domenkozar, regarding the second question from the PR TODO:

How do we subset the 13 executables -- are we clear on what is needed for deployments?

Should I judge this myself? Perhaps incremental correctness, with an optimistic baseline, can work here : -)

domenkozar · 2017-05-21T14:24:36Z

So far we need cardano-node and cardano-explorer binaries (I just grepped /bin/)

domenkozar · 2017-05-21T14:26:24Z

That should be good enough
4 results: before/after, 2 core/8 core

deepfire-pusher · 2017-05-21T14:31:43Z

@domenkozar, thanks, I'll try to do this today

domenkozar · 2017-05-21T14:33:40Z

@deepfire-pusher I see now that you have updated the issue. Those statistics are good enough.

Let's build only cardano-node and cardano-explorer executables and that should be more than good enough.

domenkozar · 2017-05-21T14:40:52Z

Note that nix already pases -j$NIX_BUILD_CORES to ghc, so it's not single threaded by default.

deepfire-pusher · 2017-05-21T14:44:38Z

If it's already parallel.. well, I that's scary, it means that hydra.iohk.io is even worse, single-threaded, than I thought : -)

domenkozar · 2017-05-21T14:45:19Z

It's slower because there are multiple packages compiling together. Up to 4.

deepfire-pusher · 2017-05-21T15:11:12Z

Ah, I see -- that was the problem I was trying to attack initially, with my Hydra instrumentation -- the direct comparability of measurements.

But I agree, just running the tests manually everywhere is faster, and the result is useful for judgement of effect of changes.

deepfire · 2017-05-26T11:18:28Z

@domenkozar, what is our next step here?

domenkozar · 2017-05-30T11:35:12Z

This will be handled as part of DEVOPS-121.

…/issue/DEVOPS-36#comment=96-3716

…efault to 8)

Details: https://factisresearch.blogspot.ru/2010/09/speeding-up-your-cabal-builds-part-ii.html

domenkozar · 2017-10-06T13:23:12Z

I'm going to close this, as expressions moved to cardano-sl repo and we can revisit the PR once DEVOPS-121 is picked up.

deepfire requested a review from domenkozar May 20, 2017 16:37

domenkozar reviewed May 21, 2017

View reviewed changes

deepfire force-pushed the cardano-sl-faster-build branch from 194b5f6 to 2f2e970 Compare May 22, 2017 10:55

deepfire changed the title ~~[DEVOPS-36] Cardano sl faster build~~ [DEVOPS-121] Cardano sl faster build May 29, 2017

deepfire force-pushed the cardano-sl-faster-build branch from 1cecf26 to 09a7bbb Compare June 8, 2017 17:40

deepfire added 7 commits June 16, 2017 17:35

shell.nix: bump nixpkgs to tip of NixOS 17.03; adapt to changes

5f97eb7

shell.nix: slight refactoring

2b41333

shell.nix: attempt to provide NIX_PATH for better development experience

3695407

shell.nix: add nix for the sake of --pure

fc58116

[DEVOPS-36] Build speedups in following of https://issues.serokell.io…

30369d2

…/issue/DEVOPS-36#comment=96-3716

[DEVOPS-36] review: add builder-threads parameter to default.nix (d…

9927c77

…efault to 8)

[DEVOPS-36] disable build subsetting, due to a cabal limitation

041f668

Details: https://factisresearch.blogspot.ru/2010/09/speeding-up-your-cabal-builds-part-ii.html

deepfire force-pushed the cardano-sl-faster-build branch from 09a7bbb to 041f668 Compare June 16, 2017 14:35

domenkozar closed this Oct 6, 2017

deepfire deleted the cardano-sl-faster-build branch October 30, 2017 14:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[DEVOPS-121] Cardano sl faster build #49

[DEVOPS-121] Cardano sl faster build #49

deepfire commented May 20, 2017 •

edited

Loading

domenkozar May 21, 2017 •

edited

Loading

deepfire May 22, 2017

domenkozar May 21, 2017

deepfire May 22, 2017 •

edited

Loading

deepfire May 22, 2017

domenkozar commented May 21, 2017

deepfire-pusher commented May 21, 2017 •

edited

Loading

deepfire-pusher commented May 21, 2017 •

edited

Loading

domenkozar commented May 21, 2017

domenkozar commented May 21, 2017

deepfire-pusher commented May 21, 2017

domenkozar commented May 21, 2017

domenkozar commented May 21, 2017

deepfire-pusher commented May 21, 2017 •

edited

Loading

domenkozar commented May 21, 2017

deepfire-pusher commented May 21, 2017 •

edited

Loading

deepfire commented May 26, 2017

domenkozar commented May 30, 2017

domenkozar commented Oct 6, 2017

[DEVOPS-121] Cardano sl faster build #49

[DEVOPS-121] Cardano sl faster build #49

Conversation

deepfire commented May 20, 2017 • edited Loading

domenkozar May 21, 2017 • edited Loading

Choose a reason for hiding this comment

deepfire May 22, 2017

Choose a reason for hiding this comment

domenkozar May 21, 2017

Choose a reason for hiding this comment

deepfire May 22, 2017 • edited Loading

Choose a reason for hiding this comment

deepfire May 22, 2017

Choose a reason for hiding this comment

domenkozar commented May 21, 2017

deepfire-pusher commented May 21, 2017 • edited Loading

deepfire-pusher commented May 21, 2017 • edited Loading

domenkozar commented May 21, 2017

domenkozar commented May 21, 2017

deepfire-pusher commented May 21, 2017

domenkozar commented May 21, 2017

domenkozar commented May 21, 2017

deepfire-pusher commented May 21, 2017 • edited Loading

domenkozar commented May 21, 2017

deepfire-pusher commented May 21, 2017 • edited Loading

deepfire commented May 26, 2017

domenkozar commented May 30, 2017

domenkozar commented Oct 6, 2017

deepfire commented May 20, 2017 •

edited

Loading

domenkozar May 21, 2017 •

edited

Loading

deepfire May 22, 2017 •

edited

Loading

deepfire-pusher commented May 21, 2017 •

edited

Loading

deepfire-pusher commented May 21, 2017 •

edited

Loading

deepfire-pusher commented May 21, 2017 •

edited

Loading

deepfire-pusher commented May 21, 2017 •

edited

Loading