Slow building environment state

[sanex3339]

How fast docker should run building environment state?

For me on MacBook air 2015 (8gb ram) with one container (node 7.4.0) it takes 1:50 min, it looks very slow. Docker takes 2 gb ram and 2 core.

With 5 containers time about 17 min.

sudo: false
language: node_js

node_js:
  - "4.3.0"
  - "4"
  - "5"
  - "6"
  - "stable"

cache:
  yarn: true
  directories:
    - node_modules

script: "yarn run travis"

after_success:
  - yarn run test:coveralls
  - rm -rf ./coverage

[gokaygurcan]

Here's my .travis.yml and the results on Windows 10 Enterprise [Version 10.0.10240]:

language: node_js

node_js:
  - '7'
  - '6'
  - '4'

before_install:
  - npm install -g npm
  - npm install -g xo

script:
  - xo

PS C:\Users\gokaygurcan\Projects\trevor-test> Measure-Command {trevor}


Days              : 0
Hours             : 0
Minutes           : 6
Seconds           : 19
Milliseconds      : 256
Ticks             : 3792569434
TotalDays         : 0,00438954795601852
TotalHours        : 0,105349150944444
TotalMinutes      : 6,32094905666667
TotalSeconds      : 379,2569434
TotalMilliseconds : 379256,9434

As a sum, 6m 19s with 3 different versions. That makes ~2m for each like yours. My virtual machine's specs are 1 GB memory and 1 CPU core up to 100% cap btw.

~~ I'll try on macOS with same specs and update the comment tonight. ~~

Same .travis.yml on macOS (I'm not sure about the specs of the VM, it's the default one):

gokaygurcan ~/Projects/trevor-test $ time trevor

   ✔  4:  success
   ✔  6:  success
   ✔  7:  success

real	1m24.424s
user	0m3.481s
sys	0m2.400s

[sanex3339]

Very strange, because with 2 cores cpu was loaded about 75-80%

[vadimdemedes]

npm install is basically responsible for the majority of the time spent in "building environment" step, so it largely depends on the connection speed and npm install performance (which is known to be not that fast).

[sanex3339]

So, can we use yarn (optional) instead of npm, yarn much faster, it depends on image?
If so, we need a way to change image to custom (separate issue).

Can we keep environment (optional) between trevor command runs?

Because 17 min only for building environment - very huge, we need a way to do it faster.

[gabrielcsapo]

You could point npm cache to your host machine? This problem usually exists if there is no shared cache, in the case of docker the cache should be stored outside the container

[vadimdemedes]

So, can we use yarn (optional) instead of npm, yarn much faster, it depends on image?
If so, we need a way to change image to custom (separate issue).

This isn't easily possible right now, because Trevor uses the official Node.js Docker image, which doesn't have yarn bundled in.

Can we keep environment (optional) between trevor command runs?

It should preserve the Docker image between runs. I barely see "building environment" step, because it's cached on future executions.

[alechp]

You could point npm cache to your host machine? This problem usually exists if there is no shared cache, in the case of docker the cache should be stored outside the container

@gabrielcsapo I assumed I would have to run a local npm registry to significantly enhance building environment's npm install lag time. Is there another way that I can get similar performance to something like verdaccio with npm cache by itself?

From what I've read so far specifying npm cache alone only saves a fraction of what running a local npm registry does.

I'm still figuring out my options; any pointers are appreciated

[gabrielcsapo]

@alechp having a local registry would be a good option, but so would sharing the state of the node_modules between the X amount of machines. A possible solution would be doing something like https://jdlm.info/articles/2016/03/06/lessons-building-node-app-docker.html

[szmarczak]

I'm also having this issue, but it may be caused by slow internet connection.

npm install is basically responsible for the majority of the time spent in "building environment" step, so it largely depends on the connection speed and npm install performance (which is known to be not that fast).

Then why not time it? It would be very useful to display how long does npm install take.

[szmarczak]

@vadimdemedes Can you try running trevor in the Got repository?