feat: add jenkins ci

[dryajov]

No description provided.

[dryajov]

//cc @victorbjelkholm

[victorb]

👍 anything special needs to be done or just running npm run test?

[dryajov]

just running npm run test should be enough.

[victorb]

@dryajov great, seems everything worked out-of-the-box! One error on windows though:

connect EADDRNOTAVAIL 0.0.0.0:33028
Error: connect EADDRNOTAVAIL 0.0.0.0:33028
    at Object._errnoException (util.js:1031:13)
    at _exceptionWithHostPort (util.js:1052:20)
    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1195:14)

Make sure the tests uses port 0 in tests. If it's not that, I'm unsure what's going on.

Edit: and localhost rather than 0.0.0.0, no reason to bind to public IP.

[dryajov]

yeah, this is how we did it before the rewritten ipfsd-ctl (https://github.com/ipfs/interop/blob/2cd707c5648144a206da2a40a5a20a8bad495714/.aegir.js#L15..L18), they had to be well known ports so that browser tests could run against external daemons. PR #5 should address that. I wonder tho, what's running on that port?

[daviddias]

@dryajov merged #5, wanna rebase master onto this PR?

[dryajov]

@daviddias will do asap, afk atm.

[dryajov]

@diasdavid do we expect these tests to run on windows %100?

[daviddias]

@dryajov the answer is yes. However, we do know that there are some blockers, can you list what's stopping Travis from running them successfully?

[dryajov]

I'll run this on my windows virtual box see what the issues are - as for travis, I believe it hasn't really started running yet, the current run is queued.

[dryajov]

jenkins is failing on file exchange tests, failing to create/find the temp ipfs repo Error: ENOENT: no such file or directory, stat 'C:\Windows\TEMP\ipfs_a86cfd3c3b2616ad9ee9059549f16bfd'

[dryajov]

After some further investigation, I was able to trace both types of failures on windows.

The first failure

1) repo read repo: go -> js:

     Error: non-zero exit code 1

  while running: c:\jenkins\workspace\IPFS_interop_PR-7-O3VA47YDRA4ZLY2HRUBVLDFXWESOC7Q3ZANJDGRP7IYKACBSY33Q\node_modules\subcomandante\subcom 3580 c:\jenkins\workspace\IPFS_interop_PR-7-O3VA47YDRA4ZLY2HRUBVLDFXWESOC7Q3ZANJDGRP7IYKACBSY33Q\node_modules\go-ipfs-dep\go-ipfs\ipfs.exe daemon



  Error: cannot acquire lock: file "C:\\Windows\\TEMP\\1550f63addf8caa8e298e23de0405e22\\repo.lock" already locked

The failure above is currently due to the fact that graceful daemon shutdown with signals on windows is not possible, due to its lack of full signal support. Node.js will emulate some signals, however as stated in the docs https://nodejs.org/api/process.html#process_signal_events (under Note at the end of that section):

Note: Windows does not support sending signals, but Node.js offers some emulation with process.kill(), and subprocess.kill(). Sending signal 0 can be used to test for the existence of a process. Sending SIGINT, SIGTERM, and SIGKILL cause the unconditional termination of the target process.

Hence, gracefully shutting down a daemon on windows will not work. In the context of the failing test, this means that, when the go-ipfs daemon creates a repo, and its subsequently shutdown (with SIGINT), on windows the process is simply killed, which causes the repo to not get cleaned up (the repo.lock file is not removed), when the js-ipfs daemon tries to use the repo it fails because there is already a repo.lock file.

Note that, <ctrl+c> is correctly simulated on windows platforms - from the docs in the same section:

SIGINT from the terminal is supported on all platforms, and can usually be generated with <Ctrl>+C (though this may be configurable). It is not generated when terminal raw mode is enabled.

There is currently no workaround to that limitation - perhaps an alternative way of shutting down daemons needs to be implemented. I'll open an issue with a proposal, but so far I've thought of either an HTTP endpoint (shutdown) or perhaps a cleaner way, a 0 bytes file that when present in the repo, will trigger shutdown of the daemon. However, this will require modifying both - go and js implementations, regardless of the approach.

The second failure:

exchange files get directory js -> js: depth: 5, num: 10:

     Error: ENOENT: no such file or directory, stat 'C:\Windows\TEMP\ipfs_caae8c1b1269e53fd65b1e4ae8eaaf49'

Is due to some issues with random-fs - it seems to fail to create the temp dir structure - and it happens quite randomly. I'm not entirely sure what the cause is, but it seems to affect windows only at this time. Running the example program provided with random-fs (no unit tests sadly) I get this output:

$ node example/example.js

Added    0 files / 0 directories.
Deleted  33 files / 33 directories.
Errors   10 files / 0 directories.

Deleted:
  [DIR]  C:\Users\IEUser\projects\random-fs\example\temp
  [DIR]  C:\Users\IEUser\projects\random-fs\example\temp\bilsteds-tachypnoea
  [FILE] C:\Users\IEUser\projects\random-fs\example\temp\bilsteds-tachypnoea\intubation-synarchies.ak
  [DIR]  C:\Users\IEUser\projects\random-fs\example\temp\bourd-pensilities
  [FILE] C:\Users\IEUser\projects\random-fs\example\temp\bourd-pensilities\monsoons-beachcombed.ilt
  [DIR]  C:\Users\IEUser\projects\random-fs\example\temp\bumfuzzles-griddled
  [FILE] C:\Users\IEUser\projects\random-fs\example\temp\bumfuzzles-griddled\touracos-afrit.ske
  [DIR]  C:\Users\IEUser\projects\random-fs\example\temp\cums-giardiases
  [FILE] C:\Users\IEUser\projects\random-fs\example\temp\cums-giardiases\nitrations-clocked.szo
  [DIR]  C:\Users\IEUser\projects\random-fs\example\temp\dabbity-expirations
  [FILE] C:\Users\IEUser\projects\random-fs\example\temp\dabbity-expirations\pinnulated-deserpidine.iu
  [DIR]  C:\Users\IEUser\projects\random-fs\example\temp\desiccate-steenings
  [FILE] C:\Users\IEUser\projects\random-fs\example\temp\desiccate-steenings\empoverishments-outreasons.ztd
  [DIR]  C:\Users\IEUser\projects\random-fs\example\temp\distributary-nemophilas
  [FILE] C:\Users\IEUser\projects\random-fs\example\temp\distributary-nemophilas\moonfish-pyramidic.hnw
  [DIR]  C:\Users\IEUser\projects\random-fs\example\temp\dogears-metols
  [FILE] C:\Users\IEUser\projects\random-fs\example\temp\dogears-metols\agglutinating-protruding.lsq
  [DIR]  C:\Users\IEUser\projects\random-fs\example\temp\empyreuma-pleasers
  [FILE] C:\Users\IEUser\projects\random-fs\example\temp\empyreuma-pleasers\kelters-gulching.jv
  [DIR]  C:\Users\IEUser\projects\random-fs\example\temp\entertainingly-booty
  [FILE] C:\Users\IEUser\projects\random-fs\example\temp\entertainingly-booty\bridgeboards-imperfectnesses.ljm
  [DIR]  C:\Users\IEUser\projects\random-fs\example\temp\enunciation-rhamphotheca
  [FILE] C:\Users\IEUser\projects\random-fs\example\temp\enunciation-rhamphotheca\symploce-leachate.bv
  [DIR]  C:\Users\IEUser\projects\random-fs\example\temp\evulgated-unwreathing
  [FILE] C:\Users\IEUser\projects\random-fs\example\temp\evulgated-unwreathing\presentism-customise.dae
  [DIR]  C:\Users\IEUser\projects\random-fs\example\temp\feminity-dimidiating
  [FILE] C:\Users\IEUser\projects\random-fs\example\temp\feminity-dimidiating\mechanists-pamperedness.qti
  [DIR]  C:\Users\IEUser\projects\random-fs\example\temp\glidings-seizure
  [FILE] C:\Users\IEUser\projects\random-fs\example\temp\glidings-seizure\craaling-preschedules.qfl
  [DIR]  C:\Users\IEUser\projects\random-fs\example\temp\haricot-tainting
  [FILE] C:\Users\IEUser\projects\random-fs\example\temp\haricot-tainting\cultured-succumb.vpx
  [DIR]  C:\Users\IEUser\projects\random-fs\example\temp\hydrolysate-lithotritists
  [FILE] C:\Users\IEUser\projects\random-fs\example\temp\hydrolysate-lithotritists\albitical-cockshut.rif
  [DIR]  C:\Users\IEUser\projects\random-fs\example\temp\inexcitable-anthers
  [FILE] C:\Users\IEUser\projects\random-fs\example\temp\inexcitable-anthers\semisolid-undecidabilities.tkz
  [DIR]  C:\Users\IEUser\projects\random-fs\example\temp\knucklebone-pyrometer
  [FILE] C:\Users\IEUser\projects\random-fs\example\temp\knucklebone-pyrometer\dermatographies-constructing.jyo
  [DIR]  C:\Users\IEUser\projects\random-fs\example\temp\lump-ruddinesses
  [FILE] C:\Users\IEUser\projects\random-fs\example\temp\lump-ruddinesses\leets-rechie.vrl
  [DIR]  C:\Users\IEUser\projects\random-fs\example\temp\mollitious-tridominium
  [FILE] C:\Users\IEUser\projects\random-fs\example\temp\mollitious-tridominium\bringdown-diglyphs.bs
  [DIR]  C:\Users\IEUser\projects\random-fs\example\temp\oreweed-piccolos
  [FILE] C:\Users\IEUser\projects\random-fs\example\temp\oreweed-piccolos\weewees-jargonizations.vj
  [DIR]  C:\Users\IEUser\projects\random-fs\example\temp\orthograde-deferral
  [FILE] C:\Users\IEUser\projects\random-fs\example\temp\orthograde-deferral\unqueenliest-bimahs.oqs
  [DIR]  C:\Users\IEUser\projects\random-fs\example\temp\outwound-grein
  [FILE] C:\Users\IEUser\projects\random-fs\example\temp\outwound-grein\ptyalisms-fritflies.zsx
  [DIR]  C:\Users\IEUser\projects\random-fs\example\temp\presbyterian-patterers
  [FILE] C:\Users\IEUser\projects\random-fs\example\temp\presbyterian-patterers\waulkmill-enprints.vb
  [DIR]  C:\Users\IEUser\projects\random-fs\example\temp\pyroclastics-rhythmises
  [FILE] C:\Users\IEUser\projects\random-fs\example\temp\pyroclastics-rhythmises\pretext-gammons.ou
  [DIR]  C:\Users\IEUser\projects\random-fs\example\temp\saboteur-cainogenetic
  [FILE] C:\Users\IEUser\projects\random-fs\example\temp\saboteur-cainogenetic\kinking-proportional.nk
  [DIR]  C:\Users\IEUser\projects\random-fs\example\temp\shtickier-unilobed
  [FILE] C:\Users\IEUser\projects\random-fs\example\temp\shtickier-unilobed\chorister-fractious.low
  [DIR]  C:\Users\IEUser\projects\random-fs\example\temp\somnambulant-herdic
  [FILE] C:\Users\IEUser\projects\random-fs\example\temp\somnambulant-herdic\overblown-totties.mlz
  [FILE] C:\Users\IEUser\projects\random-fs\example\temp\steeve-zambuk.sj
  [DIR]  C:\Users\IEUser\projects\random-fs\example\temp\vendeuses-leadoffs
  [FILE] C:\Users\IEUser\projects\random-fs\example\temp\vendeuses-leadoffs\defouls-overgreening.yx
  [DIR]  C:\Users\IEUser\projects\random-fs\example\temp\vicinal-inhumated
  [FILE] C:\Users\IEUser\projects\random-fs\example\temp\vicinal-inhumated\arbors-penologist.qi
  [DIR]  C:\Users\IEUser\projects\random-fs\example\temp\wan-entailments
  [FILE] C:\Users\IEUser\projects\random-fs\example\temp\wan-entailments\tochered-synoecises.rhv
  [DIR]  C:\Users\IEUser\projects\random-fs\example\temp\whiffings-forebodingly
  [FILE] C:\Users\IEUser\projects\random-fs\example\temp\whiffings-forebodingly\sponsions-jure.ym

Added:


Errors:
  [FILE] Could not add file: C:\Users\IEUser\projects\random-fs\example\temp\axiomatizing-trooz\viscidnesses-webmail.gu
Error: ENOENT: no such file or directory, mkdir 'C:\Users\IEUser\projects\random-fs\example\temp\axiomatizing-trooz'
  [FILE] Could not add file: C:\Users\IEUser\projects\random-fs\example\temp\ghasts-voguier\cutpurses-rulier\chillies-interrogatory\basswoods-assemblywoman.sx
Error: ENOENT: no such file or directory, mkdir 'C:\Users\IEUser\projects\random-fs\example\temp\ghasts-voguier\cutpurses-rulier\chillies-interrogatory'
  [FILE] Could not add file: C:\Users\IEUser\projects\random-fs\example\temp\languorously-chophouses\ogles-expropriations.qu
Error: ENOENT: no such file or directory, mkdir 'C:\Users\IEUser\projects\random-fs\example\temp\languorously-chophouses'
  [FILE] Could not add file: C:\Users\IEUser\projects\random-fs\example\temp\megahit-vesicle\vulgarian-pussleys\frothiest-cytasters.ixi
Error: ENOENT: no such file or directory, mkdir 'C:\Users\IEUser\projects\random-fs\example\temp\megahit-vesicle\vulgarian-pussleys'
  [FILE] Could not add file: C:\Users\IEUser\projects\random-fs\example\temp\muttonfish-bestiaries\trachea-dehypnotized\sleeplike-ferule\disempower-mystifier.fz
Error: ENOENT: no such file or directory, mkdir 'C:\Users\IEUser\projects\random-fs\example\temp\muttonfish-bestiaries\trachea-dehypnotized\sleeplike-ferule'
  [FILE] Could not add file: C:\Users\IEUser\projects\random-fs\example\temp\polyzoarial-nephroses\sarkings-vets\indigenes-bragger.stf
Error: ENOENT: no such file or directory, mkdir 'C:\Users\IEUser\projects\random-fs\example\temp\polyzoarial-nephroses\sarkings-vets'
  [FILE] Could not add file: C:\Users\IEUser\projects\random-fs\example\temp\selvaging-steatite\unemptied-nebulously.uc
Error: ENOENT: no such file or directory, mkdir 'C:\Users\IEUser\projects\random-fs\example\temp\selvaging-steatite'
  [FILE] Could not add file: C:\Users\IEUser\projects\random-fs\example\temp\snarlingly-prolers\discrediting-appetisement\smerked-refluxes.vmo
Error: ENOENT: no such file or directory, mkdir 'C:\Users\IEUser\projects\random-fs\example\temp\snarlingly-prolers\discrediting-appetisement'
  [FILE] Could not add file: C:\Users\IEUser\projects\random-fs\example\temp\synded-prebills\reggaeton-rockweeds.ta
Error: ENOENT: no such file or directory, mkdir 'C:\Users\IEUser\projects\random-fs\example\temp\synded-prebills'
  [FILE] Could not add file: C:\Users\IEUser\projects\random-fs\example\temp\unreflectively-horsepower\ploughed-dorized.mc
Error: ENOENT: no such file or directory, mkdir 'C:\Users\IEUser\projects\random-fs\example\temp\unreflectively-horsepower'

The above output, indicates that it's failing to create the directories - not entirely sure why tho, looking at the code, its pretty straight forward - just mkdir a directory, however that seems to fail.

I think we should skip this failing tests on windows for now but open individual issues so we can fix them ASAP.

[dryajov]

issue to implement graceful shutdown - ipfs/js-ipfs#1192

[daviddias]

@dryajov can we track both of those issues in individual issues? Thank you so much for investigating the cause.

I'm going to go ahead and merge this PR since it only is adding Jenkins CI