Add a job that runs master + v10.x with python tools/test.py --worker

[addaleax]

Now that nodejs/node#20876 has landed, it would be great to have something in CI that runs the test suite under workers. It’s probably not feasible to make it part of a regular make test run, or add it to all CI nodes because the test suite is going to run twice as long that way.

Instead, I’d imagine we want a CI job for this. What do we need? Something in the Makefile + a job that runs that particular make target?

[maclover7]

@addaleax In theory we should be able to get this working with https://ci.nodejs.org/job/node-test-commit-custom-suites, using --worker as the input for TEST_ARGS, taking a look now

[addaleax]

@maclover7 Does that mean it would run only if requested? Because you’re right, that sounds doable, but I’d love to have it be part of node-test-commit

[maclover7]

@addaleax Yep, it technically runs on an "as requested" basis. But we can add a run of this job to node-test-commit like all of the other subjobs (node-test-commit-plinux, node-test-commit-arm, etc), or just node-daily-master if that's what's wanted

Giving this another run: https://ci.nodejs.org/job/node-test-commit-custom-suites/10/

[addaleax]

@maclover7 https://ci.nodejs.org/job/node-test-commit-custom-suites/10/nodes=rhel72-s390x/console looks like there’s some kind of encoding trouble?

[maclover7]

@addaleax Sorry this fell off my radar. Was able to get through ~2000 tests before getting two timeouts. Would you be be able to look into these failures? I believe you should have access to trigger builds of the node-test-commit-custom-suites job, where you can customize the arguments for python tools/test.py, and also pick the specific Jenkins node(s) that will run the job.

[addaleax]

@maclover Thanks! I have fixes for the timeouts in nodejs/node#21791. :)

(This is why I want to get this into node-test-commit, btw … 😄)

[maclover7]

@addaleax Looks like --worker is green now (🎉🎉🎉). Is running with --worker on Ubuntu 16.04 "good enough" to add to node-test-commit? Or are more platforms needed to make sure there's no breakage?

[addaleax]

@maclover7 I think it's good enough -- there can always be weird combinations of platforms creating odd bugs, but having a single Linux job for Workers in there would already be a huge step :)

[maclover7]

Was able to get an all-green node-test-commit build (including --worker tests; is on my personal clone of the live job). Will look at rolling this out to the live node-test-commit job.

[maclover7]

Configuration should be updated now -- here's the first all green node-test-commit job. Will leave this open for another day or so in case problems come up, but otherwise I think this is resolved now :)

[addaleax]

Woo 🎉 – thanks for all the work on this!

[addaleax]

@maclover7 I think it’s safe to close this now – again, thank you! 💙