Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(node/cluster): cluster module for Node compat #2271

Merged
merged 30 commits into from
Jan 12, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
e954f2b
feat: `primary.ts` implementation for `cluster` compat module
cmorten May 25, 2022
a7b5c98
feat: flesh out cluster implementation
cmorten May 26, 2022
45eef43
docs: add copyright notices
cmorten May 26, 2022
76aa486
Update node/cluster.ts
cmorten May 26, 2022
e59c163
fix: formatting
cmorten May 26, 2022
0c6a178
Merge branch 'main' into feat/node-worker-compat
cmorten May 26, 2022
f729723
fix: dynamic import not playing nice
cmorten May 26, 2022
d47c392
test: env var check in `cluster` module requires permission
cmorten May 26, 2022
de185ab
chore: types
cmorten May 26, 2022
f204b21
kick ci
cmorten May 26, 2022
c792377
feat: support cluster in `net` module
cmorten May 27, 2022
10e8b56
refactor: remove cluster test as process methods not implemented yet
cmorten May 31, 2022
3498a74
fix: formatting
cmorten May 31, 2022
2adda6c
fix: remove unused variable
cmorten May 31, 2022
6643834
Merge branch 'main' into feat/node-worker-compat
cmorten Dec 20, 2022
1e9d5e1
refactor: reinstate ForkOptions from merge conflict
cmorten Dec 20, 2022
58e5df5
fix: align types
cmorten Dec 20, 2022
c000c4d
refactor: appease linter
cmorten Dec 20, 2022
7a4a6fc
fix: also export cluster as named export
cmorten Dec 20, 2022
61fd701
fix: formatting
cmorten Dec 20, 2022
f857bd5
fix: appease linter
cmorten Dec 20, 2022
3b8919d
refactor: everything an object?
cmorten Dec 20, 2022
c0c16f6
Merge branch 'main' into feat/node-worker-compat
cmorten Jan 3, 2023
67d877d
chore: copyright dates
cmorten Jan 3, 2023
f811ac8
refactor: lazy load circular dep classes
cmorten Jan 4, 2023
29ce19e
revert: default object export
cmorten Jan 4, 2023
ced5aa4
fix: cluster exports
cmorten Jan 4, 2023
b6d63da
test: remove cluster tests that fail on ubuntu
cmorten Jan 4, 2023
8f06eb2
chore: kick ci
cmorten Jan 4, 2023
75f6f6f
fix: `_fs_access_test.ts`
cmorten Jan 4, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions node/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,30 +124,30 @@ $ deno run --allow-read --allow-net --allow-write node/_tools/setup.ts -n

To run the tests you have set up, do the following:

```zsh
$ deno test --allow-read --allow-run node/_tools/test.ts
```shellsession
$ deno test -A --unstable node/_tools/test.ts
```

If you want to run specific Node.js test files, you can use the following
command

```shellsession
$ deno test -A node/_tools/test.ts -- <pattern-to-match>
$ deno test -A --unstable node/_tools/test.ts -- <pattern-to-match>
```

For example, if you want to run only
`node/_tools/test/parallel/test-event-emitter-check-listener-leaks.js`, you can
use:

```shellsession
$ deno test -A node/_tools/test.ts -- test-event-emitter-check-listener-leaks.js
$ deno test -A --unstable node/_tools/test.ts -- test-event-emitter-check-listener-leaks.js
```

If you want to run all test files which contains `event-emitter` in filename,
then you can use:

```shellsession
$ deno test -A node/_tools/test.ts -- event-emitter
$ deno test -A --unstable node/_tools/test.ts -- event-emitter
```

The test should be passing with the latest deno, so if the test fails, try the
Expand Down
4 changes: 4 additions & 0 deletions node/_events.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -289,6 +289,10 @@ interface StaticEventEmitterOptions {
* @since v0.1.26
*/
export class EventEmitter {
_events: any;
_eventsCount: any;
_maxListeners: any;

/**
* Alias for `emitter.on(eventName, listener)`.
* @since v0.1.26
Expand Down
4 changes: 2 additions & 2 deletions node/_fs/_fs_access_test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Deno.test(
async () => {
const file = await Deno.makeTempFile();
try {
Deno.chmod(file, 0o600);
await Deno.chmod(file, 0o600);
await fs.promises.access(file, fs.constants.R_OK);
await fs.promises.access(file, fs.constants.W_OK);
await assertRejects(async () => {
Expand Down Expand Up @@ -40,7 +40,7 @@ Deno.test(
() => {
const file = Deno.makeTempFileSync();
try {
Deno.chmod(file, 0o600);
Deno.chmodSync(file, 0o600);
fs.accessSync(file, fs.constants.R_OK);
fs.accessSync(file, fs.constants.W_OK);
assertThrows(() => {
Expand Down
9 changes: 1 addition & 8 deletions node/_tools/TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
NOTE: This file should not be manually edited. Please edit `config.json` and run
`deno task node:setup` instead.

Total: 2811
Total: 2804

- [abort/test-abort-backtrace.js](https://github.com/nodejs/node/tree/v18.12.1/test/abort/test-abort-backtrace.js)
- [abort/test-abort-fatal-error.js](https://github.com/nodejs/node/tree/v18.12.1/test/abort/test-abort-fatal-error.js)
Expand Down Expand Up @@ -363,8 +363,6 @@ Total: 2811
- [parallel/test-cluster-bind-privileged-port.js](https://github.com/nodejs/node/tree/v18.12.1/test/parallel/test-cluster-bind-privileged-port.js)
- [parallel/test-cluster-bind-twice.js](https://github.com/nodejs/node/tree/v18.12.1/test/parallel/test-cluster-bind-twice.js)
- [parallel/test-cluster-call-and-destroy.js](https://github.com/nodejs/node/tree/v18.12.1/test/parallel/test-cluster-call-and-destroy.js)
- [parallel/test-cluster-child-index-dgram.js](https://github.com/nodejs/node/tree/v18.12.1/test/parallel/test-cluster-child-index-dgram.js)
- [parallel/test-cluster-child-index-net.js](https://github.com/nodejs/node/tree/v18.12.1/test/parallel/test-cluster-child-index-net.js)
- [parallel/test-cluster-concurrent-disconnect.js](https://github.com/nodejs/node/tree/v18.12.1/test/parallel/test-cluster-concurrent-disconnect.js)
- [parallel/test-cluster-cwd.js](https://github.com/nodejs/node/tree/v18.12.1/test/parallel/test-cluster-cwd.js)
- [parallel/test-cluster-dgram-1.js](https://github.com/nodejs/node/tree/v18.12.1/test/parallel/test-cluster-dgram-1.js)
Expand All @@ -374,11 +372,8 @@ Total: 2811
- [parallel/test-cluster-dgram-reuse.js](https://github.com/nodejs/node/tree/v18.12.1/test/parallel/test-cluster-dgram-reuse.js)
- [parallel/test-cluster-disconnect-before-exit.js](https://github.com/nodejs/node/tree/v18.12.1/test/parallel/test-cluster-disconnect-before-exit.js)
- [parallel/test-cluster-disconnect-exitedAfterDisconnect-race.js](https://github.com/nodejs/node/tree/v18.12.1/test/parallel/test-cluster-disconnect-exitedAfterDisconnect-race.js)
- [parallel/test-cluster-disconnect-idle-worker.js](https://github.com/nodejs/node/tree/v18.12.1/test/parallel/test-cluster-disconnect-idle-worker.js)
- [parallel/test-cluster-disconnect-leak.js](https://github.com/nodejs/node/tree/v18.12.1/test/parallel/test-cluster-disconnect-leak.js)
- [parallel/test-cluster-disconnect-race.js](https://github.com/nodejs/node/tree/v18.12.1/test/parallel/test-cluster-disconnect-race.js)
- [parallel/test-cluster-disconnect-unshared-tcp.js](https://github.com/nodejs/node/tree/v18.12.1/test/parallel/test-cluster-disconnect-unshared-tcp.js)
- [parallel/test-cluster-disconnect-unshared-udp.js](https://github.com/nodejs/node/tree/v18.12.1/test/parallel/test-cluster-disconnect-unshared-udp.js)
- [parallel/test-cluster-disconnect-with-no-workers.js](https://github.com/nodejs/node/tree/v18.12.1/test/parallel/test-cluster-disconnect-with-no-workers.js)
- [parallel/test-cluster-disconnect.js](https://github.com/nodejs/node/tree/v18.12.1/test/parallel/test-cluster-disconnect.js)
- [parallel/test-cluster-eaccess.js](https://github.com/nodejs/node/tree/v18.12.1/test/parallel/test-cluster-eaccess.js)
Expand All @@ -403,7 +398,6 @@ Total: 2811
- [parallel/test-cluster-primary-error.js](https://github.com/nodejs/node/tree/v18.12.1/test/parallel/test-cluster-primary-error.js)
- [parallel/test-cluster-primary-kill.js](https://github.com/nodejs/node/tree/v18.12.1/test/parallel/test-cluster-primary-kill.js)
- [parallel/test-cluster-process-disconnect.js](https://github.com/nodejs/node/tree/v18.12.1/test/parallel/test-cluster-process-disconnect.js)
- [parallel/test-cluster-rr-domain-listen.js](https://github.com/nodejs/node/tree/v18.12.1/test/parallel/test-cluster-rr-domain-listen.js)
- [parallel/test-cluster-rr-ref.js](https://github.com/nodejs/node/tree/v18.12.1/test/parallel/test-cluster-rr-ref.js)
- [parallel/test-cluster-send-deadlock.js](https://github.com/nodejs/node/tree/v18.12.1/test/parallel/test-cluster-send-deadlock.js)
- [parallel/test-cluster-send-handle-twice.js](https://github.com/nodejs/node/tree/v18.12.1/test/parallel/test-cluster-send-handle-twice.js)
Expand All @@ -430,7 +424,6 @@ Total: 2811
- [parallel/test-cluster-worker-handle-close.js](https://github.com/nodejs/node/tree/v18.12.1/test/parallel/test-cluster-worker-handle-close.js)
- [parallel/test-cluster-worker-init.js](https://github.com/nodejs/node/tree/v18.12.1/test/parallel/test-cluster-worker-init.js)
- [parallel/test-cluster-worker-isconnected.js](https://github.com/nodejs/node/tree/v18.12.1/test/parallel/test-cluster-worker-isconnected.js)
- [parallel/test-cluster-worker-isdead.js](https://github.com/nodejs/node/tree/v18.12.1/test/parallel/test-cluster-worker-isdead.js)
- [parallel/test-cluster-worker-kill-signal.js](https://github.com/nodejs/node/tree/v18.12.1/test/parallel/test-cluster-worker-kill-signal.js)
- [parallel/test-cluster-worker-kill.js](https://github.com/nodejs/node/tree/v18.12.1/test/parallel/test-cluster-worker-kill.js)
- [parallel/test-cluster-worker-no-exit.js](https://github.com/nodejs/node/tree/v18.12.1/test/parallel/test-cluster-worker-no-exit.js)
Expand Down
7 changes: 7 additions & 0 deletions node/_tools/config.json
Original file line number Diff line number Diff line change
Expand Up @@ -202,6 +202,13 @@
"test-child-process-spawnsync-validation-errors.js",
"test-child-process-spawnsync.js",
"test-client-request-destroy.js",
"test-cluster-child-index-dgram.js",
"test-cluster-child-index-net.js",
"test-cluster-disconnect-idle-worker.js",
"test-cluster-disconnect-unshared-tcp.js",
"test-cluster-disconnect-unshared-udp.js",
"test-cluster-rr-domain-listen.js",
"test-cluster-worker-isdead.js",
"test-console-async-write-error.js",
"test-console-group.js",
"test-console-instance.js",
Expand Down
17 changes: 17 additions & 0 deletions node/_tools/require.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
// Copyright 2018-2023 the Deno authors. All rights reserved. MIT license.

/**
* This module is used as an entry point for test files utilizing the cluster
* module which forks processes and cannot use `.ts` files due to
* incompatibility with Deno's Node module resolution.
* See https://github.com/denoland/deno/blob/main/cli/node/mod.rs#L725
*
* The idea is to emulate a CommonJS environment without having to modify
* the test files in any way
*
* Running with all permissions and unstable is recommended
*
* Usage: `deno run -A --unstable require.mjs my_commonjs_file.js`
*/

import "./require.ts";
2 changes: 1 addition & 1 deletion node/_tools/test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ for await (const path of testPaths) {
console.log(`Error: "${path}" failed`);
console.log(
"You can repeat only this test with the command:",
magenta(`deno test -A node/_tools/test.ts -- ${path}`),
magenta(`deno test -A --unstable node/_tools/test.ts -- ${path}`),
);
fail(decodedStderr);
}
Expand Down
47 changes: 47 additions & 0 deletions node/_tools/test/parallel/test-cluster-child-index-dgram.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
// deno-fmt-ignore-file
// deno-lint-ignore-file

// Copyright Joyent and Node contributors. All rights reserved. MIT license.
// Taken from Node 18.12.1
// This file is automatically generated by "node/_tools/setup.ts". Do not modify this file manually

'use strict';
const common = require('../common');
const Countdown = require('../common/countdown');
if (common.isWindows)
common.skip('dgram clustering is currently not supported on Windows.');

const cluster = require('cluster');
const dgram = require('dgram');

// Test an edge case when using `cluster` and `dgram.Socket.bind()`
// the port of `0`.
const kPort = 0;

function child() {
const kTime = 2;
const countdown = new Countdown(kTime * 2, () => {
process.exit(0);
});
for (let i = 0; i < kTime; i += 1) {
const socket = new dgram.Socket('udp4');
socket.bind(kPort, common.mustCall(() => {
// `process.nextTick()` or `socket2.close()` would throw
// ERR_SOCKET_DGRAM_NOT_RUNNING
process.nextTick(() => {
socket.close(countdown.dec());
const socket2 = new dgram.Socket('udp4');
socket2.bind(kPort, common.mustCall(() => {
process.nextTick(() => {
socket2.close(countdown.dec());
});
}));
});
}));
}
}

if (cluster.isMaster)
cluster.fork(__filename);
else
child();
38 changes: 38 additions & 0 deletions node/_tools/test/parallel/test-cluster-child-index-net.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
// deno-fmt-ignore-file
// deno-lint-ignore-file

// Copyright Joyent and Node contributors. All rights reserved. MIT license.
// Taken from Node 18.12.1
// This file is automatically generated by "node/_tools/setup.ts". Do not modify this file manually

'use strict';
const common = require('../common');
const Countdown = require('../common/countdown');
const cluster = require('cluster');
const net = require('net');

// Test an edge case when using `cluster` and `net.Server.listen()` to
// the port of `0`.
const kPort = 0;

function child() {
const kTime = 2;
const countdown = new Countdown(kTime * 2, () => {
process.exit(0);
});
for (let i = 0; i < kTime; i += 1) {
const server = net.createServer();
server.listen(kPort, common.mustCall(() => {
server.close(countdown.dec());
const server2 = net.createServer();
server2.listen(kPort, common.mustCall(() => {
server2.close(countdown.dec());
}));
}));
}
}

if (cluster.isMaster)
cluster.fork(__filename);
else
child();
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
// deno-fmt-ignore-file
// deno-lint-ignore-file

// Copyright Joyent and Node contributors. All rights reserved. MIT license.
// Taken from Node 18.12.1
// This file is automatically generated by "node/_tools/setup.ts". Do not modify this file manually

// Copyright Joyent, Inc. and other Node contributors.
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the
// "Software"), to deal in the Software without restriction, including
// without limitation the rights to use, copy, modify, merge, publish,
// distribute, sublicense, and/or sell copies of the Software, and to permit
// persons to whom the Software is furnished to do so, subject to the
// following conditions:
//
// The above copyright notice and this permission notice shall be included
// in all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
// OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
// MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN
// NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
// DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
// OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
// USE OR OTHER DEALINGS IN THE SOFTWARE.

'use strict';
const common = require('../common');
const assert = require('assert');
const cluster = require('cluster');
const fork = cluster.fork;

if (cluster.isPrimary) {
fork(); // It is intentionally called `fork` instead of
fork(); // `cluster.fork` to test that `this` is not used
cluster.disconnect(common.mustCall(() => {
assert.deepStrictEqual(Object.keys(cluster.workers), []);
}));
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
// deno-fmt-ignore-file
// deno-lint-ignore-file

// Copyright Joyent and Node contributors. All rights reserved. MIT license.
// Taken from Node 18.12.1
// This file is automatically generated by "node/_tools/setup.ts". Do not modify this file manually

// Copyright Joyent, Inc. and other Node contributors.
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the
// "Software"), to deal in the Software without restriction, including
// without limitation the rights to use, copy, modify, merge, publish,
// distribute, sublicense, and/or sell copies of the Software, and to permit
// persons to whom the Software is furnished to do so, subject to the
// following conditions:
//
// The above copyright notice and this permission notice shall be included
// in all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
// OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
// MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN
// NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
// DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
// OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
// USE OR OTHER DEALINGS IN THE SOFTWARE.

'use strict';
require('../common');
process.env.NODE_CLUSTER_SCHED_POLICY = 'none';

const cluster = require('cluster');
const net = require('net');

if (cluster.isPrimary) {
const unbound = cluster.fork().on('online', bind);

function bind() {
cluster.fork({ BOUND: 'y' }).on('listening', disconnect);
}

function disconnect() {
unbound.disconnect();
unbound.on('disconnect', cluster.disconnect);
}
} else if (process.env.BOUND === 'y') {
const source = net.createServer();

source.listen(0);
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
// deno-fmt-ignore-file
// deno-lint-ignore-file

// Copyright Joyent and Node contributors. All rights reserved. MIT license.
// Taken from Node 18.12.1
// This file is automatically generated by "node/_tools/setup.ts". Do not modify this file manually

// Copyright Joyent, Inc. and other Node contributors.
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the
// "Software"), to deal in the Software without restriction, including
// without limitation the rights to use, copy, modify, merge, publish,
// distribute, sublicense, and/or sell copies of the Software, and to permit
// persons to whom the Software is furnished to do so, subject to the
// following conditions:
//
// The above copyright notice and this permission notice shall be included
// in all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
// OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
// MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN
// NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
// DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
// OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
// USE OR OTHER DEALINGS IN THE SOFTWARE.

'use strict';

const common = require('../common');

if (common.isWindows)
common.skip('on windows, because clustered dgram is ENOTSUP');

const cluster = require('cluster');
const dgram = require('dgram');

if (cluster.isPrimary) {
const unbound = cluster.fork().on('online', bind);

function bind() {
cluster.fork({ BOUND: 'y' }).on('listening', disconnect);
}

function disconnect() {
unbound.disconnect();
unbound.on('disconnect', cluster.disconnect);
}
} else if (process.env.BOUND === 'y') {
const source = dgram.createSocket('udp4');

source.bind(0);
}
Loading