Error: Command failed: Executing pkg cli from the child process of pkg cli getting failed.

[lokrajahuja]

We have a requirement to call the shell script from pkg CLI and that script is again calling the same pkg CLI.

Our CLI is calling the shell script using the child_process package of the node. In a shell script, we are again calling the Pkg CLI if some condition is met.

Following is the testing Code: test.js

const doSomething1 = () => {
  const childProcess = require('child_process');
  console.log('Do Something 1 called');
  // calling some shell script. and in script we have some condition if it is true then call doSomething2 from executables.
  childProcess.execSync('./pkgBinary 2');
}

const doSomething2 = () => {
  console.log('Do Somthing 2 called');
}

if (process.argv[2] === '1') {
  doSomething1();
} else {
  doSomething2();
}

Created binary using

pkg test.js -t 'node12-macos-x64' -o 'pkgBinary'

after the execution of command './pkgBinary 1'. I am getting the following error.

Error: Command failed: ./pkgBinary 2
internal/modules/cjs/loader.js:983
throw err;
^

Error: Cannot find module '/Users/appirio-13532/practice/2'
at Function.Module._resolveFilename (internal/modules/cjs/loader.js:980:15)
at Function.Module._resolveFilename (pkg/prelude/bootstrap.js:1346:46)
at Function.Module._load (internal/modules/cjs/loader.js:862:27)
at Function.Module.runMain (pkg/prelude/bootstrap.js:1375:12)
at internal/main/run_main_module.js:18:47 {
code: 'MODULE_NOT_FOUND',
requireStack: []
}

at checkExecSyncError (child_process.js:630:11)
at Object.execSync (child_process.js:666:15)
at Object.childProcess.execSync (pkg/prelude/bootstrap.js:1507:30)
at doSomething1 (/snapshot/practice/test.js)
at Object.<anonymous> (/snapshot/practice/test.js)
at Module._compile (pkg/prelude/bootstrap.js:1320:22)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1176:10)
at Module.load (internal/modules/cjs/loader.js:1000:32)
at Function.Module._load (internal/modules/cjs/loader.js:899:14)
at Function.Module.runMain (pkg/prelude/bootstrap.js:1375:12) {

status: 1,
signal: null,
output: [
null,
,
<Buffer 69 6e 74 65 72 6e 61 6c 2f 6d 6f 64 75 6c 65 73 2f 63 6a 73 2f 6c 6f 61 64 65 72 2e 6a 73 3a 39 38 33 0a 20 20 74 68 72 6f 77 20 65 72 72 3b 0a 20 20 ... 450 more bytes>
],
pid: 14642,
stdout: ,
stderr: <Buffer 69 6e 74 65 72 6e 61 6c 2f 6d 6f 64 75 6c 65 73 2f 63 6a 73 2f 6c 6f 61 64 65 72 2e 6a 73 3a 39 38 33 0a 20 20 74 68 72 6f 77 20 65 72 72 3b 0a 20 20 ... 450 more bytes>
}

[igorklopov]

As a first step towards solving your issue, please switch to using spawn/spawnSync and process.execPath because it is tested to work this way https://github.com/zeit/pkg/blob/64c40421396fdeaa3607037731962f7fca5365c3/prelude/bootstrap.js#L1451.

[krische]

Have you had any luck with this issue? I think I am running into the same problem.

[stripedpajamas]

Having exactly the same issue (same stack trace); @igorklopov what are you suggesting? If I want my pkg'd app to spawn an instance of itself, how should it be done?

I've tried these and they both throw MODULE_NOT_FOUND looking for arg1:

spawn('myappname', ['arg1']) // Error: Cannot find module '/path/to/my/app/arg1'
spawn(process.execPath, ['arg1']) // Error: Cannot find module '/path/to/my/app/arg1'

I've been able to solve my specific use-case by spawning a local module within my package where I've put the functionality:

spawn(process.execPath, [require.resolve('./arg1-handler')])

But the main issue remains; I think I should be able to spawn the app itself with args. Is this supported?

[eGavr]

I am running into the same problem, any news?

[AndoGhevian]

I had the same issue, and find some interesting behavior that is not documented for pkg - when you calling packed executable from outside, manually, it acts like if you call node with second argument set to internal main script ( /snapshot/project/main.js ). You can check it with console.log(process.argv) in "main.js", but you can't override it, like if it was a js function with binded first argument. But if you trying to spawn same executable recursively ( from inside that executable ), it will not be provided with second argument, which must be path to js file. so you can pass as a second argument another js file, which is responsible for your second logic branch. I create an example repo for OpenTerm I wrote, which is use same technique that i describe. - https://github.com/AndoGhevian/VT-pkg

[github-actions]

This issue is stale because it has been open 90 days with no activity. Remove the stale label or comment or this will be closed in 5 days. To ignore this issue entirely you can add the no-stale label

[github-actions]

This issue is now closed due to inactivity, you can of course reopen or reference this issue if you see fit.

[zkochan]

I don't think this issue is solved. I am trying to bundle pnpm with pkg and getting this errors when there are pnpm run commands in lifecycle scripts.

[eysi09]

Just chiming to say we're seeing the same issue in a tool we're building. Here's the relevant bug report.

Has anyone found a workaround to this?

[zkochan]

I did this fork for pnpm with a few changes: https://github.com/zoli-forks/pkg

It seems to work fine for us.

[github-actions]

This issue is stale because it has been open 90 days with no activity. Remove the stale label or comment or this will be closed in 5 days. To ignore this issue entirely you can add the no-stale label

[github-actions]

This issue is now closed due to inactivity, you can of course reopen or reference this issue if you see fit.

[robertsLando]

@zkochan Would you mind to submit a PR?

[ryanblock]

I also ran into this issue, but specifically because pkg mutates child_process calls, overwriting node with its own execution path: https://github.com/vercel/pkg/blob/main/prelude/bootstrap.js#L1874-L1894 (specifically L1880). Our app needs to execute Node, not itself.

I consider global mutation bad behavior that introduces unexpected side-effects to userland. Is this absolutely necessary? Can we at least have an opt-out?

Happy to PR if you'd like to provide some guidance!

[robertsLando]

I think it could be opted out using an env var? @jesec

[ryanblock]

I'd be fine with an env var or cli flag, whatever works for y'all.

[ryanblock]

Just checking back again: would a PR be welcome?

[zkochan]

I have an open pull request that fixes this issue: #1344

[edvald]

FYI, the way we've solved this for ourselves is this patch, applied with https://www.npmjs.com/package/patch-package:

diff --git a/node_modules/pkg/prelude/bootstrap.js b/node_modules/pkg/prelude/bootstrap.js
index 7285b7c..27d3975 100644
--- a/node_modules/pkg/prelude/bootstrap.js
+++ b/node_modules/pkg/prelude/bootstrap.js
@@ -1870,7 +1870,7 @@ function payloadFileSync(pointer) {
     }
     const opts = args[pos];
     if (!opts.env) opts.env = _extend({}, process.env);
-    if (opts.env.PKG_EXECPATH === 'PKG_INVOKE_NODEJS') return;
+    if (opts.env.PKG_EXECPATH !== undefined) return;
     opts.env.PKG_EXECPATH = EXECPATH;
   }

And setting PKG_EXECPATH: "" when creating child processes that might call the pkg'd executable (in our case, generally some user script that calls the garden binary). It's not very pretty, and requires us to remember this where we run child processes that might call the binary, but it does work for us.

[0x2b3bfa0]

It looks like pkg is explicitly monkey–patching the child_process functions to inject/overwrite the PKG_EXECPATH environment variable; that's half of the issue. The other half is acting differently when that environment variable is set; see #1565.

pkg/prelude/bootstrap.js

Lines 1851 to 1980 in 2ef4cd9

	// /////////////////////////////////////////////////////////////////
	// PATCH CHILD_PROCESS /////////////////////////////////////////////
	// /////////////////////////////////////////////////////////////////
	(() => {
	const ancestor = {
	spawn: childProcess.spawn,
	spawnSync: childProcess.spawnSync,
	execFile: childProcess.execFile,
	execFileSync: childProcess.execFileSync,
	exec: childProcess.exec,
	execSync: childProcess.execSync,
	};
	function setOptsEnv(args) {
	let pos = args.length - 1;
	if (typeof args[pos] === 'function') pos -= 1;
	if (typeof args[pos] !== 'object' || Array.isArray(args[pos])) {
	pos += 1;
	args.splice(pos, 0, {});
	}
	const opts = args[pos];
	if (!opts.env) opts.env = _extend({}, process.env);
	if (opts.env.PKG_EXECPATH === 'PKG_INVOKE_NODEJS') return;
	opts.env.PKG_EXECPATH = EXECPATH;
	}
	function startsWith2(args, index, name, impostor) {
	const qsName = `"${name} `;
	if (args[index].slice(0, qsName.length) === qsName) {
	args[index] = `"${impostor} ${args[index].slice(qsName.length)}`;
	return true;
	}
	const sName = `${name} `;
	if (args[index].slice(0, sName.length) === sName) {
	args[index] = `${impostor} ${args[index].slice(sName.length)}`;
	return true;
	}
	if (args[index] === name) {
	args[index] = impostor;
	return true;
	}
	return false;
	}
	function startsWith(args, index, name) {
	const qName = `"${name}"`;
	const qEXECPATH = `"${EXECPATH}"`;
	const jsName = JSON.stringify(name);
	const jsEXECPATH = JSON.stringify(EXECPATH);
	return (
	startsWith2(args, index, name, EXECPATH) ||
	startsWith2(args, index, qName, qEXECPATH) ||
	startsWith2(args, index, jsName, jsEXECPATH)
	);
	}
	function modifyLong(args, index) {
	if (!args[index]) return;
	return (
	startsWith(args, index, 'node') ||
	startsWith(args, index, ARGV0) ||
	startsWith(args, index, ENTRYPOINT) ||
	startsWith(args, index, EXECPATH)
	);
	}
	function modifyShort(args) {
	if (!args[0]) return;
	if (!Array.isArray(args[1])) {
	args.splice(1, 0, []);
	}
	if (
	args[0] === 'node' ||
	args[0] === ARGV0 ||
	args[0] === ENTRYPOINT ||
	args[0] === EXECPATH
	) {
	args[0] = EXECPATH;
	} else {
	for (let i = 1; i < args[1].length; i += 1) {
	const mbc = args[1][i - 1];
	if (mbc === '-c' || mbc === '/c') {
	modifyLong(args[1], i);
	}
	}
	}
	}
	childProcess.spawn = function spawn() {
	const args = cloneArgs(arguments);
	setOptsEnv(args);
	modifyShort(args);
	return ancestor.spawn.apply(childProcess, args);
	};
	childProcess.spawnSync = function spawnSync() {
	const args = cloneArgs(arguments);
	setOptsEnv(args);
	modifyShort(args);
	return ancestor.spawnSync.apply(childProcess, args);
	};
	childProcess.execFile = function execFile() {
	const args = cloneArgs(arguments);
	setOptsEnv(args);
	modifyShort(args);
	return ancestor.execFile.apply(childProcess, args);
	};
	childProcess.execFileSync = function execFileSync() {
	const args = cloneArgs(arguments);
	setOptsEnv(args);
	modifyShort(args);
	return ancestor.execFileSync.apply(childProcess, args);
	};
	childProcess.exec = function exec() {
	const args = cloneArgs(arguments);
	setOptsEnv(args);
	modifyLong(args, 0);
	return ancestor.exec.apply(childProcess, args);
	};
	childProcess.execSync = function execSync() {
	const args = cloneArgs(arguments);
	setOptsEnv(args);
	modifyLong(args, 0);
	return ancestor.execSync.apply(childProcess, args);
	};
	})();

pkg/prelude/bootstrap.js

Line 1874 in 2ef4cd9

opts.env.PKG_EXECPATH = EXECPATH;

[github-actions]

This issue is stale because it has been open 90 days with no activity. Remove the stale label or comment or this will be closed in 5 days. To ignore this issue entirely you can add the no-stale label

[dacbd]

bump

[jesec]

There are basically two camps of users. One camp wants a pkg-generated executable to behave as the bundled application in all scenarios. The other camp wants the executable to behave as the Node.js runtime when they want it to.

It is very tricky to support both cases, and I am considering to declare that "the pkg-generated executable can't be used as Node.js runtime" in the next major release.

[github-actions]

This issue is stale because it has been open 90 days with no activity. Remove the stale label or comment or this will be closed in 5 days. To ignore this issue entirely you can add the no-stale label

[github-actions]

This issue is now closed due to inactivity, you can of course reopen or reference this issue if you see fit.

[koraniar]

Hello, I am having this same issue
I see in the code that the variable PKG_EXECPATH is only filled here where we patch the child_process functions:

pkg/prelude/bootstrap.js

Line 2010 in bb04269

opts.env.PKG_EXECPATH = EXECPATH;

So, in my understanding, it seems that this logic (where I am seeing the problem) is only true when we call the binary from the same binary using the child_process functions:

pkg/prelude/bootstrap.js

Lines 77 to 84 in bb04269

	if (process.env.PKG_EXECPATH === EXECPATH) {
	process.argv.splice(1, 1);
	if (process.argv[1] && process.argv[1] !== '-') {
	// https://github.com/nodejs/node/blob/1a96d83a223ff9f05f7d942fb84440d323f7b596/lib/internal/bootstrap/node.js#L269
	process.argv[1] = path.resolve(process.argv[1]);
	}
	} else {

But if I comment the above logic (that is a very similar approach @edvald mentioned here) it starts to work.

in which other case is the above logic called? this is to understand if the validation could be improved in some way or what could I lose if I comment that logic just for my case.
@jesec @robertsLando

[Avivbens]

Hi all 👋

Any update about this one?
I see @koraniar in on to something, but still there is no PR for a fix 🥲