Can't install on Windows using an Administrator user

[0xdevalias]

This comes from an issue a user opened on the humanify repo:

• install problem jehna/humanify#135

And after some debugging/exploration, it seems it is related to node-llama-cpp:

• install problem jehna/humanify#135 (comment)

I think it might be a permissions/antivirus/similar sort of a problem, but I haven't used windows as a dev machine in MANY years, so I'm not too sure of what the 'common pitfalls' are with using node on it these days.

I figured I would open a discussion here as well, so that at the very least, if someone else runs into similar with node-llamma-cpp in future, at least these errors will show up when searching.

I believe this is the main part of the 'breaking errors':

npm error Failed to load a prebuilt binary for platform "win" "x64", falling back to building from source. Error: Error: The specified module could not be found.
npm error \\?\C:\Users\Administrator\AppData\Roaming\npm\node_modules\humanifyjs\node_modules\@node-llama-cpp\win-x64\bins\win-x64\llama-addon.node
npm error     at Module._extensions..node (node:internal/modules/cjs/loader:1586:18)
npm error     at Module.load (node:internal/modules/cjs/loader:1288:32)
npm error     at Module._load (node:internal/modules/cjs/loader:1104:12)
npm error     at Module.require (node:internal/modules/cjs/loader:1311:19)
npm error     at require (node:internal/modules/helpers:179:18)
npm error     at loadBindingModule (file:///C:/Users/Administrator/AppData/Roaming/npm/node_modules/humanifyjs/node_modules/node-llama-cpp/dist/bindings/getLlama.js:390:25)
npm error     at loadExistingLlamaBinary (file:///C:/Users/Administrator/AppData/Roaming/npm/node_modules/humanifyjs/node_modules/node-llama-cpp/dist/bindings/getLlama.js:276:37)
npm error     at async getLlamaForOptions (file:///C:/Users/Administrator/AppData/Roaming/npm/node_modules/humanifyjs/node_modules/node-llama-cpp/dist/bindings/getLlama.js:133:27)
npm error     at async Object.handler (file:///C:/Users/Administrator/AppData/Roaming/npm/node_modules/humanifyjs/node_modules/node-llama-cpp/dist/cli/commands/OnPostInstallCommand.js:12:13) {
npm error   code: 'ERR_DLOPEN_FAILED'
npm error }

Though this part also may help indicate aspects of the issue:

npm warn cleanup   [
npm warn cleanup     'C:\\Users\\Administrator\\AppData\\Roaming\\npm\\node_modules\\humanifyjs\\node_modules',
npm warn cleanup     [Error: EPERM: operation not permitted, rmdir 'C:\Users\Administrator\AppData\Roaming\npm\node_modules\humanifyjs\node_modules\isolated-vm\src'] {
npm warn cleanup       errno: -4048,
npm warn cleanup       code: 'EPERM',
npm warn cleanup       syscall: 'rmdir',
npm warn cleanup       path: 'C:\\Users\\Administrator\\AppData\\Roaming\\npm\\node_modules\\humanifyjs\\node_modules\\isolated-vm\\src'
npm warn cleanup     }
npm warn cleanup   ],
npm warn cleanup   [
npm warn cleanup     'C:\\Users\\Administrator\\AppData\\Roaming\\npm\\node_modules\\humanifyjs\\node_modules\\node-llama-cpp',
npm warn cleanup     [Error: EPERM: operation not permitted, rmdir 'C:\Users\Administrator\AppData\Roaming\npm\node_modules\humanifyjs\node_modules\node-llama-cpp\llama\llama.cpp'] {
npm warn cleanup       errno: -4048,
npm warn cleanup       code: 'EPERM',
npm warn cleanup       syscall: 'rmdir',
npm warn cleanup       path: 'C:\\Users\\Administrator\\AppData\\Roaming\\npm\\node_modules\\humanifyjs\\node_modules\\node-llama-cpp\\llama\\llama.cpp'
npm warn cleanup     }
npm warn cleanup   ],
npm warn cleanup   [
npm warn cleanup     'C:\\Users\\Administrator\\AppData\\Roaming\\npm\\node_modules\\humanifyjs',
npm warn cleanup     [Error: EPERM: operation not permitted, rmdir 'C:\Users\Administrator\AppData\Roaming\npm\node_modules\humanifyjs'] {
npm warn cleanup       errno: -4048,
npm warn cleanup       code: 'EPERM',
npm warn cleanup       syscall: 'rmdir',
npm warn cleanup       path: 'C:\\Users\\Administrator\\AppData\\Roaming\\npm\\node_modules\\humanifyjs'
npm warn cleanup     }
npm warn cleanup   ]
npm warn cleanup ]

You can see my full deep dive into tracing the code/source of the error/musing on the potential reasons in this comment:

• install problem jehna/humanify#135 (comment)

But this was sort of my summary conclusion based on what I had seen so far:

..snip..

Within that, we can see there is a call to loadExistingLlamaBinary:

• node-llama-cpp/src/bindings/getLlama.ts

  Lines 367 to 387 in 4ee10a9

  	const llama = await loadExistingLlamaBinary({
  	buildOptions,
  	canUsePrebuiltBinaries,
  	logLevel,
  	logger,
  	existingPrebuiltBinaryMustMatchBuildOptions,
  	progressLogs,
  	platform,
  	platformInfo,
  	skipLlamaInit,
  	maxThreads,
  	vramPadding,
  	fallbackMessage: !isLastItem
  	? `falling back to using ${getPrettyBuildGpuName(buildGpusToTry[i + 1])}`
  	: (
  	canBuild
  	? "falling back to building from source"
  	: null
  	),
  	debug
  	});

• node-llama-cpp/src/bindings/getLlama.ts

  Lines 464 to 623 in 4ee10a9

  	async function loadExistingLlamaBinary({
  	buildOptions,
  	canUsePrebuiltBinaries,
  	logLevel,
  	logger,
  	existingPrebuiltBinaryMustMatchBuildOptions,
  	progressLogs,
  	platform,
  	platformInfo,
  	skipLlamaInit,
  	maxThreads,
  	vramPadding,
  	fallbackMessage,
  	debug
  	}: {
  	buildOptions: BuildOptions,
  	canUsePrebuiltBinaries: boolean,
  	logLevel: Required<LlamaOptions>["logLevel"],
  	logger: Required<LlamaOptions>["logger"],
  	existingPrebuiltBinaryMustMatchBuildOptions: boolean,
  	progressLogs: boolean,
  	platform: BinaryPlatform,
  	platformInfo: BinaryPlatformInfo,
  	skipLlamaInit: boolean,
  	maxThreads: number | undefined,
  	vramPadding: Required<LlamaOptions>["vramPadding"],
  	fallbackMessage: string | null,
  	debug: boolean
  	}) {
  	const buildFolderName = await getBuildFolderNameForBuildOptions(buildOptions);
  	const localBuildFolder = path.join(llamaLocalBuildBinsDirectory, buildFolderName.withCustomCmakeOptions);
  	const localBuildBinPath = await getLocalBuildBinaryPath(buildFolderName.withCustomCmakeOptions);
  	await waitForLockfileRelease({resourcePath: localBuildFolder});
  	if (localBuildBinPath != null) {
  	try {
  	const buildMetadata = await getLocalBuildBinaryBuildMetadata(buildFolderName.withCustomCmakeOptions);
  	const shouldTestBinaryBeforeLoading = getShouldTestBinaryBeforeLoading({
  	isPrebuiltBinary: false,
  	platform,
  	platformInfo,
  	buildMetadata
  	});
  	const binaryCompatible = shouldTestBinaryBeforeLoading
  	? await testBindingBinary(localBuildBinPath)
  	: true;
  	if (binaryCompatible) {
  	const binding = loadBindingModule(localBuildBinPath);
  	return await Llama._create({
  	bindings: binding,
  	buildType: "localBuild",
  	buildMetadata,
  	logLevel,
  	logger,
  	maxThreads,
  	vramPadding,
  	skipLlamaInit,
  	debug
  	});
  	} else if (progressLogs) {
  	console.warn(
  	getConsoleLogPrefix() + "The local build binary was not built in the current system and is incompatible with it"
  	);
  	if (canUsePrebuiltBinaries)
  	console.info(getConsoleLogPrefix() + "Falling back to prebuilt binaries");
  	else if (fallbackMessage != null)
  	console.info(getConsoleLogPrefix() + fallbackMessage);
  	}
  	} catch (err) {
  	const binaryDescription = describeBinary(buildOptions);
  	console.error(getConsoleLogPrefix() + `Failed to load a local build ${binaryDescription}. Error:`, err);
  	if (canUsePrebuiltBinaries)
  	console.info(getConsoleLogPrefix() + "Falling back to prebuilt binaries");
  	else if (fallbackMessage != null)
  	console.info(getConsoleLogPrefix() + fallbackMessage);
  	}
  	}
  	if (canUsePrebuiltBinaries) {
  	const prebuiltBinDetails = await getPrebuiltBinaryPath(
  	buildOptions,
  	existingPrebuiltBinaryMustMatchBuildOptions
  	? buildFolderName.withCustomCmakeOptions
  	: buildFolderName.withoutCustomCmakeOptions
  	);
  	if (prebuiltBinDetails != null) {
  	try {
  	const buildMetadata = await getPrebuiltBinaryBuildMetadata(prebuiltBinDetails.folderPath, prebuiltBinDetails.folderName);
  	const shouldTestBinaryBeforeLoading = getShouldTestBinaryBeforeLoading({
  	isPrebuiltBinary: true,
  	platform,
  	platformInfo,
  	buildMetadata
  	});
  	const binaryCompatible = shouldTestBinaryBeforeLoading
  	? await testBindingBinary(prebuiltBinDetails.binaryPath)
  	: true;
  	if (binaryCompatible) {
  	const binding = loadBindingModule(prebuiltBinDetails.binaryPath);
  	return await Llama._create({
  	bindings: binding,
  	buildType: "prebuilt",
  	buildMetadata,
  	logLevel,
  	logger,
  	maxThreads,
  	vramPadding,
  	skipLlamaInit,
  	debug
  	});
  	} else if (progressLogs) {
  	const binaryDescription = describeBinary({
  	...buildOptions,
  	customCmakeOptions: existingPrebuiltBinaryMustMatchBuildOptions
  	? buildOptions.customCmakeOptions
  	: new Map()
  	});
  	console.warn(
  	getConsoleLogPrefix() +
  	`The prebuilt ${binaryDescription} is not compatible with the current system` + (
  	fallbackMessage != null
  	? ", " + fallbackMessage
  	: ""
  	)
  	);
  	}
  	} catch (err) {
  	const binaryDescription = describeBinary({
  	...buildOptions,
  	customCmakeOptions: existingPrebuiltBinaryMustMatchBuildOptions
  	? buildOptions.customCmakeOptions
  	: new Map()
  	});
  	console.error(
  	getConsoleLogPrefix() + `Failed to load a prebuilt ${binaryDescription}` + (
  	fallbackMessage != null
  	? ", " + fallbackMessage
  	: ""
  	) + ". Error:", err);
  	}
  	} else if (progressLogs)
  	console.warn(
  	getConsoleLogPrefix() + "A prebuilt binary was not found" + (
  	fallbackMessage != null
  	? ", " + fallbackMessage
  	: ""
  	)
  	);
  	}
  	return null;
  	}

That calls getPrebuiltBinaryPath, which then calls getPrebuiltBinariesPackageDirectoryForBuildOptions, which is what handles attempting to import the various prebuilt binary packages:

• node-llama-cpp/src/bindings/getLlama.ts

  Lines 547 to 553 in 4ee10a9

  	if (canUsePrebuiltBinaries) {
  	const prebuiltBinDetails = await getPrebuiltBinaryPath(
  	buildOptions,
  	existingPrebuiltBinaryMustMatchBuildOptions
  	? buildFolderName.withCustomCmakeOptions
  	: buildFolderName.withoutCustomCmakeOptions
  	);

• getPrebuiltBinaryPath:

  node-llama-cpp/src/bindings/utils/compileLLamaCpp.ts

  Lines 286 to 313 in 4ee10a9

  	export async function getPrebuiltBinaryPath(buildOptions: BuildOptions, folderName: string) {
  	const localPrebuiltBinaryDirectoryPath = path.join(llamaPrebuiltBinsDirectory, folderName);
  	const binaryPath = await resolvePrebuiltBinaryPath(localPrebuiltBinaryDirectoryPath);
  	if (binaryPath != null)
  	return {
  	binaryPath,
  	folderName,
  	folderPath: localPrebuiltBinaryDirectoryPath
  	};
  	const packagePrebuiltBinariesDirectoryPath = await getPrebuiltBinariesPackageDirectoryForBuildOptions(buildOptions);
  	if (packagePrebuiltBinariesDirectoryPath == null)
  	return null;
  	const packagePrebuiltBinaryDirectoryPath = path.join(packagePrebuiltBinariesDirectoryPath, folderName);
  	const binaryPathFromPackage = await resolvePrebuiltBinaryPath(packagePrebuiltBinaryDirectoryPath);
  	if (binaryPathFromPackage != null)
  	return {
  	binaryPath: binaryPathFromPackage,
  	folderName,
  	folderPath: packagePrebuiltBinaryDirectoryPath
  	};
  	return null;
  	}

• getPrebuiltBinariesPackageDirectoryForBuildOptions:

  node-llama-cpp/src/bindings/utils/compileLLamaCpp.ts

  Lines 364 to 426 in 4ee10a9

  	function getPrebuiltBinariesPackageDirectoryForBuildOptions(buildOptions: BuildOptions) {
  	async function getBinariesPathFromModules(moduleImport: () => Promise<{getBinsDir(): {binsDir: string, packageVersion: string}}>) {
  	try {
  	const [
  	binariesModule,
  	currentModuleVersion
  	] = await Promise.all([
  	moduleImport(),
  	getModuleVersion()
  	]);
  	const {binsDir, packageVersion} = binariesModule?.getBinsDir?.() ?? {};
  	if (binsDir == null || packageVersion !== currentModuleVersion)
  	return null;
  	return binsDir;
  	} catch (err) {
  	return null;
  	}
  	}
  	if (buildOptions.platform === "mac") {
  	if (buildOptions.arch === "arm64" && buildOptions.gpu === "metal")
  	// @ts-ignore
  	return getBinariesPathFromModules(() => import("@node-llama-cpp/mac-arm64-metal"));
  	else if (buildOptions.arch === "x64" && buildOptions.gpu === false)
  	// @ts-ignore
  	return getBinariesPathFromModules(() => import("@node-llama-cpp/mac-x64"));
  	} else if (buildOptions.platform === "linux") {
  	if (buildOptions.arch === "x64") {
  	if (buildOptions.gpu === "cuda")
  	// @ts-ignore
  	return getBinariesPathFromModules(() => import("@node-llama-cpp/linux-x64-cuda"));
  	else if (buildOptions.gpu === "vulkan")
  	// @ts-ignore
  	return getBinariesPathFromModules(() => import("@node-llama-cpp/linux-x64-vulkan"));
  	else if (buildOptions.gpu === false)
  	// @ts-ignore
  	return getBinariesPathFromModules(() => import("@node-llama-cpp/linux-x64"));
  	} else if (buildOptions.arch === "arm64")
  	// @ts-ignore
  	return getBinariesPathFromModules(() => import("@node-llama-cpp/linux-arm64"));
  	else if (buildOptions.arch === "arm")
  	// @ts-ignore
  	return getBinariesPathFromModules(() => import("@node-llama-cpp/linux-armv7l"));
  	} else if (buildOptions.platform === "win") {
  	if (buildOptions.arch === "x64") {
  	if (buildOptions.gpu === "cuda")
  	// @ts-ignore
  	return getBinariesPathFromModules(() => import("@node-llama-cpp/win-x64-cuda"));
  	else if (buildOptions.gpu === "vulkan")
  	// @ts-ignore
  	return getBinariesPathFromModules(() => import("@node-llama-cpp/win-x64-vulkan"));
  	else if (buildOptions.gpu === false)
  	// @ts-ignore
  	return getBinariesPathFromModules(() => import("@node-llama-cpp/win-x64"));
  	} else if (buildOptions.arch === "arm64")
  	// @ts-ignore
  	return getBinariesPathFromModules(() => import("@node-llama-cpp/win-arm64"));
  	}
  	return null;
  	}

    } else if (buildOptions.platform === "win") {
        if (buildOptions.arch === "x64") {
            if (buildOptions.gpu === "cuda")
                // @ts-ignore
                return getBinariesPathFromModules(() => import("@node-llama-cpp/win-x64-cuda"));
            else if (buildOptions.gpu === "vulkan")
                // @ts-ignore
                return getBinariesPathFromModules(() => import("@node-llama-cpp/win-x64-vulkan"));
            else if (buildOptions.gpu === false)
                // @ts-ignore
                return getBinariesPathFromModules(() => import("@node-llama-cpp/win-x64"));
        } else if (buildOptions.arch === "arm64")
            // @ts-ignore
            return getBinariesPathFromModules(() => import("@node-llama-cpp/win-arm64"));
    }

This then calls getBinariesPathFromModules which seems to do some stuff to find the bins dir:

• node-llama-cpp/src/bindings/utils/compileLLamaCpp.ts

  Lines 365 to 383 in 4ee10a9

  	async function getBinariesPathFromModules(moduleImport: () => Promise<{getBinsDir(): {binsDir: string, packageVersion: string}}>) {
  	try {
  	const [
  	binariesModule,
  	currentModuleVersion
  	] = await Promise.all([
  	moduleImport(),
  	getModuleVersion()
  	]);
  	const {binsDir, packageVersion} = binariesModule?.getBinsDir?.() ?? {};
  	if (binsDir == null || packageVersion !== currentModuleVersion)
  	return null;
  	return binsDir;
  	} catch (err) {
  	return null;
  	}
  	}

Jumping back out to loadExistingLlamaBinary, we can see the 'failed to load a prebuilt binary' message here within a catch block:

• node-llama-cpp/src/bindings/getLlama.ts

  Lines 598 to 611 in 4ee10a9

  	} catch (err) {
  	const binaryDescription = describeBinary({
  	...buildOptions,
  	customCmakeOptions: existingPrebuiltBinaryMustMatchBuildOptions
  	? buildOptions.customCmakeOptions
  	: new Map()
  	});
  	console.error(
  	getConsoleLogPrefix() + `Failed to load a prebuilt ${binaryDescription}` + (
  	fallbackMessage != null
  	? ", " + fallbackMessage
  	: ""
  	) + ". Error:", err);
  	}

The actual triggering error is appended to the end of that message, so the root issue appears to be this part:

Error: Error: The specified module could not be found.
npm error \\?\C:\Users\Administrator\AppData\Roaming\npm\node_modules\humanifyjs\node_modules\@node-llama-cpp\win-x64\bins\win-x64\llama-addon.node
npm error     at Module._extensions..node (node:internal/modules/cjs/loader:1586:18)
npm error     at Module.load (node:internal/modules/cjs/loader:1288:32)
npm error     at Module._load (node:internal/modules/cjs/loader:1104:12)
npm error     at Module.require (node:internal/modules/cjs/loader:1311:19)
npm error     at require (node:internal/modules/helpers:179:18)
npm error     at loadBindingModule (file:///C:/Users/Administrator/AppData/Roaming/npm/node_modules/humanifyjs/node_modules/node-llama-cpp/dist/bindings/getLlama.js:390:25)
npm error     at loadExistingLlamaBinary (file:///C:/Users/Administrator/AppData/Roaming/npm/node_modules/humanifyjs/node_modules/node-llama-cpp/dist/bindings/getLlama.js:276:37)
npm error     at async getLlamaForOptions (file:///C:/Users/Administrator/AppData/Roaming/npm/node_modules/humanifyjs/node_modules/node-llama-cpp/dist/bindings/getLlama.js:133:27)
npm error     at async Object.handler (file:///C:/Users/Administrator/AppData/Roaming/npm/node_modules/humanifyjs/node_modules/node-llama-cpp/dist/cli/commands/OnPostInstallCommand.js:12:13) {
npm error   code: 'ERR_DLOPEN_FAILED'
npm error }

@Yusufkulcu So I guess the first thing I would be doing is checking whether that file actually exists, and if there is anything that might be blocking it from being able to be loaded (permissions, antivirus, etc):

• C:\Users\Administrator\AppData\Roaming\npm\node_modules\humanifyjs\node_modules\@node-llama-cpp\win-x64\bins\win-x64\llama-addon.node

If that doesn't help, you may be able to get some more specific ideas/support on the GitHub Discussions page for node-llama-cpp:

• https://github.com/withcatai/node-llama-cpp/discussions/categories/general

Originally posted by @0xdevalias in jehna/humanify#135 (comment)

[giladgd]

@0xdevalias I run some tests on the latest version (3.0.3), and it seems to work as expected on Windows.
I've tested it on both a Windows 10 machine and on a Windows 11 machine.

From the logs you attached, I think there's something wrong in the installation on the user machine, whether it's npm that hasn't installed the prebuilt binaries properly (maybe they're using some npm mirror that doesn't work as expected), maybe it's related to permissions on the machine itself (like running npm install in the folder from one user and trying to use it from another use that doesn't have proper access to the node_modules directory), and maybe it has something to do with some program on the machine the blocks the access to the .node file.

From the linked issue I can see in the logs that it attempted to fallback to building from source, and since CMake is not installed on the user's machine, it tried to call xpm to install CMake locally inside node-llama-cpp, but it failed with this error:

npm error ^[[?25l^[[?25h'xpm' is not recognized as an internal or external command,
npm error operable program or batch file.
npm error Failed to build llama.cpp with no GPU support. Error: SpawnError: Command npm exec --yes -- xpm@^0.16.3 install @xpack-dev-tools/cmake@latest --no-save exited with code 1

The ^[[?25l^[[?25h'xpm' part seems to indicate one of the following:

• There's and issue with the installation of nodejs/npm on the user machine.

  npm exec --yes is equivalent to npx -y, but npm exec --yes works better with other runtimes such as Bun, which is why I used it.

• There's an issue with the escaping done by cross-spawn to run the npm exec --yes -- xpm@^0.16.3 command. I used cross-spawn because it solved many issues with process spawning on Windows, so I think this is less probable.

What I would suggest to try to fix this issue would be:

• Uninstall all global npm modules, including npm, and reinstalling nodejs, since it seems like there's an issue with its installation.
• Try to use Bun, since it seems they've fixed many Windows compatibility issues.
• Install CMake manually, so node-llama-cpp can use it.

Before doing all of the above, it would be helpful if you can check if C:\Users\Administrator\AppData\Roaming\npm\node_modules\humanifyjs\node_modules\@node-llama-cpp\win-x64\bins\win-x64\llama-addon.node exists.

[giladgd]

Actually, this may indeed be the permission issue I mentioned.
The attached error shows that node-llama-cpp is installed under the Administrator user, which should never be done, and is very prone to permission issues.

[0xdevalias]

@giladgd Thanks for testing + your notes. I'm unable to test these directly myself as it was another user who originally had the issue, but I have copied your notes above back onto the original thread, so hopefully we'll hear back from them about it:

• install problem jehna/humanify#135 (comment)