[bug] 'moon' is not recognized as an internal or external command, operable program or batch file.

[FunkMonkey]

Describe the bug

The moon.cmd is not properly set up, when installing @moonrepo/cli version 1.24.4 on Windows with yarn. This may be related to this yarn bug. Unfortunately this makes moon mostly unusable for me.

Steps to reproduce

yarn install

# when running after the first yarn install, we get this error
# .bin/moon.cmd is set up, but is missing the .exe extension, which leads to moon somehow 
# being called wrongly. With .exe manually added, it works fine!
npx moon --help
error: unrecognized subcommand 'XXX\node_modules\.bin\\..\@moonrepo\cli\moon'

# after a second yarn install, the .bin/moon.cmd file disappears completely
yarn install
npx moon --help
npm ERR! could not determine executable to run

# or when running as a package.json script
npm run moon_help
'moon' is not recognized as an internal or external command, operable program or batch file.

Expected behavior

Moon binary should work.

Environment

  System:
    OS: Windows 11 10.0.22631
    CPU: (16) x64 AMD Ryzen 7 7700X 8-Core Processor
    Memory: 10.71 GB / 31.14 GB
  Binaries:
    Node: 20.11.0 - C:\Program Files\nodejs\node.EXE
    Yarn: 3.6.1 - C:\Program Files\nodejs\yarn.CMD
    npm: 10.2.4 - C:\Program Files\nodejs\npm.CMD
    pnpm: 8.15.6 - ~\AppData\Local\pnpm\pnpm.EXE
  Managers:
    Cargo: 1.75.0 - ~\.cargo\bin\cargo.EXE
    pip3: 23.2.1 - ~\AppData\Local\Programs\Python\Python312\Scripts\pip3.EXE
  Utilities:
    CMake: 3.27.9
    Git: 2.45.0.
    Curl: 8.4.0 - C:\Windows\system32\curl.EXE
  Virtualization:
    Docker: 25.0.3 - C:\Program Files\Docker\Docker\resources\bin\docker.EXE
  IDEs:
    VSCode: 1.89.1 - C:\Users\Andre\AppData\Local\Programs\Microsoft VS Code\bin\code.CMD
    Visual Studio: 17.8.34330.188 (Visual Studio Community 2022)
  Languages:
    Bash: 5.2.15 - C:\Windows\system32\bash.EXE
    Python: 3.12.1
    Rust: 1.75.0
  Browsers:
    Edge: Chromium (123.0.2420.97)
    Internet Explorer: 11.0.22621.3527
  Monorepos:
    Yarn Workspaces: 3.6.1

Thank you for your help!

[FunkMonkey]

Creating a stub binary like this comment suggests, might be an easy solution

[milesj]

So we actually do have a stub file: https://github.com/moonrepo/moon/blob/master/packages/cli/moon But maybe it needs to be a bash script?

I'm also not positive if npx runs post installs?

[FunkMonkey]

Thanks for the quick reply!

I'm also not positive if npx runs post installs?

Well npx does not, but yarn does. npx simply executes an executable (e.g. .cmd) in a node_modules/.bin directory, as if it was in the PATH. Basically similar to how package.json scripts work, only that you can use npx directly with an arbitrary command.

So we actually do have a stub file: https://github.com/moonrepo/moon/blob/master/packages/cli/moon But maybe it needs to be a bash script?

Maybe. Though the stub file apparently gets removed after the first installation. On my windows machine I only see the moon.exe in the node_modules/@moonrepo/cli directory. This explains why the second yarn install removes node_modules/.bin/moon.cmd, since the target of the bin (the stub file) does not exist anymore. It works fine on linux, since there neither moon stub nor binary have an extension.

I quickly looked how other packages, that use post-installed binary executables handle this (e.g. electron, esbuild, playwright-core). It seems they all use a node.js script that calls the binary via child_process, similar to the way it is explained here.

[FunkMonkey]

This is basically also what the comment that I linked above suggested. He didn't actually create an empty stub file, but a node.js script that calls the executable. Here is the file from the repo that he linked:

#!/usr/bin/env node

/**
 * This file is a pass-through for the actual just binary. It exists because for two reasons:
 *
 * - Yarn does not allow references to anything other than .js files in the "bin" field in package.json.
 * - Windows does not allow executing binaries that don't end in .exe, and we need the package.json "bin" field to
 *   point to the same file on all platforms.
 */

import child_process from 'node:child_process';
import path from 'node:path';
import process from 'node:process';
import url from "url";

const ext = process.platform === 'win32' ? '.exe' : '';

const __filename = url.fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);

child_process.execFileSync(
  path.resolve(__dirname, 'just' + ext),
  process.argv.slice(2),
  { stdio: 'inherit' }
);

[milesj]

Thanks, I'll dig further.

I'm still curious why this is only happening to you at the moment. We have a lot of Windows users, and Yarn users, and this hasn't crept up yet.

[FunkMonkey]

I'm still curious why this is only happening to you at the moment. We have a lot of Windows users, and Yarn users, and this hasn't crept up yet.

I was asking myself the same thing. Maybe it's because I am using Yarn 3/4 and not Yarn 1... Though even then, the combination of Windows and Yarn 2+ should be common enough...

[FunkMonkey]

Ok, I dug a little deeper and realized that this problem also happens with npm.

Even directly calling F:\path\to\yarn-moon\node_modules\@moonrepo\cli\moon --help will result in the subcommand error - but only in cmd.exe and not with powershell (where it opens a new shell for a second). Calling F:\path\to\yarn-moon\node_modules\@moonrepo\cli\moon.exe --help works just fine. Similarly going to node_modules\@moonrepo\cli and calling moon (without file extension) directly works fine!

Tests with moon.cmd

I tampered with the moon.cmd file created by yarn / npm a bit, which looks like this:

@"%~dp0\..\@moonrepo\cli\moon"   %*

Calling this command will produce the error in moon.exe (which is actually being called):

error: unrecognized subcommand 'F:\path\to\node_modules\.bin\\..\@moonrepo\cli\moon'

So it seems, moon.exe somehow interprets the path as its first argument.

When simply adding .exe, everything works fine again (just like in the shell):

@"%~dp0\..\@moonrepo\cli\moon.exe"   %*

Interestingly though, removing %~dp0\ works fine as well!

@"..\@moonrepo\cli\moon"   %*

I did some additional tests using a custom cmd file to print the arguments, because I thought there may be a bug in windows, where the arguments are passed wrongly for whatever reason, but my custom cmd always printed everything correctly independent of using / not using the file extension or %~dp0\.

Conclusion

My suspicion is: most other Windows users don't install moon itself via npm or yarn. That's why they simply have moon.exe in their PATH and everything just works fine. But it seems something is broken within moon in combination with windows that when calling moon using an absolute path and without an extension, it interprets args[0] (the program name) as args[1] (the first argument, here the subcommand).

Hope this helps

[FunkMonkey]

p.s. I also checked: moon.cmd being removed with the second yarn install only happens in Yarn 2+ and not in Yarn 1. So even if this weird bug with the argument is fixed, there still needs to be a fix for Yarn 2+ (probably just leaving the moon stub file in place instead of deleting it, should be enough though)

[milesj]

Thanks for debugging! .cmd always seems to crop up and cause issues :P

I think the easiest solution here is to just have a node script call a child process like you mentioned above. I can easily tackle this on Monday.

[FunkMonkey]

great, thanks!

Btw, I think one of the problematic code lines is this one in main.rs. This one will fail to remove the binary path on windows, if it does not end with .exe, which is why calling moon via F:\path\to\yarn-moon\node_modules\@moonrepo\cli\moon --help is problematic.

I still don't understand though why calling moon with a relative path, e.g. .\node_modules\@moonrepo\cli\moon --help, works just fine