git warns "could not open directory: Function not implemented" when encountering junctions with very long paths

[mcdurdin]

• I was not able to find an open or closed issue matching what I'm seeing.

Possibly vaguely related: #607, #1270?

Setup

• Which version of Git for Windows are you using? Is it 32-bit or 64-bit?

$ git --version --build-options

git version 2.27.0.windows.1
cpu: x86_64
built from commit: 907ab1011dce9112700498e034b974ba60f8b407
sizeof-long: 4
sizeof-size_t: 8

• Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?

$ cmd.exe /c ver

Microsoft Windows [Version 10.0.18363.836]

Windows 10 1909 64-bit.

• What options did you set as part of the installation? Or did you choose the
  defaults?

# One of the following:
> type "C:\Program Files\Git\etc\install-options.txt"
> type "C:\Program Files (x86)\Git\etc\install-options.txt"
> type "%USERPROFILE%\AppData\Local\Programs\Git\etc\install-options.txt"
$ cat /etc/install-options.txt

Editor Option: VisualStudioCode
Custom Editor Path:
Path Option: Cmd
SSH Option: OpenSSH
Tortoise Option: false
CURL Option: OpenSSL
CRLF Option: CRLFAlways
Bash Terminal Option: MinTTY
Git Pull Behavior Option: Merge
Performance Tweaks FSCache: Enabled
Use Credential Manager: Enabled
Enable Symlinks: Disabled
Enable Pseudo Console Support: Disabled

Note: the issue presents identically, whether or not core.symlinks is true or false.

• Any other interesting things about your environment that might be related
  to the issue you're seeing?

Using NPM with long paths and junctions (e.g. npm link).

Details

• Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

Either Bash or CMD. Not tested in PowerShell.

• What commands did you run to trigger this issue? If you can provide a
  Minimal, Complete, and Verifiable example
  this will help us understand the issue.

MCVE: https://github.com/mcdurdin/git-windows-junction-longpath-error

This relies on a small node script to generate the junction with a long path because mklink does not yet support long paths. This simulates the issue we are seeing with npm link ultimately calling fs.symlink(target, link, 'junction').

node sl.js
git status

sl.js contents:

let fs = require('fs');
const longpath = '1234567890_1234567890_1234567890_1234567890_1234567890_1234567890_1234567890_1234567890_1234567890_1234567890_1234567890';
const p1 = 'module_1_'+longpath, p2 = 'module_2_'+longpath;

if(!fs.existsSync(p1))
  fs.mkdirSync(p1);
if(!fs.existsSync(p2))
  fs.mkdirSync(p2);
//fs.writeFileSync(p1 + '/foo', 'foo');
//fs.writeFileSync(p2 + '/foo', 'foo');
fs.symlinkSync(p1, p2+'/'+p1, 'junction');

• What did you expect to occur after running these commands?

$ git status
On branch master
nothing to commit, working tree clean

• What actually happened instead?

$ git status
warning: could not open directory 'module_2_1234567890_1234567890_1234567890_1234567890_1234567890_1234567890_1234567890_1234567890_1234567890_1234567890_1234567890/module_1_1234567890_1234567890_1234567890_1234567890_1234567890_1234567890_1234567890_1234567890_1234567890_1234567890_1234567890/': Function not implemented
On branch master
nothing to commit, working tree clean

This same error crops up with a number of other commands, such as git clean -fdx or git add. We have also seen what may be a related issue (on another repo), where git warns, for example:

warning: could not open directory 'common/core/web/input-processor/node_modules/@keymanapp/keyboard-processor/node_modules/@keymanapp/resources-gosh/': No such file or directory
... repeated for many other paths ...

These paths contain nested junctions (no cycles as far as I can tell) and some of the full paths within those junctions hit the 260 character limit for traditional Windows paths. This however is proving much harder to reproduce in a standalone repository, so I refer to this mainly as it was the trigger for creating the MCVE.

• If the problem was occurring with a specific repository, can you provide the
  URL to that repository to help us with testing?

https://github.com/mcdurdin/git-windows-junction-longpath-error

The repo https://github.com/keymanapp/keyman can trigger this issue as well but it's not trivial to get it into the state required to reproduce the problem. If the two issues don't end up with the same root cause (that is, a fix for the first issue doesn't make the second one go away...), then I will try again to create an MCVE for the 'No such file or directory' issue.

[dscho]

I think this use case might work if you set git config --global core.longpaths true first.

[mcdurdin]

Unfortunately not -- I've tested with core.longpaths both false and true.

[rimrul]

I did some digging. FindFirstFileW() in open_dirent() seems to fail with ERROR_CANT_RESOLVE_FILENAME (1921). err_win_to_posix() then doesn't know how to convert that to an errno and defaults to ENOSYS.

[rimrul]

I did some more digging. Windows explorer also doesn't like the junction point node sl.js creates. So I've had a look at the documentation for fs.SymlinkSync. It recommends absolute paths for type 'junction'. So I tried changing the type to 'dir' first. Git now likes the symlink, Windows Explorer still doesn't. Next I tried absolute paths with both types. Those symlinks worked with both Git and Windows Explorer.

Turns out Node normalizes these to absolute paths in 8.3 format. Windows does not like those 8.3 symlinks.

[dscho]

Turns out Node normalizes these to absolute paths in 8.3 format. Windows does not like those 8.3 symlinks.

So this is not a Git issue, but a node.js issue?

[mcdurdin]

Thank you for your investigation.

I've done some more testing here and realise that despite all my care, sl.js is actually doing something dumb: in the symlinkSync call, the target path is not relative to the working directory, but is relative to the parent folder of the link path -- so effectively, this call is trying to make a link to the link itself. Not surprising that Windows and git choke on the result (somewhat surprising that Windows doesn't reject the attempt, but that's a different can of worms).

Apologies for the false trail. I'm not sure that there is any issue in node.js at this point either. I will close this issue and continue to research the 'No such file or directory' issue I mentioned above (if I come up with anything I will throw a link to the new issue in here).