os: handle relative paths in fixLongPath on Windows

[rsc]

The os package has a function fixLongPath that is used on Windows to turn a very long name like c:\very\long\name.txt to \\?\c:\very\long\name.txt.

This function bails out on relative path names. It also bails out on paths containing ...

This has caused no end of problems and confusion with certain APIs that do or don't accept certain paths.

We should fix fixLongPath to correctly handle all possible inputs and eliminate all those problems.

[networkimprov]

Related: #36375 & #21782

cc @alexbrainman @mattn @zx2c4

[gopherbot]

Change https://golang.org/cl/263538 mentions this issue: os: on Windows, implement fixLongPath also for relative paths

[networkimprov]

@rasky I think fixLongPath() will need to return an error since syscall.FullPath() can.

[gopherbot]

Change https://golang.org/cl/291291 mentions this issue: os: support long paths without fixup on Windows 10 >= 1607

[zx2c4]

I've just submitted a patch that avoids the need for any long path nonsense on Windows 10: https://golang.org/cl/291291

That's not as good as a solution that works on all Windows, but we'll eventually drop support for old Windows anyway. So consider this a forward-looking improvement.

cc @jstarks

[jstarks]

@zx2c4, this is quite clever, but I have some reservations about force-setting the bit--Windows only sets the bit for a process if both the system has long paths enabled and the application statically opts in via manifest (ugh). Replacing this policy (which is process-global, of course) with your own for all Go processes might cause problems. Or it may work just fine!

A safer approach for now would be to query the bit. This would rely on Go developers including a manifest in their binary, though, and most won't. Not great.

I will follow up with the team who built this mechanism to see if they have any guidance.

[networkimprov]

Many Go users build for Windows on Linux, etc, so don't have easy access to a Windows manifest tool. Maybe Go needs a Windows manifest tool or go-build option?

[jstarks]

See also #17835.

[zx2c4]

@zx2c4, this is quite clever, but I have some reservations about force-setting the bit--Windows only sets the bit for a process if both the system has long paths enabled and the application statically opts in via manifest (ugh). Replacing this policy (which is process-global, of course) with your own for all Go processes might cause problems. Or it may work just fine!

I'm pretty sure it would work fine. Unlike random win32 programs, the Go runtime actually controls what's going on, and we're not worried about MAX_PATH buffer overflows or whatever. And if there's a place where it doesn't work fine, we're in a position for it to work fine.

And since the "policy" is per process, Go is in a good place to change it. Currently the thing setting it is just good old kernel32.dll at initialization time (actually kernelbase.dll), nothing fancy.

A safer approach for now would be to query the bit. This would rely on Go developers including a manifest in their binary, though, and most won't. Not great.

I'm not a huge fan of the manifest approach. And I don't like the idea of us twiddling with the registry or even having to only have this if the registry key is enabled. The Go runtime owns its process, so we can very safely just unconditionally enable it.

I will follow up with the team who built this mechanism to see if they have any guidance.

Thanks. Please do relay any interesting concerns that the team that owns this code has. Maybe there's something I don't know about that will come up.

[ianlancetaylor]

I don't know anything about this, but what happens if a -buildmode=c-archive Go archive is linked into a C program?

[zx2c4]

Then the resultant C program will handle long paths! IMO, that's a good thing, and a side effect that we can document. If people are writing new software with new Go they're in a place to do things well. OTOH, if you disagree, we could enable this automatically only for exe/pie mode, and put it behind some sort of flag for c-archive/c-shared.

[networkimprov]

See also #44466.

[zx2c4]

See also #44466.

I do not believe that issue is very relevant to this issue, because enabling long paths in a manifest file still requires that a global registry knob be turned, which is not something we can rely on happening, and wouldn't want to induce.

On the other hand, https://golang.org/cl/291291 enables the feature per-process, with no global affect, which sounds much more like exactly what we want.

[gopherbot]

Change https://go.dev/cl/570995 mentions this issue: os: support UNC paths and .. segments in fixLongPath

[qmuntal]

I've marked CL 570995 as a fix for this issue. It adds support for UNC paths and .. segments.

Relative paths can't be used with the extended path prefix, so there is nothing to fix there. Quote from Microsoft docs:

Because you cannot use the "\?" prefix with a relative path, relative paths are always limited to a total of MAX_PATH characters.

[qmuntal]

Relative paths can't be used with the extended path prefix, so there is nothing to fix there. Quote from Microsoft docs:

I'm reopening this issue, as it should be possible to support relative paths in fixLongPath by converting them to absolute when required.

[gopherbot]

Change https://go.dev/cl/574695 mentions this issue: os: support relative paths in fixLongPath

[gopherbot]

Change https://go.dev/cl/607437 mentions this issue: os: improve Windows fixLongPath