Add longPathAware to the app manifest on Windows

[MOZGIII]

Describe the problem you are trying to solve

On Windows, when the directory where the project files are located is too long, cargo fails with various file-related errors.

Describe the solution you'd like

Rust should handle long path ok. We just have to add a longPathAware to the app manifest:
https://docs.microsoft.com/en-us/windows/win32/fileio/naming-a-file

<application xmlns="urn:schemas-microsoft-com:asm.v3">
    <windowsSettings xmlns:ws2="http://schemas.microsoft.com/SMI/2016/WindowsSettings">
        <ws2:longPathAware>true</ws2:longPathAware>
    </windowsSettings>
</application>

Notes

This issue only occurs if the file paths cargo wants to touch are too long.
This happens with paths like these:

C:\Users\User\AppData\Local\Yarn\Cache\v6\.tmp\19aa03367eb9c4791798fcfbbdd8d7d4.e2c16fa02b2d9250fe13c2054c37a11aef5c0da3.prepare\native\target\x86_64-pc-windows-msvc\release\.fingerprint\semver-parser-dc26c18a352c24f5\lib-semver_parser-dc26c18a352c24f5.json

[ehuss]

I think this would be a very hard problem to solve, if not impossible. Some issues I can think of:

• Rust doesn't have a way to embed the app manifest AFAIK.
• The default registry key needed for that is OFF, so I don't think it would help most people.
• All the windows-interfacing APIs (in C and Rust) would need to be audited/updated for the larger constant.
• All of rustc and cargo would need to handle this.
• IIRC, MSVC's linker can't handle long filenames, so it probably wouldn't work even if we tried.
• Many other tools would not be able to handle these paths.
• We could maybe try to use canonicalized device paths (\\?\) everywhere, but I think that would be very difficult and problematic. We already have problems with device paths on various filesystems. They are also not user-friendly, so would probably need some abstraction that could show different representations.

[MOZGIII]

Thanks for the response! Maybe there's a workaround I could apply to handle this? Like setting a custom CARGO_TARGET_DIR - would that work? I think it should work...

[victoryang00]

It could be disable by the regedit.msc. so just stopping checking it in the cargo for windows'll be fine.

[RoguePointer80]

Rust doesn't have a way to embed the app manifest AFAIK.

If your CI has the MSVC toolsets installed, you have all that you need. Don't have them? Here is the direct download link.

The default registry key needed for that is OFF, so I don't think it would help most people

But for the people who DO have it on, it would help. In my opinion, that's a pretty good start! And when developers start seeing that their colleague got it working, they will ask "hey dude, what did you do?", and soon their IT department will turn the switch on for all new dev computers.

All the windows-interfacing APIs (in C and Rust) would need to be audited/updated for the larger constant.

If your windows-interfacing APIs are still using a static 260-char buffer, it's past time to fix that code. All Windows APIs have a way to query the required buffer length. The typical way is calling the function with a 0-length buffer, and the return code is the required length. Any skilled Windows developer should know that.

All of rustc and cargo would need to handle this.

Yep. Let's start with this tool (this issue is about Cargo, right?). When it's done, other executables will copy the solution. Lead by example!

IIRC, MSVC's linker can't handle long filenames, so it probably wouldn't work even if we tried.

Have you tried? Or are you assuming other tools might not work as an excuse not to try? How will the situation improve if every maintainer used the argument "tool X is not updated, so I'm not updating tool Y" ? I strongly believe that if everyone does their part, the ecosystem will quickly improve for everybody.

Many other tools would not be able to handle these paths

Maybe they already do, but unless cargo is updated, we'll never know. And what's the worst-case here? That cargo supports it, and we find out that a downstream tool gives an error. Then we can create an issue and a PR for that downstream tool, and keep making progress.

We could maybe try to use canonicalized UNC paths everywhere

Good point, excellent explanation. This is not a very good solution. The manifest solution seems a better approach.

[nitsky]

I just ran into this issue because I have a crate with a long name, over 100 characters, so the path to the crate's location in the target folder exceeds the 260 character limit. Because of this, even cargo build fails. Does anyone following this issue know of any workarounds?

[ghost]

don't get your hopes up,
as far as I can tell, at least for explorer.exe,
on Windows 10 Enterprise Preview build 19044.1200.210812-1726.21H2 - August 2021
you, still, can not handle path with length longer than 260 (actually 258) characters.

that's because No version of windows's Explorer.exe embeds longPathAware in it's manifest file.
longPathAware has to embedded per executable.
If explorer.exe embeds longPathAware in it's manifest, legacy executables that don't embed in the manifest will simply fail to launch from long path. That's the reason Win32 Long Paths registry key is off by default.

Manifest files are an integral part of Modern Windows. A system programming lang like Rust shouldn't do dilly-dally in this space.

[ghost]

@ehuss as @RoguePointer80 addresses your points, and regarding Registry Key off by default point, you should know the person installs a system programming lang like in the first place will most likely be A Developer, not a average-joe. so this point is moot.

First of all , Rust shouldn't call undocumented apis.
secondly, NT UNC(\\?\) paths are designed to be used by the Subsystems and KMDF Drivers, not user mode programs like Rust.
Third, that leaves the only way , that is to use embedded manifests per executables.
Fourth, all of these you listed above apparently are assumptions that seem to come from not using windows regularly.

[ghost]

@nitsky see this post https://gal.hagever.com/posts/windows-long-paths-in-rust/
how side BurntSushi/ripgrep#364 (comment)

[ghost]

I'll just export the embedded manifests from explorer exe/dll/mui, patch them

Wah ! Clever. forgot about SxS manifest support. I'm afraid about that "Patch part" i.e the post-patching consequences here, especially about touching core executables

[ChrisDenton]

rustc has been using an application manifest file for a while now (since 1.63). There have yet to be any reported issues.

A manifest allows setting more than just longPathAware. rustc sets the supported OSes and also sets the code page to UTF-8. It's implemented as a build script which leverages MSVC's link.exe to do the actual embedding (alternatively, .cargo/config.toml might be used). So it's actually easy to do for that target.

I think at a minimum cargo should set the OSes it supports so as to avoid unnecessary compatibility shims. I'd also advocate for setting the other two options even if they're less likely to be needed in Rust itself (due to using the UTF-16 APIs).

[ehuss]

Closing as #13131 has added a manifest. That is just the beginning of long-path support. See the meta tracking issue at #9770, and #13141 for some more examples of what still doesn't work.

For people hitting the issue in git clones, be sure to check out #13020 which has more information (in short, add core.longpaths=true to your git config).

[mati865]

This issue has been solved only for MSVC targets, should the same be done for Windows GNU?

[ChrisDenton]

Last I looked (which was awhile ago) that was more complicated because gnu didn't have linker arguments for adding a manifest so you either had to use a post-build step to edit the binary or else manually construct an object file with the right tables and link them (which is a much more involved process). Maybe that's changed though.

[mati865]

Oh, that could be a problem, we would nee to use LLD to circumvent it.