Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update runtimeconfig.json and deps.json paths when these break past the MAX_PATH threshold #56224

Merged
merged 6 commits into from
Jul 27, 2021

Conversation

mateoatr
Copy link
Contributor

We use the path of an app to build the paths of the runtimeconfig.dev.json, runtimeconfig.json and deps.json files. If the length of the app's path is behind the MAX_PATH threshold (< 260) but close enough ([246-251]) so that the length of any of the above paths break this limit, the app breaks with:

App runtimeconfig.json from [D:\clr\runtime\artifacts\tests\Debug\ha\cy53l5rx.r1c\PortableApp\bin\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\PortableApp.dll]
Runtime config is cfg=D:\clr\runtime\artifacts\tests\Debug\ha\cy53l5rx.r1c\PortableApp\bin\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\PortableApp.runtimeconfig.json dev=D:\clr\runtime\artifacts\tests\Debug\ha\cy53l5rx.r1c\PortableApp\bin\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\PortableApp.runtimeconfig.dev.json
Attempting to read runtime config: D:\clr\runtime\artifacts\tests\Debug\ha\cy53l5rx.r1c\PortableApp\bin\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\PortableApp.runtimeconfig.json
Attempting to read dev runtime config: D:\clr\runtime\artifacts\tests\Debug\ha\cy53l5rx.r1c\PortableApp\bin\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\PortableApp.runtimeconfig.dev.json
Cannot use file stream for [D:\clr\runtime\artifacts\tests\Debug\ha\cy53l5rx.r1c\PortableApp\bin\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\PortableApp.runtimeconfig.json]: No such file or directory
Cannot use file stream for [D:\clr\runtime\artifacts\tests\Debug\ha\cy53l5rx.r1c\PortableApp\bin\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\PortableApp.runtimeconfig.json]: No such file or directory
Runtime config [D:\clr\runtime\artifacts\tests\Debug\ha\cy53l5rx.r1c\PortableApp\bin\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\PortableApp.runtimeconfig.json] is valid=[0]
Invalid runtimeconfig.json [D:\clr\runtime\artifacts\tests\Debug\ha\cy53l5rx.r1c\PortableApp\bin\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\PortableApp.runtimeconfig.json] [D:\clr\runtime\artifacts\tests\Debug\ha\cy53l5rx.r1c\PortableApp\bin\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\PortableApp.runtimeconfig.dev.json]
Invalid runtimeconfig.json [D:\clr\runtime\artifacts\tests\Debug\ha\cy53l5rx.r1c\PortableApp\bin\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\PortableApp.runtimeconfig.json] [D:\clr\runtime\artifacts\tests\Debug\ha\cy53l5rx.r1c\PortableApp\bin\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\PortableApp.runtimeconfig.dev.json]

This is because we check for the existence of the above paths (which do exist) but we don't update them if we find that they need to be treated as long paths.

Fixes #53223.

@ghost
Copy link

ghost commented Jul 23, 2021

Tagging subscribers to this area: @vitek-karas, @agocke, @VSadov
See info in area-owners.md if you want to be subscribed.

Issue Details

We use the path of an app to build the paths of the runtimeconfig.dev.json, runtimeconfig.json and deps.json files. If the length of the app's path is behind the MAX_PATH threshold (< 260) but close enough ([246-251]) so that the length of any of the above paths break this limit, the app breaks with:

App runtimeconfig.json from [D:\clr\runtime\artifacts\tests\Debug\ha\cy53l5rx.r1c\PortableApp\bin\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\PortableApp.dll]
Runtime config is cfg=D:\clr\runtime\artifacts\tests\Debug\ha\cy53l5rx.r1c\PortableApp\bin\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\PortableApp.runtimeconfig.json dev=D:\clr\runtime\artifacts\tests\Debug\ha\cy53l5rx.r1c\PortableApp\bin\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\PortableApp.runtimeconfig.dev.json
Attempting to read runtime config: D:\clr\runtime\artifacts\tests\Debug\ha\cy53l5rx.r1c\PortableApp\bin\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\PortableApp.runtimeconfig.json
Attempting to read dev runtime config: D:\clr\runtime\artifacts\tests\Debug\ha\cy53l5rx.r1c\PortableApp\bin\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\PortableApp.runtimeconfig.dev.json
Cannot use file stream for [D:\clr\runtime\artifacts\tests\Debug\ha\cy53l5rx.r1c\PortableApp\bin\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\PortableApp.runtimeconfig.json]: No such file or directory
Cannot use file stream for [D:\clr\runtime\artifacts\tests\Debug\ha\cy53l5rx.r1c\PortableApp\bin\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\PortableApp.runtimeconfig.json]: No such file or directory
Runtime config [D:\clr\runtime\artifacts\tests\Debug\ha\cy53l5rx.r1c\PortableApp\bin\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\PortableApp.runtimeconfig.json] is valid=[0]
Invalid runtimeconfig.json [D:\clr\runtime\artifacts\tests\Debug\ha\cy53l5rx.r1c\PortableApp\bin\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\PortableApp.runtimeconfig.json] [D:\clr\runtime\artifacts\tests\Debug\ha\cy53l5rx.r1c\PortableApp\bin\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\PortableApp.runtimeconfig.dev.json]
Invalid runtimeconfig.json [D:\clr\runtime\artifacts\tests\Debug\ha\cy53l5rx.r1c\PortableApp\bin\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\PortableApp.runtimeconfig.json] [D:\clr\runtime\artifacts\tests\Debug\ha\cy53l5rx.r1c\PortableApp\bin\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\PortableApp.runtimeconfig.dev.json]

This is because we check for the existence of the above paths (which do exist) but we don't update them if we find that they need to be treated as long paths.

Fixes #53223.

Author: mateoatr
Assignees: -
Labels:

area-Host

Milestone: -

@mateoatr mateoatr changed the title Update runtimeconfig.json and deps.json paths when these break Update runtimeconfig.json and deps.json paths when these break past the MAX_PATH threshold Jul 23, 2021
Copy link
Member

@vitek-karas vitek-karas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good. But I'm wondering if there are other places where we might have a similar problem...
Could you maybe look through the code base a little bit and see if there's something "interesting"?

// is just 1 char behind MAX_PATH (260) so that the runtimeconfig(.dev).json files
// break this threshold. This will cause hostfxr to normalize these paths -- here we
// are checking that the updated paths are used.
var dirName = new string('a', 259 - outputDir.Length - appExeName.Length - 2);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: In theory this would break if the location of the repo was in a too deep. Maybe we should create a folder in temp (which is basically guaranteed to the short enough on all systems).

We do have other tests creating long paths, but those don't need specific path length, they just need path longer than something. This test needs a specific path length and so the arithmetic above could end up with negative size (and the test will fail).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, this totally relies on the user having the repo on a path not so far from root. Wasn't sure if it was good to put this on %TMP%, now I remember that we do that for bundling tests... changing this.

@mateoatr
Copy link
Contributor Author

This looks good. But I'm wondering if there are other places where we might have a similar problem...
Could you maybe look through the code base a little bit and see if there's something "interesting"?

I'll have a look at all places where we call file_exists.

src/native/corehost/deps_format.cpp Outdated Show resolved Hide resolved
src/native/corehost/runtime_config.cpp Outdated Show resolved Hide resolved
Mateo Torres Ruiz added 2 commits July 26, 2021 12:48
@mateoatr mateoatr merged commit 866808d into dotnet:main Jul 27, 2021
thaystg added a commit to thaystg/runtime that referenced this pull request Jul 28, 2021
…bug_tests

* origin/main: (274 commits)
  Disable test ConnectWithCertificateForDifferentName_Throws (dotnet#56456)
  Update dependencies from https://github.com/mono/linker build 20210726.2 (dotnet#56374)
  Cleanup disabled test conditions (dotnet#56381)
  [mono] Add GC unsafe transition to mono_unhandled_exception  (dotnet#56380)
  don't fail the file extraction when we can't set last write time (dotnet#56370)
  Properly rebuild optimization data when it changes (dotnet#56397)
  Make open function calls in coreclr EINTR resilient on macOS (dotnet#56403)
  Fix dependency from EventLog to TraceSource ref (dotnet#56417)
  Fix comments in asm with JitDiffableDasm=1 (dotnet#56416)
  Catch TcpClient ctor exceptions in FtpWebRequest.CreateConnectionAsync (dotnet#56379)
  Add interop between serializer and DOMs (dotnet#56112)
  Fix type loader not recognizing overridden method (dotnet#56337)
  Prevent Segfault in EventPipe on disable (dotnet#56104)
  Update runtimeconfig.json and deps.json paths when these break past the MAX_PATH threshold  (dotnet#56224)
  Use native allocator instead of pinning when decompressing embedded PDB (dotnet#56336)
  Specify win-x64 as a valid platform in the microsoft-net-runtime-* workloads for iOS/tvOS/MacCatalyst (dotnet#56311)
  Fix FailFast message formatting race (dotnet#56388)
  Try to fix finalizer-based async tests (dotnet#56384)
  Fix MetricsEventSource tests (dotnet#56382)
  Remove invalid Castle.DynamicProxy.Internal.AbstractInvocation from ILLink descriptor files (dotnet#56392)
  ...
thaystg added a commit to thaystg/runtime that referenced this pull request Jul 28, 2021
* origin/main: (95 commits)
  Disable test ConnectWithCertificateForDifferentName_Throws (dotnet#56456)
  Update dependencies from https://github.com/mono/linker build 20210726.2 (dotnet#56374)
  Cleanup disabled test conditions (dotnet#56381)
  [mono] Add GC unsafe transition to mono_unhandled_exception  (dotnet#56380)
  don't fail the file extraction when we can't set last write time (dotnet#56370)
  Properly rebuild optimization data when it changes (dotnet#56397)
  Make open function calls in coreclr EINTR resilient on macOS (dotnet#56403)
  Fix dependency from EventLog to TraceSource ref (dotnet#56417)
  Fix comments in asm with JitDiffableDasm=1 (dotnet#56416)
  Catch TcpClient ctor exceptions in FtpWebRequest.CreateConnectionAsync (dotnet#56379)
  Add interop between serializer and DOMs (dotnet#56112)
  Fix type loader not recognizing overridden method (dotnet#56337)
  Prevent Segfault in EventPipe on disable (dotnet#56104)
  Update runtimeconfig.json and deps.json paths when these break past the MAX_PATH threshold  (dotnet#56224)
  Use native allocator instead of pinning when decompressing embedded PDB (dotnet#56336)
  Specify win-x64 as a valid platform in the microsoft-net-runtime-* workloads for iOS/tvOS/MacCatalyst (dotnet#56311)
  Fix FailFast message formatting race (dotnet#56388)
  Try to fix finalizer-based async tests (dotnet#56384)
  Fix MetricsEventSource tests (dotnet#56382)
  Remove invalid Castle.DynamicProxy.Internal.AbstractInvocation from ILLink descriptor files (dotnet#56392)
  ...
@ghost ghost locked as resolved and limited conversation to collaborators Aug 26, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

"Invalid runtimeconfig.json" on Windows for paths precisely from 260 to 274 characters
3 participants