From 0fea6f5472db5aa7f99a245953b46488d86f2566 Mon Sep 17 00:00:00 2001 From: Daniel Plaisted Date: Fri, 18 Nov 2022 17:32:29 -0500 Subject: [PATCH 01/10] Add proposal for simplified output paths --- accepted/2022/simplify-output-paths.md | 58 ++++++++++++++++++++++++++ 1 file changed, 58 insertions(+) create mode 100644 accepted/2022/simplify-output-paths.md diff --git a/accepted/2022/simplify-output-paths.md b/accepted/2022/simplify-output-paths.md new file mode 100644 index 000000000..e3ab5dbba --- /dev/null +++ b/accepted/2022/simplify-output-paths.md @@ -0,0 +1,58 @@ +# Simplify .NET Output Paths + +We'd like to simplify the output paths for .NET 8. + +Currently, the default output path for a .NET project includes folders for the `TargetFramework` and the `RuntimeIdentifier` (if it is set). Additionally, the publish output goes in a `publish` subfolder of the output folder. The resulting paths are as follows: + +- `bin\\` - Build output with no `RuntimeIdentifier` +- `bin\\\` - Build output with `RuntimeIdentifier` +- `bin\\\publish` - Publish output with no `RuntimeIdentifier` +- `bin\\\\publish` - Publish output with `RuntimeIdentifier` + +This is rather complicated an inconsistent, which can make for a poor first impression of the .NET platform. + +We'd like to try to improve the output path structure. Desired qualities include: + +- Simple and consistent +- Avoid cases where one output path is nested inside of another one +- Avoid excessively deep directory heirarchies +- Get rid of `obj` folder in project root + +## Proposed behavior + +Projects targeting .NET and higher will by default use a new output path format. The output path will consist of the following 3 nested folders + +- `bin` - All output (including intermediate output, will go under this folder) +- Output Type - Such as `build`, `publish`, `obj`, or `packages` +- Pivots - This will at minimum be the `Configuration`, such as `Debug` or `Release`. Other pivots such as `TargetFramework` or `RuntimeIdentifier` may also be included, and the pivots will be joined by the underscore (`_`) character + - `TargetFramework` will be included in the folder name if the project is multi-targeted (`TargetFrameworks` is non-empty), or if the `TargetFramework` property was set via the command line (ie is a global property) + - `RuntimeIdentifier` will be included in the folder name if it was explicitly set (either in a project file or on the command line). If it is set automatically by the SDK (for exmaple because `SelfContained` was set) + +Some examples: + +- `bin\build\Debug` - The build output path for a simple project when you run `dotnet build` +- `bin\obj\Debug` - The intermediate output path for a simple project when you run `dotnet build` +- `bin\build\Debug_net8.0` - The build output path for the `net8.0` build of a multi-targeted project +- `bin\publish\Release_linux-x64` - The publish path for a simple app when publishing for `linux-x64` +- `bin\package\Release` - The folder where the release .nupkg will be created for a project + +## Considerations + +### Breaking changes + +These changes could break things such as scripts that copy the output of the build or custom MSBuild logic that hard-codes these paths. Tieing these changes to the project TargetFramework ensures that these breaks will be encountered when the project is modified to target a new TargetFramework, not when updating to a new version of the .NET SDK. + +### Opting in or out of the new behavior + +We need a way to explicitly opt in to or out of the new behavior. This will let projects targeting prior versions of .NET opt in to the new folder structure, which is especially desirable for multi-targeted projects so that the inner builds have consistent output paths with each other. It will also let projects that target .NET 8 use the old output path structure, for example if the new paths break scripts or CI setup. + +We need to come up with a name for the property that controls this. `UseNewOutputPathFormat` is probably not a good name. + +### Can't we use `bin\Debug`? + +Before .NET Core, .NET Framework projects generally put their output directly in `bin\Debug` or `bin\Release`. It would be desirable if we could go back to putting the output in `bin\Debug`. However, to do so we would need to do one of the following: + +- Add more top-level project output folders (for example `pub/` for publish output next to `bin/` and `obj`) +- Have an inconsistent folder structure (for example `bin\publish` next to `bin\Debug`) + +This is a classic "pick two of three things" situation, and we believe the least important of the three was trying to preserve `bin\` as an output path. From a78334ae936c898d31105b43873c8271ce92e85f Mon Sep 17 00:00:00 2001 From: Daniel Plaisted Date: Tue, 22 Nov 2022 05:21:52 -0800 Subject: [PATCH 02/10] Apply suggestions from code review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Rolf Bjarne Kvinge Co-authored-by: Alexander Köplinger Co-authored-by: Chet Husk --- accepted/2022/simplify-output-paths.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/accepted/2022/simplify-output-paths.md b/accepted/2022/simplify-output-paths.md index e3ab5dbba..40a25bd50 100644 --- a/accepted/2022/simplify-output-paths.md +++ b/accepted/2022/simplify-output-paths.md @@ -9,24 +9,24 @@ Currently, the default output path for a .NET project includes folders for the ` - `bin\\\publish` - Publish output with no `RuntimeIdentifier` - `bin\\\\publish` - Publish output with `RuntimeIdentifier` -This is rather complicated an inconsistent, which can make for a poor first impression of the .NET platform. +This is rather complicated and inconsistent, which can make for a poor first impression of the .NET platform. We'd like to try to improve the output path structure. Desired qualities include: - Simple and consistent - Avoid cases where one output path is nested inside of another one -- Avoid excessively deep directory heirarchies +- Avoid excessively deep directory hierarchies - Get rid of `obj` folder in project root ## Proposed behavior -Projects targeting .NET and higher will by default use a new output path format. The output path will consist of the following 3 nested folders +Projects targeting .NET 8 and higher will by default use a new output path format. The output path will consist of the following 3 nested folders - `bin` - All output (including intermediate output, will go under this folder) - Output Type - Such as `build`, `publish`, `obj`, or `packages` - Pivots - This will at minimum be the `Configuration`, such as `Debug` or `Release`. Other pivots such as `TargetFramework` or `RuntimeIdentifier` may also be included, and the pivots will be joined by the underscore (`_`) character - `TargetFramework` will be included in the folder name if the project is multi-targeted (`TargetFrameworks` is non-empty), or if the `TargetFramework` property was set via the command line (ie is a global property) - - `RuntimeIdentifier` will be included in the folder name if it was explicitly set (either in a project file or on the command line). If it is set automatically by the SDK (for exmaple because `SelfContained` was set) + - `RuntimeIdentifier` will be included in the folder name if it was explicitly set (either in a project file or on the command line). If it is set automatically by the SDK (for example because `SelfContained` was set) Some examples: From 8d3d67f742674a3d4edd6b5cfcd3fd1478d2d246 Mon Sep 17 00:00:00 2001 From: Daniel Plaisted Date: Tue, 22 Nov 2022 08:24:08 -0500 Subject: [PATCH 03/10] Fix incomplete sentence --- accepted/2022/simplify-output-paths.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/accepted/2022/simplify-output-paths.md b/accepted/2022/simplify-output-paths.md index 40a25bd50..5fcb221dd 100644 --- a/accepted/2022/simplify-output-paths.md +++ b/accepted/2022/simplify-output-paths.md @@ -26,7 +26,7 @@ Projects targeting .NET 8 and higher will by default use a new output path forma - Output Type - Such as `build`, `publish`, `obj`, or `packages` - Pivots - This will at minimum be the `Configuration`, such as `Debug` or `Release`. Other pivots such as `TargetFramework` or `RuntimeIdentifier` may also be included, and the pivots will be joined by the underscore (`_`) character - `TargetFramework` will be included in the folder name if the project is multi-targeted (`TargetFrameworks` is non-empty), or if the `TargetFramework` property was set via the command line (ie is a global property) - - `RuntimeIdentifier` will be included in the folder name if it was explicitly set (either in a project file or on the command line). If it is set automatically by the SDK (for example because `SelfContained` was set) + - `RuntimeIdentifier` will be included in the folder name if it was explicitly set (either in a project file or on the command line). If it is set automatically by the SDK (for example because `SelfContained` was set), then the `RuntimeIdentifier` will not be included in the folder name Some examples: From 7d2a6cd8e4369dd046fbd95ecab64d4d44b5dfd4 Mon Sep 17 00:00:00 2001 From: Daniel Plaisted Date: Wed, 23 Nov 2022 12:04:22 -0500 Subject: [PATCH 04/10] Switch to artifacts output format --- accepted/2022/simplify-output-paths.md | 49 ++++++++++++++++++-------- 1 file changed, 34 insertions(+), 15 deletions(-) diff --git a/accepted/2022/simplify-output-paths.md b/accepted/2022/simplify-output-paths.md index 5fcb221dd..3bbba3444 100644 --- a/accepted/2022/simplify-output-paths.md +++ b/accepted/2022/simplify-output-paths.md @@ -20,33 +20,51 @@ We'd like to try to improve the output path structure. Desired qualities includ ## Proposed behavior -Projects targeting .NET 8 and higher will by default use a new output path format. The output path will consist of the following 3 nested folders +The new output path format will consist of the following 3 nested folders: -- `bin` - All output (including intermediate output, will go under this folder) -- Output Type - Such as `build`, `publish`, `obj`, or `packages` -- Pivots - This will at minimum be the `Configuration`, such as `Debug` or `Release`. Other pivots such as `TargetFramework` or `RuntimeIdentifier` may also be included, and the pivots will be joined by the underscore (`_`) character +- `artifacts` - All output will go under this folder +- Output Type - Such as `bin`, `publish`, `intermediates`, or `package` +- Pivots - This will at minimum be the `Configuration` (lower-cased), such as `debug` or `release`. Other pivots such as `TargetFramework` or `RuntimeIdentifier` may also be included, and the pivots will be joined by the underscore (`_`) character - `TargetFramework` will be included in the folder name if the project is multi-targeted (`TargetFrameworks` is non-empty), or if the `TargetFramework` property was set via the command line (ie is a global property) - `RuntimeIdentifier` will be included in the folder name if it was explicitly set (either in a project file or on the command line). If it is set automatically by the SDK (for example because `SelfContained` was set), then the `RuntimeIdentifier` will not be included in the folder name Some examples: -- `bin\build\Debug` - The build output path for a simple project when you run `dotnet build` -- `bin\obj\Debug` - The intermediate output path for a simple project when you run `dotnet build` -- `bin\build\Debug_net8.0` - The build output path for the `net8.0` build of a multi-targeted project -- `bin\publish\Release_linux-x64` - The publish path for a simple app when publishing for `linux-x64` -- `bin\package\Release` - The folder where the release .nupkg will be created for a project +- `artifacts\bin\debug` - The build output path for a simple project when you run `dotnet build` +- `artifacts\intermediates\debug` - The intermediate output path for a simple project when you run `dotnet build` +- `artifacts\bin\debug_net8.0` - The build output path for the `net8.0` build of a multi-targeted project +- `artifacts\publish\release_linux-x64` - The publish path for a simple app when publishing for `linux-x64` +- `artifacts\package\release` - The folder where the release .nupkg will be created for a project -## Considerations +### Controlling the output path format + +We would like the new output format to be used by default. However, a variety of things may depend on the output path, such as scripts that copy the output of the build, or custom MSBuild logic that hard-codes the output path. So it would be a breaking change to change the output path format for all projects. + +One strategy we often use for situations like this is to make new behavior the default only when targeting the version of .NET that the behavior was introduced in or higher. This prevents the new .NET SDK from breaking existing, unmodified projects, and ensures that the change in behavior will only be encountered when projects are modified to target a new `TargetFramework`. + +The logic for determining which output path format will be used will (mostly) be as follows: + +- If a project targets .NET 8 or higher, it will by default use the new output path format. If it targets .NET 7 or lower, it will by default use the old output path format +- A project can explicitly choose which output path format to use by setting the `UseArtifactsOutput` property to `true` or `false` in a `Directory.Build.props` file. + +### That pesky intermediate output path -### Breaking changes +Unfortunately, the base intermediate output path can't depend on the project's `TargetFramework`, or anything else in the project file. The only place to set properties which affect the base intermediate output path is in a `Directory.Build.props` file. This is because MSBuild imports "project extensions" of the form `.*.props` and `.*.targets` from the base intermediate output path, and the project extensions .props files are imported early on in evaluation, before the body of the project file is evaluated. -These changes could break things such as scripts that copy the output of the build or custom MSBuild logic that hard-codes these paths. Tieing these changes to the project TargetFramework ensures that these breaks will be encountered when the project is modified to target a new TargetFramework, not when updating to a new version of the .NET SDK. +So, the way we will handle this is: -### Opting in or out of the new behavior +- If `UseArtifactsOutput` is set to `true` in a `Directory.Build.props` file, then the new output format will be used, including for the intermediate output path +- Otherwise, if `UseArtifactsOutput` is set to true later, either in the project file or in .NET SDK logic due to the `TargetFramework` being at least .NET 8, then the old output format will be used for the intermedate folder (so it will start with `obj\` by default), but the new `artifacts` format will be used for all other output. -We need a way to explicitly opt in to or out of the new behavior. This will let projects targeting prior versions of .NET opt in to the new folder structure, which is especially desirable for multi-targeted projects so that the inner builds have consistent output paths with each other. It will also let projects that target .NET 8 use the old output path structure, for example if the new paths break scripts or CI setup. +### Mixed output path formats in multi-targeted builds -We need to come up with a name for the property that controls this. `UseNewOutputPathFormat` is probably not a good name. +If a project is multi-targeted to `net7.0` and `net8.0` and doesn't specify the output path format, then different formats will be used for the output for the different target frameworks. The .NET 7 build output will be in `bin\Debug\net7.0`, while the .NET 8 output will be in `artifacts\bin\debug_net8.0`. + +To prevent the output from one target framework to be globbed as part of the inputs to another target framework, both the `bin` and `artifacts` folder will need to be excluded from the default item globs (via the `DefaultItemExcludes` property). This means that even projects that target .NET 7 or lower could be impacted by breaking changes if they currently have source files or other assets in an `artifacts` folder. + +Question: Could we have the outer build of a multi-targeted build query the inner builds to determine whether any of them target .NET 8 or higher, so we wouldn't have to have inconsistent output path formats? Would this work with Visual Studio builds? + +## Considerations ### Can't we use `bin\Debug`? @@ -56,3 +74,4 @@ Before .NET Core, .NET Framework projects generally put their output directly in - Have an inconsistent folder structure (for example `bin\publish` next to `bin\Debug`) This is a classic "pick two of three things" situation, and we believe the least important of the three was trying to preserve `bin\` as an output path. + From c1e281e61df8c92546b4d6b9fcc75c80c705c5c1 Mon Sep 17 00:00:00 2001 From: Daniel Plaisted Date: Mon, 28 Nov 2022 16:58:19 -0500 Subject: [PATCH 05/10] Add root artifacts path support --- accepted/2022/simplify-output-paths.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/accepted/2022/simplify-output-paths.md b/accepted/2022/simplify-output-paths.md index 3bbba3444..c2f1b74df 100644 --- a/accepted/2022/simplify-output-paths.md +++ b/accepted/2022/simplify-output-paths.md @@ -64,6 +64,16 @@ To prevent the output from one target framework to be globbed as part of the inp Question: Could we have the outer build of a multi-targeted build query the inner builds to determine whether any of them target .NET 8 or higher, so we wouldn't have to have inconsistent output path formats? Would this work with Visual Studio builds? +### Solution or repo level output paths + +Many projects want to put all the output for the repo or the solution in a single folder. We will support this with a new `RootArtifactsPath` property which can be set in a `Directory.Build.props file`. If set, then we will use the following format for the output path: + +`\\\` + +If `RootArtifactsPath` is set, then it will not be necessary to separately set `UseArtifactsOutput`. + +We will consider updating commands such as `dotnet build` and `dotnet publish` so that when used on a `.sln` file, the `--output` parameter sets the `RootArtifactsPath`. + ## Considerations ### Can't we use `bin\Debug`? From 2e06586bbf27097ce58e0dc68ac654ed4288fba7 Mon Sep 17 00:00:00 2001 From: Daniel Plaisted Date: Fri, 13 Jan 2023 10:51:55 -0500 Subject: [PATCH 06/10] Updates around pack logic --- accepted/2022/simplify-output-paths.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/accepted/2022/simplify-output-paths.md b/accepted/2022/simplify-output-paths.md index c2f1b74df..0f8ba7131 100644 --- a/accepted/2022/simplify-output-paths.md +++ b/accepted/2022/simplify-output-paths.md @@ -27,6 +27,7 @@ The new output path format will consist of the following 3 nested folders: - Pivots - This will at minimum be the `Configuration` (lower-cased), such as `debug` or `release`. Other pivots such as `TargetFramework` or `RuntimeIdentifier` may also be included, and the pivots will be joined by the underscore (`_`) character - `TargetFramework` will be included in the folder name if the project is multi-targeted (`TargetFrameworks` is non-empty), or if the `TargetFramework` property was set via the command line (ie is a global property) - `RuntimeIdentifier` will be included in the folder name if it was explicitly set (either in a project file or on the command line). If it is set automatically by the SDK (for example because `SelfContained` was set), then the `RuntimeIdentifier` will not be included in the folder name + - For the `package` output type, the pivots will only be the `Configuration`, and other values won't be included in the path Some examples: @@ -85,3 +86,10 @@ Before .NET Core, .NET Framework projects generally put their output directly in This is a classic "pick two of three things" situation, and we believe the least important of the three was trying to preserve `bin\` as an output path. +### NuGet pack logic + +Currently, as part of the NuGet pack command, the `_WalkEachTargetPerFramework` target runs the `_GetBuildOutputFilesWithTfm` target for each `TargetFramework`. This means in this inner build the `TargetFramework` will be a global property, even if there is only one. + +This interferes with the logic where the output path depends on whether the `TargetFramework` is specified as a global property. To fix this, the NuGet pack logic would need to change so that if there is only one target framework, it does not pass the `TargetFramework` global property in this case. + +https://github.com/NuGet/Home/issues/12323 \ No newline at end of file From 1ebb7881ba01daee2c1bb2c444cc9f8dc0d58f73 Mon Sep 17 00:00:00 2001 From: Daniel Plaisted Date: Fri, 3 Feb 2023 16:23:47 -0500 Subject: [PATCH 07/10] Update proposal --- INDEX.md | 1 + accepted/2022/simplify-output-paths.md | 95 ------------------ accepted/2023/simplify-output-paths.md | 130 +++++++++++++++++++++++++ 3 files changed, 131 insertions(+), 95 deletions(-) delete mode 100644 accepted/2022/simplify-output-paths.md create mode 100644 accepted/2023/simplify-output-paths.md diff --git a/INDEX.md b/INDEX.md index 246917242..0568d48df 100644 --- a/INDEX.md +++ b/INDEX.md @@ -81,6 +81,7 @@ Use update-index to regenerate it: | 2021 | [TFM for .NET nanoFramework](accepted/2021/nano-framework-tfm/nano-framework-tfm.md) | [Immo Landwerth](https://github.com/terrajobst), [Laurent Ellerbach](https://github.com/Ellerbach), [José Simões](https://github.com/josesimoes) | | 2021 | [Tracking Platform Dependencies](accepted/2021/platform-dependencies/platform-dependencies.md) | [Matt Thalman](https://github.com/mthalman) | | 2022 | [.NET 7 Version Selection Improvements](accepted/2022/version-selection.md) | [Rich Lander](https://github.com/richlander) | +| 2023 | [Simplify .NET Output Paths](accepted/2023/simplify-output-paths.md) | [Daniel Plaisted](https://github.com/dsplaisted) | ## Drafts diff --git a/accepted/2022/simplify-output-paths.md b/accepted/2022/simplify-output-paths.md deleted file mode 100644 index 0f8ba7131..000000000 --- a/accepted/2022/simplify-output-paths.md +++ /dev/null @@ -1,95 +0,0 @@ -# Simplify .NET Output Paths - -We'd like to simplify the output paths for .NET 8. - -Currently, the default output path for a .NET project includes folders for the `TargetFramework` and the `RuntimeIdentifier` (if it is set). Additionally, the publish output goes in a `publish` subfolder of the output folder. The resulting paths are as follows: - -- `bin\\` - Build output with no `RuntimeIdentifier` -- `bin\\\` - Build output with `RuntimeIdentifier` -- `bin\\\publish` - Publish output with no `RuntimeIdentifier` -- `bin\\\\publish` - Publish output with `RuntimeIdentifier` - -This is rather complicated and inconsistent, which can make for a poor first impression of the .NET platform. - -We'd like to try to improve the output path structure. Desired qualities include: - -- Simple and consistent -- Avoid cases where one output path is nested inside of another one -- Avoid excessively deep directory hierarchies -- Get rid of `obj` folder in project root - -## Proposed behavior - -The new output path format will consist of the following 3 nested folders: - -- `artifacts` - All output will go under this folder -- Output Type - Such as `bin`, `publish`, `intermediates`, or `package` -- Pivots - This will at minimum be the `Configuration` (lower-cased), such as `debug` or `release`. Other pivots such as `TargetFramework` or `RuntimeIdentifier` may also be included, and the pivots will be joined by the underscore (`_`) character - - `TargetFramework` will be included in the folder name if the project is multi-targeted (`TargetFrameworks` is non-empty), or if the `TargetFramework` property was set via the command line (ie is a global property) - - `RuntimeIdentifier` will be included in the folder name if it was explicitly set (either in a project file or on the command line). If it is set automatically by the SDK (for example because `SelfContained` was set), then the `RuntimeIdentifier` will not be included in the folder name - - For the `package` output type, the pivots will only be the `Configuration`, and other values won't be included in the path - -Some examples: - -- `artifacts\bin\debug` - The build output path for a simple project when you run `dotnet build` -- `artifacts\intermediates\debug` - The intermediate output path for a simple project when you run `dotnet build` -- `artifacts\bin\debug_net8.0` - The build output path for the `net8.0` build of a multi-targeted project -- `artifacts\publish\release_linux-x64` - The publish path for a simple app when publishing for `linux-x64` -- `artifacts\package\release` - The folder where the release .nupkg will be created for a project - -### Controlling the output path format - -We would like the new output format to be used by default. However, a variety of things may depend on the output path, such as scripts that copy the output of the build, or custom MSBuild logic that hard-codes the output path. So it would be a breaking change to change the output path format for all projects. - -One strategy we often use for situations like this is to make new behavior the default only when targeting the version of .NET that the behavior was introduced in or higher. This prevents the new .NET SDK from breaking existing, unmodified projects, and ensures that the change in behavior will only be encountered when projects are modified to target a new `TargetFramework`. - -The logic for determining which output path format will be used will (mostly) be as follows: - -- If a project targets .NET 8 or higher, it will by default use the new output path format. If it targets .NET 7 or lower, it will by default use the old output path format -- A project can explicitly choose which output path format to use by setting the `UseArtifactsOutput` property to `true` or `false` in a `Directory.Build.props` file. - -### That pesky intermediate output path - -Unfortunately, the base intermediate output path can't depend on the project's `TargetFramework`, or anything else in the project file. The only place to set properties which affect the base intermediate output path is in a `Directory.Build.props` file. This is because MSBuild imports "project extensions" of the form `.*.props` and `.*.targets` from the base intermediate output path, and the project extensions .props files are imported early on in evaluation, before the body of the project file is evaluated. - -So, the way we will handle this is: - -- If `UseArtifactsOutput` is set to `true` in a `Directory.Build.props` file, then the new output format will be used, including for the intermediate output path -- Otherwise, if `UseArtifactsOutput` is set to true later, either in the project file or in .NET SDK logic due to the `TargetFramework` being at least .NET 8, then the old output format will be used for the intermedate folder (so it will start with `obj\` by default), but the new `artifacts` format will be used for all other output. - -### Mixed output path formats in multi-targeted builds - -If a project is multi-targeted to `net7.0` and `net8.0` and doesn't specify the output path format, then different formats will be used for the output for the different target frameworks. The .NET 7 build output will be in `bin\Debug\net7.0`, while the .NET 8 output will be in `artifacts\bin\debug_net8.0`. - -To prevent the output from one target framework to be globbed as part of the inputs to another target framework, both the `bin` and `artifacts` folder will need to be excluded from the default item globs (via the `DefaultItemExcludes` property). This means that even projects that target .NET 7 or lower could be impacted by breaking changes if they currently have source files or other assets in an `artifacts` folder. - -Question: Could we have the outer build of a multi-targeted build query the inner builds to determine whether any of them target .NET 8 or higher, so we wouldn't have to have inconsistent output path formats? Would this work with Visual Studio builds? - -### Solution or repo level output paths - -Many projects want to put all the output for the repo or the solution in a single folder. We will support this with a new `RootArtifactsPath` property which can be set in a `Directory.Build.props file`. If set, then we will use the following format for the output path: - -`\\\` - -If `RootArtifactsPath` is set, then it will not be necessary to separately set `UseArtifactsOutput`. - -We will consider updating commands such as `dotnet build` and `dotnet publish` so that when used on a `.sln` file, the `--output` parameter sets the `RootArtifactsPath`. - -## Considerations - -### Can't we use `bin\Debug`? - -Before .NET Core, .NET Framework projects generally put their output directly in `bin\Debug` or `bin\Release`. It would be desirable if we could go back to putting the output in `bin\Debug`. However, to do so we would need to do one of the following: - -- Add more top-level project output folders (for example `pub/` for publish output next to `bin/` and `obj`) -- Have an inconsistent folder structure (for example `bin\publish` next to `bin\Debug`) - -This is a classic "pick two of three things" situation, and we believe the least important of the three was trying to preserve `bin\` as an output path. - -### NuGet pack logic - -Currently, as part of the NuGet pack command, the `_WalkEachTargetPerFramework` target runs the `_GetBuildOutputFilesWithTfm` target for each `TargetFramework`. This means in this inner build the `TargetFramework` will be a global property, even if there is only one. - -This interferes with the logic where the output path depends on whether the `TargetFramework` is specified as a global property. To fix this, the NuGet pack logic would need to change so that if there is only one target framework, it does not pass the `TargetFramework` global property in this case. - -https://github.com/NuGet/Home/issues/12323 \ No newline at end of file diff --git a/accepted/2023/simplify-output-paths.md b/accepted/2023/simplify-output-paths.md new file mode 100644 index 000000000..b6d9fede3 --- /dev/null +++ b/accepted/2023/simplify-output-paths.md @@ -0,0 +1,130 @@ +# Simplify .NET Output Paths + +We'd like to make some improvements to project output paths for .NET 8. + +Goals: + +- Support output paths in a repo or solution-specific folder, rather than each project folder having its own output paths +- Provide simple and consistent output paths +- Avoid cases where one output path is nested inside of another one +- Avoid excessively deep directory heirarchies + +## Motivation + +Some developers would like to put all the output for their solution or repo [under a single folder](https://github.com/dotnet/sdk/issues/867). This would make it easier to delete all of the output, as MSBuild-based clean only deletes output for a single configuration, and doesn't delete everything from the intermediate output folder, as that's where it keeps track of what was produced by the build and hence what should be deleted in a clean. It would support building with [read only](https://github.com/dotnet/sdk/issues/887) [source trees](https://github.com/dotnet/designs/pull/281#discussion_r1072949097). It could also make it easier to set up CI and deployment. + +The current output paths for .NET projects are rather complicated and inconsistent. The default output path for a .NET project includes folders for the `TargetFramework` and the `RuntimeIdentifier` (if it is set). Additionally, the publish output goes in a `publish` subfolder of the output folder. The resulting paths are as follows: + +- `bin\\` - Build output with no `RuntimeIdentifier` +- `bin\\\` - Build output with `RuntimeIdentifier` +- `bin\\\publish` - Publish output with no `RuntimeIdentifier` +- `bin\\\\publish` - Publish output with `RuntimeIdentifier` + +## Proposed behavior + +We will introduce a new output path format, called the "artifacts" output format after the default folder name it will use. By convention, the artifacts folder will be located at the solution or repo level rather within each project folder. + +The recommended way to opt in to the new output path format will be to set the `UseArtifactsOutput` property to `true` in a `Directory.Build.props` file. + +The artifacts folder will by default be named `.artifacts`. The convention we will use to determine which folder the artifacts folder should go in will be: + +- If a `Directory.Build.props` file was used in the build, put the artifacts folder in the same folder where `Directory.Build.props` was found, which will typically be at the root of a repo. Since MSBuild already probes for `Directory.Build.props`, we should be able to use this folder for the artifacts with minimal perf impact +- Look for a folder with a `.sln` file in it, walking up the directory tree from the current project folder to the root. When a `.sln` file is found, put the artifacts folder in that same directory. We may need an additional intrinsic MSBuild function to do this, as the `GetDirectoryNameOfFileAbove` function doesn't support wildcards for the file name +- If neither a `Directory.Build.props` file nor a `.sln` file was found, then put the artifacts folder in the project folder. + +To override the default artifacts folder path, the `ArtifactsPath` property can be used. If `ArtifactsPath` is set, then `UseArtifactsOutput` will default to `true` and it won't be necessary to set both properties. `ArtifactsPath` should normally be set in a `Directory.Build.props` file. For example: + +```xml + + + $(MSBuildThisFileDirectory)\output + + +``` + +Under the artifacts folder will be nested folders for the type of the output, the project name, and the pivots, ie `\\\`. + +- Type of Output - This will be a value such as `bin`, `publish`, `intermediates`, or `package`. Projects will be able to override this value, for example to separate shipping and non-shipping artifacts into different folders + - `bin` will be the folder for the normal build output. Projects can override this with `ArtifactsOutputName` + - `publish` will be the folder for the publish output. Projects can override this with `ArtifactsPublishOutputName` + - `package` will be the folder for the package output, where .nupkg folders are placed. (Or should it be `packages`? Feedback welcome.) Projects can override this with `ArtifactsPackageOutputName` + - `intermediates` will be the folder for the intermediate output. (Or should we stick with `obj`? Feedback welcome.) +- The Project Name. This will ensure that each project has a separate output folder. By default this will be `$(MSBuildProjectName)`, but projects can override this with the `ArtifactsProjectName` property. If `ArtifactsPath` was not explicitly specified, and neither a `Directory.Build.props` nor a `.sln` file was found, then the artifacts folder will already be inside the project folder, and an additional Project Name folder will *not* be included in the output paths. Additionally, for package output, this folder won't be included in the path, so that all the `.nupkg` files that are built can be in a single folder. +- Pivots - This is used to distinguish between builds of a project for different configurations, target frameworks, runtime identifiers, or other values. It can be specified with `ArtifactsPivots`. By default, the pivot will include: + - The `Configuration` value (lower-cased, using the invariant culture) + - The `TargetFramework` if the project is multi-targeted + - The `RuntimeIdentifier`, if it was explicitly set (either in the project file or on the command line). The `RuntimeIdentifier` won't be appended if it was automatically set by the SDK, for example because `SelfContained` was set + +If multiple pivot elements are used, they will be joined by the underscore (`_`) character. For example, a multi-targeted project with the RuntimeIdentifer specified could have a pivot of `debug_net8.0-windows10.0.19041.0_win-x64`. For the `package` output type, the pivots will only be the `Configuration`, and other values won't be included in the path + +Some examples: + +- `.artifacts\bin\debug` - The build output path for a simple project when you run `dotnet build` +- `.artifacts\intermediates\debug` - The intermediate output path for a simple project when you run `dotnet build` +- `.artifacts\bin\MyApp\debug_net8.0` - The build output path for the `net8.0` build of a multi-targeted project +- `.artifacts\publish\MyApp\release_linux-x64` - The publish path for a simple app when publishing for `linux-x64` +- `.artifacts\package\release` - The folder where the release .nupkg will be created for a project + +## Customizing the output paths + +The `Directory.Build.props` and `Directory.Build.targets` files can be used to [customize builds](https://learn.microsoft.com/visualstudio/msbuild/customize-your-build#directorybuildprops-and-directorybuildtargets) with logic that will apply to multiple projects. However, they do not work very well to customize the output path. This is because custom output path logic usually depends on the contents of the project file, for example the `TargetFramework` value or a custom property such as `IsShipping`. `Directory.Build.props` is imported before the body of the project file, so it can't read those properties. On the other hand, `Directory.Build.targets` is imported after common targets, so it is too late to correctly set the output paths. + +There is already a `BeforeTargetFrameworkInferenceTargets` property which can be set to import a targets file after the project file is evaluated but before the `TargetFramework` is parsed. Custom output path logic may depend on the target framework, and it is better to write this logic in terms of the properties which are parsed out of the `TargetFramework` such as `TargetFrameworkIdentifier` and `TargetFrameworkVersion`. Because of this, we will add support for an `AfterTargetFrameworkInferenceTargets` property which can be used to inject a target file that will be imported after `TargetFramework` parsing. + +As an example of how to use this, a project that wants to put the output for "Shipping" projects in a separate folder could set the `IsShipping` property to true in each shipping project, and put the following in `Directory.Build.props`: + +```xml + + + true + $(MSBuildThisFileDirectory)\Directory.AfterTargetFrameworkInference.targets + + +``` + +The `Directory.AfterTargetFrameworkInference.targets` would include the following: + +```xml + + + Shipping\bin + Shipping\publish + Shipping\package + + +``` + +## Changing the default output path format + +We are proposing this new output format because they represent an improvement over the existing behavior. If they do represent an improvement, then we would like to change the default to use the new behavior, so that new projects and new users can get the improved behavior without having to know to opt in. However, there are several obstacles to changing the default. + +First of all, we don't want to affect existing projects that haven't opted in to the new behavior. To do otherwise would be a massive breaking change, as many CI setups, build scripts, custom build logic, etc. all depend on knowing where the output of a project is and would break if it changes. + +Often what we do in situations like this is to only make the new behavior the default for a new target framework. For example, projects that hadn't explicitly opted in or out would only get the new behavior if they targeted .NET 8 or higher. This would work for single-targeted projects, but if a project multi-targeted to .NET 7 and .NET 8, then it could be very confusing if the output for .NET 7 was in the traditional `bin` folder, but the output for .NET 8 was in an `.artifacts` folder that wasn't even in the project folder but was at the solution level. + +Ideally in that case we might want to use the new output path format by default for all target frameworks if any of them are .NET 8 or greater. Unfortunately this is not possible, as the output paths need to be determined during MSBuild evaluation, and at that point it's not possible to query the build of another target framework to see if it targets a given version of .NET or higher. + +So for multi-targeted projects, we'd be left with these options: + +- Default to the new output paths - too breaking +- Use different output path formats for different target frameworks - probably too confusing +- Default to the old output paths + +Another issue with defaulting to the new output path format is that it can produce output paths that are longer, which would make it more likely for projects to run up against path length limitations. .NET Maui projects in particular have hit this limitation in the past with nested Java builds in the intermediate output path, and has taken steps to shorten their paths. + +## Considerations + +### Using `.artifacts` for the folder name + +The default `.artifacts` folder name was chosen for a few reasons: + +- Folders starting with `.` are already ignored by the default item globs for .NET SDK projects. This is important so that if a project switches back and forth between output formats, the output path (and especially the intermediate output path, where C# source code files are generated) won't be included in the globs. +- The `.` prefix sorts the folder separately from the rest of the source folders + +There is also at least one drawback: + +- Folders starting with `.` are hidden in some contexts (MacOS finder for example). This will make it harder to find the output or run the app. + +### Finding the output path from scripts + +Today, the output path is likely hardcoded in many places that interact with MSBuild (build scripts, CI pipelines, etc.). If those are going to need to be updated, it would be better if they could be updated in a way that makes them resilient to output path changes. To do this, we should provide a way for a script to call in to MSBuild and get the output path value. This will be a separate design proposal, right now we have it tracked with the following work item: https://github.com/dotnet/msbuild/issues/3911 \ No newline at end of file From a0978709188de1e17313587bde928e3db5068503 Mon Sep 17 00:00:00 2001 From: Daniel Plaisted Date: Thu, 23 Feb 2023 19:53:12 -0500 Subject: [PATCH 08/10] Switch to obj folder, other updates --- accepted/2023/simplify-output-paths.md | 24 +++++++++++++++++------- 1 file changed, 17 insertions(+), 7 deletions(-) diff --git a/accepted/2023/simplify-output-paths.md b/accepted/2023/simplify-output-paths.md index b6d9fede3..4c430d521 100644 --- a/accepted/2023/simplify-output-paths.md +++ b/accepted/2023/simplify-output-paths.md @@ -8,6 +8,7 @@ Goals: - Provide simple and consistent output paths - Avoid cases where one output path is nested inside of another one - Avoid excessively deep directory heirarchies +- Allow output path customization when needed ## Motivation @@ -29,8 +30,7 @@ The recommended way to opt in to the new output path format will be to set the ` The artifacts folder will by default be named `.artifacts`. The convention we will use to determine which folder the artifacts folder should go in will be: - If a `Directory.Build.props` file was used in the build, put the artifacts folder in the same folder where `Directory.Build.props` was found, which will typically be at the root of a repo. Since MSBuild already probes for `Directory.Build.props`, we should be able to use this folder for the artifacts with minimal perf impact -- Look for a folder with a `.sln` file in it, walking up the directory tree from the current project folder to the root. When a `.sln` file is found, put the artifacts folder in that same directory. We may need an additional intrinsic MSBuild function to do this, as the `GetDirectoryNameOfFileAbove` function doesn't support wildcards for the file name -- If neither a `Directory.Build.props` file nor a `.sln` file was found, then put the artifacts folder in the project folder. +- If a `Directory.Build.props` file was not found, then put the artifacts folder in the project folder. To override the default artifacts folder path, the `ArtifactsPath` property can be used. If `ArtifactsPath` is set, then `UseArtifactsOutput` will default to `true` and it won't be necessary to set both properties. `ArtifactsPath` should normally be set in a `Directory.Build.props` file. For example: @@ -44,12 +44,12 @@ To override the default artifacts folder path, the `ArtifactsPath` property can Under the artifacts folder will be nested folders for the type of the output, the project name, and the pivots, ie `\\\`. -- Type of Output - This will be a value such as `bin`, `publish`, `intermediates`, or `package`. Projects will be able to override this value, for example to separate shipping and non-shipping artifacts into different folders +- Type of Output - This will be a value such as `bin`, `publish`, `obj`, or `package`. Projects will be able to override this value, for example to separate shipping and non-shipping artifacts into different folders - `bin` will be the folder for the normal build output. Projects can override this with `ArtifactsOutputName` - `publish` will be the folder for the publish output. Projects can override this with `ArtifactsPublishOutputName` - `package` will be the folder for the package output, where .nupkg folders are placed. (Or should it be `packages`? Feedback welcome.) Projects can override this with `ArtifactsPackageOutputName` - - `intermediates` will be the folder for the intermediate output. (Or should we stick with `obj`? Feedback welcome.) -- The Project Name. This will ensure that each project has a separate output folder. By default this will be `$(MSBuildProjectName)`, but projects can override this with the `ArtifactsProjectName` property. If `ArtifactsPath` was not explicitly specified, and neither a `Directory.Build.props` nor a `.sln` file was found, then the artifacts folder will already be inside the project folder, and an additional Project Name folder will *not* be included in the output paths. Additionally, for package output, this folder won't be included in the path, so that all the `.nupkg` files that are built can be in a single folder. + - `obj` will be the folder for the intermediate output. +- The Project Name. This will ensure that each project has a separate output folder. By default this will be `$(MSBuildProjectName)`, but projects can override this with the `ArtifactsProjectName` property. If `ArtifactsPath` was not explicitly specified, and a `Directory.Build.props` was not found, then the artifacts folder will already be inside the project folder, and an additional Project Name folder will *not* be included in the output paths. Additionally, for package output, this folder won't be included in the path, so that all the `.nupkg` files that are built can be in a single folder. - Pivots - This is used to distinguish between builds of a project for different configurations, target frameworks, runtime identifiers, or other values. It can be specified with `ArtifactsPivots`. By default, the pivot will include: - The `Configuration` value (lower-cased, using the invariant culture) - The `TargetFramework` if the project is multi-targeted @@ -60,7 +60,7 @@ If multiple pivot elements are used, they will be joined by the underscore (`_`) Some examples: - `.artifacts\bin\debug` - The build output path for a simple project when you run `dotnet build` -- `.artifacts\intermediates\debug` - The intermediate output path for a simple project when you run `dotnet build` +- `.artifacts\obj\debug` - The intermediate output path for a simple project when you run `dotnet build` - `.artifacts\bin\MyApp\debug_net8.0` - The build output path for the `net8.0` build of a multi-targeted project - `.artifacts\publish\MyApp\release_linux-x64` - The publish path for a simple app when publishing for `linux-x64` - `.artifacts\package\release` - The folder where the release .nupkg will be created for a project @@ -94,6 +94,10 @@ The `Directory.AfterTargetFrameworkInference.targets` would include the followin ``` +## Specifying the output path via command line + +The .NET CLI supports an `--output` parameter to specify the output path for various commands. This [does not work well](https://learn.microsoft.com/en-us/dotnet/core/compatibility/sdk/7.0/solution-level-output-no-longer-valid) for solutions, since it results in multiple projects building to the same folder. We will add an `--artifactspath` parameter that sets the `ArtifactsPath` property. Because each project will be built into a separate subfolder, this will be a safer option to use for solution builds. + ## Changing the default output path format We are proposing this new output format because they represent an improvement over the existing behavior. If they do represent an improvement, then we would like to change the default to use the new behavior, so that new projects and new users can get the improved behavior without having to know to opt in. However, there are several obstacles to changing the default. @@ -127,4 +131,10 @@ There is also at least one drawback: ### Finding the output path from scripts -Today, the output path is likely hardcoded in many places that interact with MSBuild (build scripts, CI pipelines, etc.). If those are going to need to be updated, it would be better if they could be updated in a way that makes them resilient to output path changes. To do this, we should provide a way for a script to call in to MSBuild and get the output path value. This will be a separate design proposal, right now we have it tracked with the following work item: https://github.com/dotnet/msbuild/issues/3911 \ No newline at end of file +Today, the output path is likely hardcoded in many places that interact with MSBuild (build scripts, CI pipelines, etc.). If those are going to need to be updated, it would be better if they could be updated in a way that makes them resilient to output path changes. To do this, we should provide a way for a script to call in to MSBuild and get the output path value. This will be a separate design proposal, right now we have it tracked with the following work item: https://github.com/dotnet/msbuild/issues/3911 + +## Looking for a .sln file in the convention + +If we ever enable the new output path format by default, we would want it to work for projects without a Directory.Build.props file. In that case we would want to update the convention so that if a Directory.Build.props file isn't found, it will look for a folder with a `.sln` file in it, walking up the directory tree from the current project folder to the root. When a `.sln` file is found, the artifacts folder would be put in that same directory. + +To implement this, we would probably need an additional intrinsic MSBuild function, as the `GetDirectoryNameOfFileAbove` function doesn't support wildcards for the file name \ No newline at end of file From e542ce53e732570dfe01b4b084bb78b50c3e407e Mon Sep 17 00:00:00 2001 From: Daniel Plaisted Date: Thu, 2 Mar 2023 18:38:35 -0500 Subject: [PATCH 09/10] Apply feedback --- accepted/2023/simplify-output-paths.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/accepted/2023/simplify-output-paths.md b/accepted/2023/simplify-output-paths.md index 4c430d521..e6735d95d 100644 --- a/accepted/2023/simplify-output-paths.md +++ b/accepted/2023/simplify-output-paths.md @@ -45,9 +45,9 @@ To override the default artifacts folder path, the `ArtifactsPath` property can Under the artifacts folder will be nested folders for the type of the output, the project name, and the pivots, ie `\\\`. - Type of Output - This will be a value such as `bin`, `publish`, `obj`, or `package`. Projects will be able to override this value, for example to separate shipping and non-shipping artifacts into different folders - - `bin` will be the folder for the normal build output. Projects can override this with `ArtifactsOutputName` + - `bin` will be the folder for the normal build output. Projects can override this with `ArtifactsBinOutputName` - `publish` will be the folder for the publish output. Projects can override this with `ArtifactsPublishOutputName` - - `package` will be the folder for the package output, where .nupkg folders are placed. (Or should it be `packages`? Feedback welcome.) Projects can override this with `ArtifactsPackageOutputName` + - `package` will be the folder for the package output, where .nupkg files that are created when packing the project are placed. (Or should the folder name be `packages`? Feedback welcome.) Projects can override this with `ArtifactsPackageOutputName` - `obj` will be the folder for the intermediate output. - The Project Name. This will ensure that each project has a separate output folder. By default this will be `$(MSBuildProjectName)`, but projects can override this with the `ArtifactsProjectName` property. If `ArtifactsPath` was not explicitly specified, and a `Directory.Build.props` was not found, then the artifacts folder will already be inside the project folder, and an additional Project Name folder will *not* be included in the output paths. Additionally, for package output, this folder won't be included in the path, so that all the `.nupkg` files that are built can be in a single folder. - Pivots - This is used to distinguish between builds of a project for different configurations, target frameworks, runtime identifiers, or other values. It can be specified with `ArtifactsPivots`. By default, the pivot will include: From 134b8b0f21bee005b38fac19c31acabb7cef8e60 Mon Sep 17 00:00:00 2001 From: Daniel Plaisted Date: Thu, 9 Mar 2023 13:44:25 -0500 Subject: [PATCH 10/10] Match normal pattern for CLI parameters --- accepted/2023/simplify-output-paths.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/accepted/2023/simplify-output-paths.md b/accepted/2023/simplify-output-paths.md index e6735d95d..192b823b6 100644 --- a/accepted/2023/simplify-output-paths.md +++ b/accepted/2023/simplify-output-paths.md @@ -96,7 +96,7 @@ The `Directory.AfterTargetFrameworkInference.targets` would include the followin ## Specifying the output path via command line -The .NET CLI supports an `--output` parameter to specify the output path for various commands. This [does not work well](https://learn.microsoft.com/en-us/dotnet/core/compatibility/sdk/7.0/solution-level-output-no-longer-valid) for solutions, since it results in multiple projects building to the same folder. We will add an `--artifactspath` parameter that sets the `ArtifactsPath` property. Because each project will be built into a separate subfolder, this will be a safer option to use for solution builds. +The .NET CLI supports an `--output` parameter to specify the output path for various commands. This [does not work well](https://learn.microsoft.com/en-us/dotnet/core/compatibility/sdk/7.0/solution-level-output-no-longer-valid) for solutions, since it results in multiple projects building to the same folder. We will add an `--artifacts-path` parameter that sets the `ArtifactsPath` property. Because each project will be built into a separate subfolder, this will be a safer option to use for solution builds. ## Changing the default output path format