Projects need to be able to specify manifest location - determining it by "navigating up" is problematic

[aggieNick02]

Is your feature request related to a problem? Please describe.
Currently the manifest used for a project is determined by looking next to the project, and then navigating up directories, until vcpkg.json is found. In a standard flat solution which has many projects all at the same level, this is really inflexible. It requires either manifest and installed package duplication, or reorganizing solution file structure. It is not unreasonable or uncommon to have a solution with several related projects that need to use one of two triplets (e.g., some projects use x64-windows, some use x64-windows-static). There is not a good option available for using manifests in such solutions.

Proposed solution
Allow projects to specify the directory that contains their vcpkg.json file

Describe alternatives you've considered
As mentioned in the problem statement, you can put individual per-project vcpkg.json files next to each project, but this means any dependency changes must be copy pasted, and duplicates the installed ports for each project

Additional context
By combining this feature with allowing overlay port and overlay triplet directories to be specified within a vcpkg.json (#12289), manifests would be a viable option for many more clients. Instead, clients have to come up with their own solutions, and the recent versioning capability is not available to them.

[autoantwort]

I assume you use Visual Studio. I don't understand your problem. Why can you not use one vcpkg.json file for all projects in your solution? You can use one vcpkg.json file with multiple triplets since #19767. You can set the overlay-ports dir via additional arguments in the vcpkg visual studio settings

[aggieNick02]

My apologies. I wasn't able to determine that either of those were possible based on looking at open issues and documentation. Yes, I'm using visual studio.

#12289 was opened by @RAS2019 in July 2020 and says manifest mode doesn't support overlay ports and is still open. But it sounds like the issue should be closed since it can be accomplished via vcpkg visual studio settings? Or is there a difference between that issue and what I'm describing that I'm not seeing?

I had already read #19767 but it didn't quite click for me that it meant different triplets would now work. Thank you for pointing that out.

I do think though that there are other use cases besides the different triplets where having the manifest location based off project location still makes things awkward. A simple example would be a solution with a single project with two different configurations, where each configuration needs to use the same dependency (e.g., ffmpeg) but with different, conflicting features enabled (e.g., gpl vs no gpl feature). The natural way to do this would be to point one configuration to a vckpg.json specifying ffmpeg with gpl, and the other to a vcpkg.json not specifying gpl .

There may be some tricks you can play with overlay triplets to somehow accomplish this, but it's not nearly as natural or straightforward as just pointing to different manifests. I guess my point is that while basing the manifest off project location is a reasonable default option, allowing a project to explicitly specify it enables usages that are otherwise difficult/awkward/impossible.

[aggieNick02]

Another example in case it helps... this one I'm hitting right now. My solution has 27 projects, a decent chunk of which use vcpkg. Most projects are typical using the dynamic runtime and all kinds of dependencies (e.g., triplet x64-windows), but I have a couple utilities that need the c++ runtime linked statically (triplet x64-windows-static). The set of dependencies for the typical projects is pretty large (lots of boost, ffmpeg, etc). The set of dependencies for the little utilities using static runtime is small.

With my old manual vcpkg build/install script, I built the dependencies I needed for each triplet, which meant only a few dependencies had to be built and installed for x64-windows-static. But moving to manifests, all dependencies have to be built for all triplets (building all of boost and ffmpeg unnecessarily is a lot of wasted time, on the order of 1-2 hours), and I have to hope I don't run into some unused dependency that has an issue with x64-windows-static.

[aggieNick02]

Just saw the CLI has --x-manifest-root - maybe this is all I need. I'll play with it and report back.

[aggieNick02]

Alright, so --x-manifest-root isn't enough because it doesn't tell visual studio where to look for the manifest - it only tells vcpkg. But I can set VcpkgManifestRoot in the vcxproj and it works pretty well. It gets me halfway to how I'd like things to work.

(In my setup, vcpkg is a subrepo in my repo, so I want all of my projects in that repo to point to use the vcpkg.exe in the folder that sits next to them, and to use one of the manifests that are present at the top-level in the repo too. With VcpkgManifestRoot, I can get it to use those manifests just fine.)

The remaining problem is getting my projects to use the local vcpkg folder. This means I need to use per-project integration instead of site-wide. Using the nuget mechanism for this works fine, but isn't something I want to check into source code control because the name of the nuget package is based on its local path on disk. The vcpkg FAQ (https://vcpkg.readthedocs.io/en/latest/about/faq/) says I can accomplish the same thing by using the lower level mechanism of just importing the vcpkg.targets file in the vcxproj. I tried this, and the project acts like it has vcpkg integration, but it ignores values I set for various vcpkg variables - namely, VcpkgManifestRoot is no longer honored. It always looks in the project directory and up regardless of the variable's value.

Is importing the vcpkg.targets to get per-project integration still supported or perhaps somehow broken right now? Here's a screenshot of VcpkgManifestRoot not being honored when I do it:

[dg0yt]

Most projects are typical using the dynamic runtime and all kinds of dependencies (e.g., triplet x64-windows), but I have a couple utilities that need the c++ runtime linked statically (triplet x64-windows-static). The set of dependencies for the typical projects is pretty large (lots of boost, ffmpeg, etc). The set of dependencies for the little utilities using static runtime is small.

Did you consider using a single custom triplet based on x64-windows, which sets static linkage for the desired set of little utilities?

[aggieNick02]

Do you mean to do that and then use that triplet for all of my projects to avoid having to build all of the dependencies with the static runtime? The dependencies of the little utilities intersect with the dependencies of the typical projects, so there are some dependencies that need to be built both with static and dynamic runtime linkage, which isn't workable within a single triplet.

Apologies if I'm misunderstanding the suggestion and you meant something different.

[dg0yt]

The dependencies of the little utilities intersect with the dependencies of the typical projects, so there are some dependencies that need to be built both with static and dynamic runtime linkage, which isn't workable within a single triplet.

Yes, this isn't workable then.

[aggieNick02]

So debugging the failure by turning up build verbosity made clear that my problem was that the vcpkg.targets import was in the vcxproj file prior to where the vcpkg UI places its variable definitions in the vcxproj file. Moving the import to the bottom of the vcxproj file seemed to fix this. I'm guessing the fact that the nuget stuff gets added to the end of the vcxproj is why it doesn't have this problem.

So I think with all of what I've learned here, I can actually do everything I want now. I've got it working in some toy examples so I'll try it on my real solution and report back.

[aggieNick02]

I've played enough that with the various things pointed out by @autoantwort in addition to setting $(VcpkgManifestRoot) manually in the vcxproj file allows me to do what I wanted. I have everything working now except for a corner case involving build order (manifests installing dependencies too late), and I've opened a bug for that at #20830 . I think this issue is good to close. Thanks for the discussion and ideas.