[LibLZMA] automatic configuration #6000
Conversation
|
please when looking diff, enable the ignore whitespace. |
|
I like this approach. The idea is basically to provide a config and let find_package find that one first before going into the cmake internals. The last time I defined an exportet package I used code like this (DESTINATIONs must of course be changed): with a config.cmake.in file like this: theoretically this could be extended to arbitrary ports |
[OpenCV] add an explicit dependency on LibLZMA for static linking
|
😉 |
|
My only concern with this approach is that if people don't realise they need to add |
|
Here are the test results from the current CI system:
Most libs failed and it looks like this error: |
|
@NancyLi1013 thanks. I reworked the install path for the include files, it should be better now. |
|
@cenit - ah yes of course, sounds like a good plan! |
|
related: #616 |
Attached failure logsHere are the current CI results:
As usual, |
|
Hmm currently removing the extra "d"s from conflicting ports in #5543 and noticed something interesting. rockdbs and folly define basically the same findsnappy.cmake to find snappy for debug and release. Could we not do the same, but instead having the findsnappy in the consuming ports have it be installed by the source port in an appropriate vcpkg cmake search folder? this should also solve the required call to find_package in config mode |
|
i think we already discussed creating a common folder for custom cmake module files, store them there inside that single folder instead of inside each share/portname folder and add that folder to the CMAKE_MODULE_PATH directly within vcpkg toolchain. I don't remember what blocked that idea... (note that I am talking about module files, config files should and must be kept inside specific port folders) |
|
@cenit Does it directly mess with cmake modules files included normally within ports/packages? Would be great if you could find the issue with the discussion about it. |
|
appending (and not prepending) that folder to the searched path should not mess. It would be the last resort for a find_package(). |
|
I modified a little bit the TIFF wrapper and fixed a typo in a symbol, I hope it could be enough to fix most of the issues. In fact, I also have some doubts on the logs. Just open as an example liblzma:x64-linux, everything went fine! Why regression? |
|
@cenit probably post build validation which is not included in the CI logs. |
|
any update on the error logs? |
…are/lib folder written in lowercase
…arget public definition
|
any news why CI is not running anymore here? |
|
@vicroms selene on arm was definitely broken, since it was using intrinsics closely related to x86 arch. Since it was definitely broken also before I doubt that was the regression marking the red cross, but since we are just working on selene now it to finish this work I don't know what could be the problem. Waiting for the logs, on monday ofc :) |
Replacing custom find_package logic is the better way. Any custom find logic can at best generate havok in a system as vcpkg where files have a clear default location. I'm not a cmake expert, but it seem as if find_package has a bad design because it combines "find files" and "pass configuration options" which are tow steps that should really be separated. |
the only thing find_package does is (excluding any additional options):
The rest of the behavior is complete up to the find module or the config. The rest of the behavior is complete up to the included *.cmake file. That is also why vcpkg_cmake_wrapper.cmake is totally superflous because you got get the same behavior by defining a config or find module. The main difference between the config and the find module is: the config is generated by the project to be consumed. So it knows where it has been installed to and how it has been build. It thus encodes the paths to the libraries and required definitions within the config. The find module on the other hand is supplied by the consumer and can only rely on find_library (or others like find_path, _programm etc.) calls to discover its dependencies. |
|
@Neumann-A the problem with LZMA is that it's often a second-layer dependency, unfortunately nested in shared config files for some projects instead of specific target file (a common file instead of a specific build-type file) About the broad scope of this PR: it is already a second level disentanglement of a long work pursued here on vcpkg, which predates many things and people. Unfortunately dependencies work like this and it's difficult to disentangle broken ports when fixing a very basic one. If you had read the discussion here or the code you'd know that the "CONFIG" tag is not required anymore to opt-in the new LZMA mechanism since commit 375a8bc. It was used at the beginning as an opt-in to the new system, to check step-by-step if it could even work (I was sure but I needed proof). Recently it became superfluous, since it is mandatory for every LZMA request, but it was left "on" for vcpkg-supplied build scripts and patches already in place, and discussed with Victor. I recently removed them since they were adding too much noise to this PR, being absolutely not necessary. #5543 is a flying target and unsuitable for any production environment IMHO, so it's totally out of question for me, for now. It's already a huge risk to live on the master here, since it's not considered mandatory to keep it "sane" even on a basic level (do you remember your patch that you didn't even test and was accepted too quickly? Any people cloning vcpkg during that time found a broken project, distributed by MS, and maybe decided that it was a toy and totally not deserving any other time. Wow! If many people took their time to write an issue, just imagine how many didn't even write and just abandon it...) Declaring vcpkg_cmake_wrapper.cmake totally superfluous means that you have not fully understood the internal mechanisms of vcpkg, since that's the only point to intercept calls for find_package resolving to an official module. How would you intercept calls to a CMake "official" module otherwise? |
|
@vicroms when CI will be over on this PR, please let me know results. I hope to have fixed many regressions (thanks to public CI), I couldn't understand results anymore and so now I need your help :) |
so its that issue:
I fully understand this issue. You could have done a local CI build of master where everything "works/compiles" and then start a search for "lzma" in the installed/share folders and find all the incomplete configs and just fix them for the time being. (That would have been a simple replace operation within all configs.)
Sorry your are touching 11 ports in this PR and 8 or so could be free standing PRs correcting behavior independent of what has been done to correct LZMA. In my opinion you should just split those off and get them merged into master already. (If you do that they probably get merged in less then 2 days.)
yeah it is still a WIP because I spend too much time trying to fix find_package. Should have started with find_library from the beginning and currently trying to find all the corner cases which must be properly considered. But let me tell you: It will be the only way to fix all those LZMA cases within vcpkg with a reasonable amount of work which also works for multi configuration generators. There are many libraries which do not even use find_package but use find_library instead and a lot of them don't even have the names right either missing or adding debug suffixes. Furthermore all find_package implementation also rely on find_library. (The alternativ would be find modules for all affected ports.)
From https://cmake.org/cmake/help/v3.14/command/find_package.html
so vcpkg could have just define/appended to CMAKE_MODULE_PATH to get the same behavior as vcpkg_cmake_wrapper.cmake with the benefit that people actually know how that works and how to use it. I am strongly for deprecating vcpkg_cmake_wrapper and actually introducing a module path. With good find modules we could probably deprecate the _find_package "hack" which is as far as i know undocumented cmake behavior. (yeah I know I use it myself in my PR but I am not strictly against that "hack" because it is really good for debugging issues.) |
what do you think people do to work ;) of course that's the way we use it now, it was just deserving a proper fix
it's just enabling thanks to a proper LZMA port that revealed those problems. I don't need them, but it's nice to have them fixed.
why do you call it hack? You know that every overloaded function in CMake is kept in the original form with an underscore prepended? That's documented and well known, I'd not call it a hack |
I'd not mess with CMAKE_MODULE_PATH. That's the perfect way to break many projects that use it. On the contrary, I appreciate @ras0219-msft idea of |
I don't think so. From my experience there can't be a solution that works for everything, and here in particular since it's a bazaar by definition. It was a long journey, made by small steps, one after the other, on many fronts (compiler, tooling, many volunteers fixing code, ...), but it wouldn't be possible in any other easier way. |
That was the same for my PR and that’s why I split them from the main PR. It is a way of preparing master for another PR.
What happens if an external toolchain also overwrites the same function? Then _find_package will no longer be what you think it will be, infinite loop included (https://crascit.com/2018/09/14/do-not-redefine-cmake-commands/)
It probably does not break as many as you think. Also it is easily checked if CMAKE_MODULE_PATH has been altered and if delegating to a different find module is necessary.
And I thought you are a theoretical physicist ;). Have you given up on the theory which unifies everything? Even though it is a bazaar I see a lot of common usage patterns within all ports which can be dealt with on a general basis.
A long journey which also included a lot of trial and error just like my PRs ;) and since your a speaking of small steps do you mind splitting your PR ;) And there was probably also a lot of heated discussions on how to move forward but just how it is. People have different views on how to solve the same problem and either the maintainers or the crowed will decide how it will be solved.
The PR actually reached a point were it completly worked with just fixing find_package but I decided to restart from scratch because I could do a lot more by overwritting find_library (e.g. searching for libraries with common debug suffixes) with less changes to different ports. Unfortunally there where some side effects which had to be dealt with: like IMPORT_LOCATION not accepting generator expression under some circumstances and so on. That is not something you can figure out by reading the docs alone. |
😆 good catch... @Neumann-A Thanks for the long explanation of your point of view. I really appreciated reading it. Now I think it’s better to go back in topic. I hope also to have some feedback about regressions here (maybe only vxl? who knows :) ) |
|
@vicroms I merged the master. SInce the vxl problem should be gone, I'm targeting a green mark 😄 |
|
No more regressions! 🎉 As a side effect this PR also enables some new ports:
|
|
@vicroms Thanks. To be honest (said only for debug purposes) in reality I enabled more ports than those you listed. For example OpenSSL was not working on any form of arm64, but it's not listed there (arm64-windows, arm64-uwp, ...) |
Different approach to the same problem as #5625
Just as curiosity, I tried to improve our own CMakeLists.txt to export targets and better deal with debug/release configurations, while trying to be as much backwards compatible as possible.
What happens now? Is everything broken? Waiting for the CI to tell the truth!