WIP: meson on windows (clang.exe)

[h-vetinari]

Version of #246 with vanilla clang (not the MSVC-compatible clang-cl.exe).

[conda-forge-webservices]

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

[h-vetinari]

@rgommers
scipy/scipy#19289 did not change the error for unuran:

[1023/1629] Linking target scipy/stats/_unuran/unuran_wrapper.cp311-win_amd64.pyd
FAILED: scipy/stats/_unuran/unuran_wrapper.cp311-win_amd64.pyd 
"clang.exe" @scipy/stats/_unuran/unuran_wrapper.cp311-win_amd64.pyd.rsp
clang: error: no such file or directory: 'C:bldscipy-split_1695551724191_h_envlibspython311.lib'

[rgommers]

Too bad. I think that this is simply a matter of the RSP format not working at all for lld-link.exe on Windows, and unuran_wrapper simply being the first target that exceeds the link line limit for a response file being used. Here is how Meson determines whether to use a .rsp file:

def get_rsp_threshold() -> int:
    '''Return a conservative estimate of the commandline size in bytes
    above which a response file should be used.  May be overridden for
    debugging by setting environment variable MESON_RSP_THRESHOLD.'''

    if mesonlib.is_windows():
        # Usually 32k, but some projects might use cmd.exe,
        # and that has a limit of 8k.
        limit = 8192
    else:
        # On Linux, ninja always passes the commandline as a single
        # big string to /bin/sh, and the kernel limits the size of a
        # single argument; see MAX_ARG_STRLEN
        limit = 131072
    # Be conservative
    limit = limit // 2
    return int(os.environ.get('MESON_RSP_THRESHOLD', limit))

And here are link line lengths:

$ rg _LINKER build/build.ninja | awk '{print length "  " $2}' | sort -n | tail 
2629  scipy/optimize/_highs/libipx.a:
2676  scipy/optimize/_highs/libbasiclu.a:
2957  scipy/special/libcdflib.a:
2957  scipy/special/libcephes.a:
3554  scipy/linalg/_interpolative.cpython-310-x86_64-linux-gnu.so:
4868  scipy/interpolate/libfitpack_lib.a:
5996  scipy/sparse/linalg/_eigen/arpack/libarpack_lib.a:
8315  scipy/optimize/_highs/libhighs.a:
12973  scipy/sparse/linalg/_dsolve/libsuperlu_lib.a:
18368  scipy/stats/_unuran/unuran_wrapper.cpython-310-x86_64-linux-gnu.so:

So probably this can be worked around by setting MESON_RSP_THRESHOLD to a value close to 32k?

[h-vetinari]

So probably this can be worked around by setting MESON_RSP_THRESHOLD to a value close to 32k?

That sounds like an excellent analysis, but unfortunately it didn't seem to change anything...

It seems that meson has metadata for the linker to advertise rsp compatibility, and it seems that just falls through to the default, which is set to always true on windows.

I'm going to try patching that in conda-forge/meson-feedstock#86

[h-vetinari]

I'm going to try patching that in conda-forge/meson-feedstock#86

That didn't seem to have any effect either 😑

[rgommers]

Then I'm out of workaround ideas - I guess it needs a proper fix in Meson. Unfortunately I don't have a Windows machine at hand, and this isn't a debug-via-CI kinda problem.