Unix support for pre-generated src/bindings.rs

[MarijnS95]

For https://github.com/microsoft/windows-rs/discussions/566#discussioncomment-701806.

Currently all platform-specific decisions happen at code generation time, leading to the pre-generated, checked-in (and part of the crate!) source file src/bindings.rs to only work on Windows. This is not ideal for users who are seeking to use windows-rs for cross-platform COM bindings (pretty much only need the generated types, vtables and plumbing), which otherwise has been a pleasure to work with.

Fortunately the only difference (for now) is in native functions. These are #[link]'ed into Windows-specific binaries when generated for Windows, and unimplemented!() on others.

In this issue I'd like to describe and discuss some possible ways to solve that. Additionally some of these functions (those related to BSTR, see the discussion linked above) should be replaced by a manual implementation at some point, but that's for a different time.

Generate two files

We can generate two files (src/{unix,windows}/bindings.rs) and conditionally include them based on the platform.

See an initial approach to this at https://github.com/MarijnS95/windows-rs/compare/unix-bindings

Downsides

• Only functions are different, the rest of the file contains massive duplication;
• Lots of churn to chain the desired target/platform all the way down into the right generation functions. Perhaps this can be passed through struct Gen (is that how it was intended to be used in the first place?) but it looks like that incurs many extra changes too (particularly to transform some gen into another gen while retaining the desired target);
• proc_macro cannot take extra arguments. Having a duplicate bindings_windows! and bindings_unix! isn't ideal but we probably have to resort to proc_macro2 to resolve this, resulting in even more churn.

Use #[cfg] to switch between implementations

This is much easier and only requires changing the implementation of types/function.rs to emit #[cfg(windows)] and #[cfg(not(windows))] instead of picking an implementation at generation time.

See: https://github.com/MarijnS95/windows-rs/compare/unix-bindings-cfg

Downsides

• This makes for extra code in generated bindings. Not a problem for src/bindings.rs, but windows.rs in the output directory can potentially increase dramatically in size, despite being specific to the current target it was generated for.

Combine above methods

(Not tried this yet)

Emit native function bodies when building windows.rs, while emitting cross-compatible bodies with #[cfg] when generating src/bindings.rs.

Downsides

• Massive change/churn like the first option of generating two files, to pass down this "operation mode" all the way into function generation.

Functions in a separate file

(Not tried this yet)

A fourth solution is to emit functions to a separate file that is selected with #[cfg] at runtime, and reexport the functions from bindings.rs/windows.rs. This does not suffer blown-up files due to many #[cfg] switches nor massive duplication if the entire bindings.rs file were generated for every platform.

Downsides

• Probably lots of ugly, specific code and churn to take functions out of normal generation and put them in a separate place - as well as emitting reexport statements in its place;
• More than just functions might become platform specific in the future (but I certainly hope not!), that would have to go through the same mechanism.

---

Let me know what you think! I can clean up and submit the suggested approach when we have a decision :)

[kennykerr]

Thanks @MarijnS95, I have not had a moment to investigate this yet. I just wanted to set expectations again that while I do not see any reason to prevent unusual uses for the Windows crate, I am generally opposed to anything that will complicate it further that does not directly benefit the project's primary focus. The Windows crate is already pretty complicated. As I work on new features or fixes, I am always on the lookout for ways to simplify and pay back the debt in complexity. I would thus be very reluctant to take on any changes that would add complexity and not directly benefit the use of the Windows crate for calling or implementing APIs on Windows. I see the cross-platform potential but having gone through that exercise before with cppwinrt and xlang, the benefits are not broadly applicable and can add considerable complexity.

[MarijnS95]

I just wanted to set expectations again that while I do not see any reason to prevent unusual uses for the Windows crate, I am generally opposed to anything that will complicate it further that does not directly benefit the project's primary focus.

I totally understand, hence you're looking at this issue with a few possible approaches and an eye towards longevity instead of rushing one of the linked branches into a PR.

Regarding the rest of your reply, are you able to comment on whether any of the above approaches count as too much technical debt or complexity? We're this close to using windows-rs for autogenerated COM bindings across all platforms.

https://github.com/MarijnS95/windows-rs/compare/unix-bindings-cfg shouldn't be too bad, it's mostly shuffling around a few bits in function generation, to move cfg!(windows) to compile-time for the generated file. Is there any generate! macro you're doing performance regression tests (generation and build time) with?

OTOH in order to support my final usecase as outlined in https://github.com/microsoft/windows-rs/discussions/566#discussioncomment-701806 - DXC - more work needs to go in: in particular 32-bit widestring support and implementations for BSTR functions given its format defined here. Can you comment on that in the linked discussion too?

[wez]

I have a related but perhaps simpler question/use-case: I'd like to be able to use this crate to generate rust constants for eg: virtual key codes on windows, but use them from unix systems.

The rationale is that ConPTY has an enhanced keyboard input/output escape sequence built around win32 VK key codes and it is currently "impossible" for a unix program to know those codes and either produce or consume those escape sequences in scenarios where unix is either the client of a Windows host, or the host to a Window client.

In this scenario I wouldn't want any linkage at all.
Is that supportable?

Do you have other recommendations on how I could automate producing those definitions?

[MarijnS95]

@wez In that case you don't want to use the windows crate at build/runtime at all, just the generated constants in the output directory which should theoretically be able to stand completely on their own. Decoupling that however seems more work than it's worth, so you're probably looking for one of the approaches above.

[kennykerr]

While I am sympathetic to this idea, having explored it in C++, I don't have the bandwidth to support something like this at the moment as it adds a non-trivial amount of complexity. Perhaps once the Windows crate is feature-complete we can reopen this discussion.

[MarijnS95]

having explored it in C++

I feel like Rust makes it significantly easier to write cross-platform code, having no obscure headers and breaking compiler differences.

This is not supposed to add much complexity at all, in fact for now moving cfg!(windows) from the generator into the generated code (https://github.com/MarijnS95/windows-rs/compare/unix-bindings-cfg) is all that is needed. You'll have to think about something similar anyway to support multiple architectures with pre-generated and checked-in bindings and it seems to be a good idea to discuss this up-front. This time I created an issue to discuss possible approaches (which appears to be what is desired) and it has been closed without any discussion at all.

Otherwise, if this crate is not open to support non-Windows at all, why is that cfg!(windows) there in the first place? To make the Linux CI succeed and falsely claim that this crate could possibly work on Linux? Why is this crate tested on Linux at all?

[kennykerr]

The reason Linux builds are part of CI is purely to enable compiler testing - to enable testing and profiling of the compiler when the compiler developers are predominantly running on non-Windows machines and when profiling tools are only available on other platforms.