Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix build fail with i686 toolchain #33

Closed
wants to merge 1 commit into from
Closed

Conversation

volks73
Copy link

@volks73 volks73 commented Dec 13, 2017

As noted in #31, I was experiencing a similar linker error and failure with the i686 toolchain. I had some time to dig into it a little bit and I believe I have a resolution for this issue that I would like to share.

The problem appeared to be the calling convention for the kernel32 DLL function and a u64 being used even with the i686 (32-bit) toolchain. Apparently on Windows, a DWORD is always 32-bit (u32) even on a 64-bit system (x86_64). The GetStdHandle declaration was using a u64.

Additionally, the system calling convention needed to be added because in 64-bit environments (x86_64), the calling convention is C but in 32-bit environments the calling convention is stdcall, as noted in the Rust book's FFI section:

Most of the abis in this list are self-explanatory, but the system abi may seem a little odd. This constraint selects whatever the appropriate ABI is for interoperating with the target's libraries. For example, on win32 with a x86 architecture, this means that the abi used would be stdcall. On x86_64, however, windows uses the C calling convention, so C would be used. This means that in our previous example, we could have used extern "system" { ... } to define a block for all windows systems, not only x86 ones.

The system calling convention automatically handles the difference depending on the environment/toolchain. Without this notation, the linker would fail because the calling convention would not match the declarations for all three functions.

After making these changes, I was able to successfully build the crate using the stable-i686-pc-windows-msvc toolchain with a x86 environment (Developer Prompt for x86) and pass all tests. I was also able to successfully build the crate with the stable-x86_64-pc-windows-msvc. I used the following commands from the x86 Developer Prompt on a 64-bit architecture running Windows 10 to build and pass the tests that were previously failing to build because of the linker error:

C:\set RUSTUP_TOOLCHAIN=stable-i686-pc-windows-msvc
C:\cargo test
...
test result: ok. 56 passed; 0 failed, 0 ignored; 0 measured; 0 filtered out
...
test result: ok. 18 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

The problem appeared to be the calling convention of of the kernel32 DLL
function and u64 being used even with i686 (32-bit) toolchain. On
Windows, a `DWORD` is always 32-bit (u32) even on a 64-bit system
(x86_64). The `GetStdHandle` declaration was using a u64.

Additionally, the `system` calling convention needed to be added because
in 64-bit environments (x86_64), the calling convention is "C" but in
32-bit environments the calling convention is "stdcall". The `system`
calling convention automatically handles the difference depending on the
environment/toolchain.
extern {
fn GetStdHandle(handle: u64) -> *const i32;
extern "system" {
fn GetStdHandle(handle: u32) -> *const i32;
fn SetConsoleMode(handle: *const i32, mode: u32) -> bool;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What a mess with handle types.

@volks73
Copy link
Author

volks73 commented Apr 15, 2018

As mentioned in #31, this pull request can be ignored.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants