Idiomatic bool marshalling

[saul]

Win32 BOOL should be represented as [MarshalAs(UnmanagedType.Bool)] bool. C bool should be represented as [MarshalAs(UnmanagedType.U1)] bool.

The Boolean data type has multiple representations in unmanaged code. When the MarshalAsAttribute is not specified, the default marshaling behavior for the Boolean data type is System.Runtime.InteropServices.UnmanagedType. This is a 32-bit integer, which is not appropriate in all circumstances. The Boolean representation that is required by the unmanaged method should be determined and matched to the appropriate System.Runtime.InteropServices.UnmanagedType. UnmanagedType.Bool is the Win32 BOOL type, which is always 4 bytes. UnmanagedType.U1 should be used for C++ bool or other 1-byte types.

Source: https://docs.microsoft.com/en-us/visualstudio/code-quality/ca1414?view=vs-2019

[AArnott]

BOOL appears in many places, including fields of structs, where marshaling is not always desirable or possible. However your doc snippet suggests that we can use bool (without marshaling) everywhere that BOOL appears, and that we should use a 1-byte representation wherever the native code uses bool.

@sotteson1, does the Win32 API ever use the native bool type (1-byte in length)? And if so, how is it represented in the metadata? It sounds like we can use System.Boolean for the 4-byte BOOL and should use something else for the C 1-byte bool type other than System.Boolean.

[sotteson1]

I'm not sure why we would use the System.Boolean, as BOOL in Win32 is simply an 32-bit int. Using a NativeTypedef struct for the Win32 BOOL seems to be closer to the spirit of what the Win32 BOOL is.

[AArnott]

Thanks for chiming in, @sotteson1. I guess for proper interop on the CLR and better C# language experience, we can change all references from BOOL to bool in CsWin32.

[AArnott]

I just tried replacing all uses of BOOL with bool in our projection, but although the default interop size may be the same (4 bytes) between the two, evidently the CLR still considers bool to be a marshaled type, which makes it intolerable for use as a field in a struct, at least until #26 is delivered and only for customers using that setting.