Native interop: not possible to get/use a pointer of a managed struct #843
Comments
|
More or less (C# 7) related issues: |
|
According to the TastOps comments it's possible to get the address of a struct if the value is immutable (+some condition). https://github.com/Microsoft/visualfsharp/blob/master/src/fsharp/TastOps.fs#L5340 https://github.com/Microsoft/visualfsharp/blob/master/src/fsharp/TastOps.fs#L5356 https://github.com/Microsoft/visualfsharp/blob/master/src/fsharp/TastOps.fs#L5367 However the following program will show an error: open System
open System.Runtime.InteropServices
open System.Runtime.CompilerServices
open Microsoft.FSharp.NativeInterop
#nowarn "9"
[<NoComparison;NoEquality>]
[<Struct; StructLayout(LayoutKind.Sequential, Pack=1)>]
type Vector =
val V0: int
val V1: int
val V2: int
val V3: int
val V4: int
val V5: int
val V6: int
val V7: int
new(v0,v1,v2,v3,v4,v5,v6,v7) = { V0=v0; V1=v1; V2=v2; V3=v3; V4=v4; V5=v5; V6=v6; V7=v7 }
module Main = begin
let read (x: nativeptr<Vector>) =
let ptr = x |> NativePtr.toNativeInt |> NativePtr.ofNativeInt<int>
NativePtr.read ptr
let get (x: nativeptr<Vector>) i =
let ptr = x |> NativePtr.toNativeInt |> NativePtr.ofNativeInt<int>
NativePtr.get ptr i
[<EntryPoint>]
let main argv =
let s = Vector(0,1,2,3,4,5,6,7)
//error FS0256: A value must be mutable in order to mutate the contents or take the address of a value type, e.g. 'let mutable x = ...'
let ptr = &&s |> NativePtr.toNativeInt
if nativeint(0) <> ptr then
System.Console.WriteLine("ptr={0}", ptr)
let tptr = ptr |> NativePtr.ofNativeInt<Vector>
let v0 = read tptr
System.Console.WriteLine(v0)
let v1 = get tptr 1
System.Console.WriteLine(v1)
let v2 = get tptr 2
System.Console.WriteLine(v2)
else
System.Console.WriteLine("ptr=0x0")
0
end
Why the "error FS0256: A value must be mutable in order to mutate the contents or take the address of a value type, e.g. 'let mutable x = ...'" (tastValueMustBeLocalAndMutable) showed in this case? Maybe I miss something but the Vector is an immutable struct and there are no mutation operation on the immutable struct. // Give a nice error message for DefinitelyMutates on immutable values, or mutable values in other assemblies
| Expr.Val(v, _,m) when mut = DefinitelyMutates
->
if isByrefTy g v.Type then error(Error(FSComp.SR.tastUnexpectedByRef(),m));
if v.IsMutable then
error(Error(FSComp.SR.tastInvalidAddressOfMutableAcrossAssemblyBoundary(),m));
else
error(Error(FSComp.SR.tastValueMustBeLocalAndMutable(),m));
| _ ->
let ty = tyOfExpr g e
if isStructTy g ty then
match mut with
| NeverMutates -> ()
| DefinitelyMutates ->
errorR(Error(FSComp.SR.tastInvalidMutationOfConstant(),m));
| PossiblyMutates ->
warning(DefensiveCopyWarning(FSComp.SR.tastValueHasBeenCopied(),m));
let tmp,_ = mkMutableCompGenLocal m "copyOfStruct" ty
(fun rest -> mkCompGenLet m tmp e rest), (mkValAddr m (mkLocalValRef tmp))
https://github.com/Microsoft/visualfsharp/blob/master/src/fsharp/TastOps.fs#L5439 |
|
I would be also happy with some kind of (un)safe compiler generated property indexer or a way to add custom built property indexer, so the local pointer will not live longer than the struct itself: Compiler generated:
Update: Custom property indexer in C#: unsafe public int this[int i]
{
get {
fixed (Vector* ptr = &this) {
if ((i < 0) || (i > 7)) {
throw new IndexOutOfRangeException();
} else {
int* fptr =&(ptr->V0);
return *(fptr + i);
}
}
}
}Generated IL from the earlier C# code: Is there any way to express this in F#? Compile time code generation with F# or inline IL F# functions is preferred, if not possible the only remaining way is to use some kind of Cecil based post processing [2] [3] or modify the F# compiler. [1] https://msdn.microsoft.com/en-us/library/system.reflection.emit.opcodes.ldloca%28v=vs.110%29.aspx |
|
I figured out an ugly and unsafe way to do it with DynamicMethod code generation. It should be possible to do it (compile time) in F# at least in inline IL (but I failed to figure out the proper inline IL syntax for the "ldarg 0" "ret" IL sequence). The "unaligned." prefix will needed in non-aligned case according to ECMA 335. This prefix may also need for the new F# initblk, cpblk intrinsics: #16 open System.Runtime.InteropServices
open System.Runtime.CompilerServices
open Microsoft.FSharp.NativeInterop
#nowarn "9"
module StructPtr = begin
open System.Reflection.Emit
type StructPtrDelegate<'T when 'T : unmanaged> = delegate of 'T byref -> nativeint
let createStructPtrDelegate<'T when 'T : unmanaged>() =
let dm = new DynamicMethod("structPtr",
typeof<System.Void>,
[| typeof<'T>.MakeByRefType() |])
let ilGenerator = dm.GetILGenerator()
ilGenerator.Emit(OpCodes.Ldarg_0)
ilGenerator.Emit(OpCodes.Ret)
dm.CreateDelegate(typeof<StructPtrDelegate<'T>>) :?> StructPtrDelegate<'T>
let structPtr (structPtrDelegate:StructPtrDelegate<'T>) (s: 'T byref) = structPtrDelegate.Invoke(&s)
end
open StructPtr
[<NoComparison;NoEquality>]
[<Struct; StructLayout(LayoutKind.Sequential, Pack=1)>]
type Vector =
val Item0: uint64
val Item1: uint64
val Item2: uint64
val Item3: uint64
val Item4: uint64
val Item5: uint64
val Item6: uint64
val Item7: uint64
internal new(
i0,
i1,
i2,
i3,
i4,
i5,
i6,
i7
) = {
Item0=i0;
Item1=i1;
Item2=i2;
Item3=i3;
Item4=i4;
Item5=i5;
Item6=i6;
Item7=i7;
}
static member Get = createStructPtrDelegate<Vector>()
member this.Item
with get index =
let ptr = Vector.Get.Invoke(&this)
let fptr = ptr |> NativePtr.ofNativeInt<uint64>
if (index < 0) && (index > 7) then
raise (new System.IndexOutOfRangeException())
else
NativePtr.get fptr index
[<EntryPoint>]
let main argv =
printfn "%A" argv
let x = Vector(1UL,2UL,3UL,4UL,5UL,6UL,7UL,8UL)
let y0 = x.[0]
let y1 = x.[1]
let y2 = x.[2]
System.Console.WriteLine("[0]: {0}", y0)
System.Console.WriteLine("[1]: {0}", y1)
System.Console.WriteLine("[2]: {0}", y2)
0 // return an integer exit code
|
|
I suspect this should go to http://fslang.uservoice.com as it looks like a language design issue (?) |
|
@zpodlovics Per CONTRIBUTING.md this should go to http://fslang.uservoice.com if it is a language design suggestion, or else to stackoverflow or some other forum if it's an actual question about how to get the address of the struct in question. If you think it's a bug w.r.t. the implementation of the language spec, then please give a smaller repro (a handful of lines should suffice?) Thanks |
|
According to the F# Language Specification 3.1 get/use an address of a value type (struct) should be possible: 6.4.5 The AddressOf Operators [..] 6.9.4 Taking the Address of an Elaborated Expression Repro: [<Struct>]
type Vector =
val V0: int
val V1: int
new(v0,v1) = { V0=v0; V1=v1 }
[<EntryPoint>]
let main argv =
let mutable s = Vector(0,1)
let sAddr = &&s
0 According to the spec, this should be possible. However when I try to compile, it will show the following error: fsharpc struct.fs |
|
What's the status on this @zpodlovics |
|
@realvictorprm It seems better now, thank to the relaxed byref rules, hopefully the Span<'T> work will provide additional capabilities. open System
open System.Runtime.InteropServices
open System.Runtime.CompilerServices
open Microsoft.FSharp.NativeInterop
#nowarn "9"
#nowarn "42"
[<NoComparison;NoEquality>]
[<Struct; StructLayout(LayoutKind.Sequential, Pack=1)>]
type Vector =
val Item0: int
val Item1: int
val Item2: int
val Item3: int
val Item4: int
val Item5: int
val Item6: int
val Item7: int
internal new(
i0,
i1,
i2,
i3,
i4,
i5,
i6,
i7
) = {
Item0=i0;
Item1=i1;
Item2=i2;
Item3=i3;
Item4=i4;
Item5=i5;
Item6=i6;
Item7=i7;
}
[<MethodImpl(MethodImplOptions.NoInlining)>]
static member RaiseIndexOutOfRangeException() =
raise (new System.IndexOutOfRangeException())
member this.Item
with get index =
//bound check
//if uint32(index) >= uint32(7) then
// Vector.RaiseIndexOutOfRangeException()
let ptr = &&this |> NativePtr.toNativeInt |> NativePtr.ofNativeInt<int>
NativePtr.get ptr index
[<EntryPoint>]
let main argv =
printfn "%A" argv
let x = Vector(1,2,3,4,5,6,7,8)
let y0 = x.[0]
let y1 = x.[1]
let y2 = x.[2]
System.Console.WriteLine("[0]: {0}", y0)
System.Console.WriteLine("[1]: {0}", y1)
System.Console.WriteLine("[2]: {0}", y2)
0 // return an integer exit codeNow compiles to: Now with some nativeptr "magic" it's possible to provide readonly array like structures. The readonly field addressof (````&this.Item0 |
I would like to have direct (pointer) access to the managed structs, in order to provide array like access to the structs internals (both the fields, both the whole struct internal memory area - the safe access is my responsibility), however it seems not possible to do it from F#. Every struct type is unmanaged (blittable), so it has the same native representation as the managed representation. The heap allocation unfortunately is not an option, due the excessive amount of generated data.
AddressOf operator (&&) not working on structs, and I also tried a similar trick (nativeptr<'T> -> byref<'T>) that worked on #409 but on the reverse (byref<'T> -> nativeint, byref<'T> -> nativeptr<'T>) order. A similar example works fine in C#, but the C# IL seems to use a different opcode: ldind.i4
The same NativePtr operations works just fine when the ptr is created with localloc opcode, which is also stack allocated (but will be destroyed on fn exit) as the managed structs.
C# Main (partial) IL looks just like the original code:
F Main (partial) IL looks a bit weird:
C# indexed access IL:
F# Indexed access IL:
According to the documentation [1]:
"The ldind.i4 instruction indirectly loads an int32 value from the specified address (of type native int, &, or *) onto the stack as an int32.
All of the ldind instructions are shortcuts for a Ldobj instruction that specifies the corresponding built-in value class."
I know that the match based field access could work, but it generate ineffective code (giant switch with lot's of branches which will be compiled to native cmp/jmp pairs) compared to a simple address based calculation and load.
How to drive F# Adoption PartN:
Please provide at least the same interoperability features as C# provides.
Please provide full opcode (it seems the IL writer not support every opcode due the missing parser and/or pickler support) and full documentation for inline IL, so it would be easier to add new IL features later, similarly how the initblk, cpblk added to NativePtr.
[1] https://msdn.microsoft.com/en-us/library/system.reflection.emit.opcodes.ldind_i4%28v=vs.110%29.aspx
The text was updated successfully, but these errors were encountered: