Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce design doc for Span/ReadOnlySpan<T> marshalling #1103

Merged

Conversation

jkoritzinsky
Copy link
Member

This PR introduces a design doc for us to discuss how to implement marshalling support for Span/ReadOnlySpan<T> marshalling.

I started implementing Design 1, but when I started writing out the design doc, I realized the limitations of the design. I wrote up Design 2 as a proposal for handling the concept of "collection" marshallers so developers won't have to fork the DllImport marshaller to support default marshalling for generic collection-like types. This way, we could provide support for Span/ReadOnlySpan marshallers without fully inlining them into the generator iself and we can provide additional marshalers for Span and ReadOnlySpan without weird drawbacks and inconsistencies.

Finish the in-code collection marshalling support design doc.
@jkoritzinsky jkoritzinsky added the area-DllImportGenerator Source Generated stubs for P/Invokes in C# label May 10, 2021

<!--
Idea: Extend the StructMarshalling proposal with a design for a generic collection-based marshalling pattern. That will enable outlining the Span/ReadOnlySpan/array/etc. marshallers while enabling developers to enable marshalling any collection type they want without us adding types to the generator.
-->
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Examples of unmanaged APIs that would be good candidates for the generic collection-based marshaling patterns?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's a few ideas I had for the patterns:

  • Move arrays to use this pattern (intrinsic support in the generator)
  • Span -> native array
  • List -> linked-list
    • This could be useful if for example the JIT exposed a linked list type across the JIT interface body and crossgen2 needs to use it.
    • LLVM also has a few linked list types that LLVMSharp could use this with if they exposed those APIs
  • Dictionary -> array of keys or Dictionary -> array of values
    • One good example of this is the coreclr hosting API
  • Dictionary -> array of key-value-pair structures
    • The JVM's JavaVMInitArgs options field is a good example of this one.
  • Win32 SAFEARRAY support could be implemented with this model for SAFEARRAYs containing a specific element type in the VARIANTs.

The nice part about this design is that one developer could provide a "marshal a Dictionary as an array of the keys" and someone else could provide a "marshal an object as the result of it's ToString method" and then a user who wants to pass a Dictionary down as the string values of its keys doesn't have to write a marshaller for the whole scenario. They'd only have to say "use the Dictionary keys marshaller for the dictionary, and use the ToString marshaller for the elements."

designs/SpanMarshallers.md Outdated Show resolved Hide resolved
designs/SpanMarshallers.md Outdated Show resolved Hide resolved
designs/SpanMarshallers.md Outdated Show resolved Hide resolved
designs/SpanMarshallers.md Outdated Show resolved Hide resolved
designs/SpanMarshallers.md Outdated Show resolved Hide resolved
designs/SpanMarshallers.md Outdated Show resolved Hide resolved
designs/SpanMarshallers.md Outdated Show resolved Hide resolved
designs/SpanMarshallers.md Outdated Show resolved Hide resolved
designs/SpanMarshallers.md Outdated Show resolved Hide resolved
designs/SpanMarshallers.md Outdated Show resolved Hide resolved
designs/SpanMarshallers.md Show resolved Hide resolved
@jkoritzinsky
Copy link
Member Author

I've updated Design 2 to focus on contiguous collections and mention non-contiguous ones as an extension we can look at in the future but don't need to adopt today.

designs/SpanMarshallers.md Show resolved Hide resolved
designs/SpanMarshallers.md Outdated Show resolved Hide resolved
designs/SpanMarshallers.md Outdated Show resolved Hide resolved
designs/SpanMarshallers.md Outdated Show resolved Hide resolved
designs/SpanMarshallers.md Show resolved Hide resolved
designs/SpanMarshallers.md Outdated Show resolved Hide resolved
designs/SpanMarshallers.md Outdated Show resolved Hide resolved
designs/SpanMarshallers.md Outdated Show resolved Hide resolved
designs/SpanMarshallers.md Outdated Show resolved Hide resolved
designs/SpanMarshallers.md Show resolved Hide resolved
@AaronRobinsonMSFT
Copy link
Member

This LGTM. The only thing I would like understand better would be #1103 (comment). I didn't find an answer to that but admit I could have missed it due to terminology.

@jkoritzinsky
Copy link
Member Author

jkoritzinsky commented May 26, 2021

I added a blurb for that towards the end of the section (if theres a marshaller for List<T> named ListAsArrayMarshaller<T>):

The sequence of managed types for ElementIndirectionLevel is based on the elements of the ManagedValues span on the collection marshaller of the previous indirection level. For example, for the marshalling info for ElementIndirectionLevel = 1 above, the managed type is the type of the following C# expression: ListAsArrayMarshaller<List<Foo>>.ManagedValues[0].

Does that answer your question?

@AaronRobinsonMSFT
Copy link
Member

Does that answer your question?

Yes. I was misinterpreting the utility here. I understand the proposal now.

@jkoritzinsky jkoritzinsky merged commit 1bc504d into dotnet:feature/DllImportGenerator May 27, 2021
@jkoritzinsky jkoritzinsky deleted the span-design-doc branch May 27, 2021 03:55
jkoritzinsky added a commit to jkoritzinsky/runtime that referenced this pull request Sep 20, 2021
…timelab#1103)

* First draft of the heart of the span marshalling design.

Finish the in-code collection marshalling support design doc.

* Apply suggestions from code review

Co-authored-by: Aaron Robinson <[email protected]>

* Feedback.

* Empty spans will be marshalled as null.

* Make the primary design in Design 2 be contiguous collection support.

* pr feedback and add example stub.

* Fix typo

Co-authored-by: Aaron Robinson <[email protected]>

Commit migrated from dotnet/runtimelab@1bc504d
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-DllImportGenerator Source Generated stubs for P/Invokes in C#
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants