Implement RFC 1861: Extern types #44295

plietar · 2017-09-03T18:57:18Z

A few notes :

Type parameters are not supported. This was an unresolved question from the RFC. It is not clear how useful this feature is, and how variance should be treated. This can be added in a future PR.
size_of_val / align_of_val can be called with extern types, and respectively return 0 and 1. This differs from the RFC, which specified that they should panic, but after discussion with @eddyb on IRC this seems like a better solution.
If/when a DynSized trait is added, this will be disallowed statically.
Auto traits are not implemented by default, since the contents of extern types is unknown. This means extern types are !Sync, !Send and !Freeze. This seems like the correct behaviour to me.
Manual unsafe impl Sync for Foo is still possible.
This PR allows extern type to be used as the tail of a struct, as described by the RFC :

extern {
    type OpaqueTail;
}

#[repr(C)]
struct FfiStruct {
    data: u8,
    more_data: u32,
    tail: OpaqueTail,
}

However this is undesirable, as the alignment of tail is unknown (the current PR assumes an alignment of 1). Unfortunately we can't prevent it in the general case as the tail could be a type parameter :

#[repr(C)]
struct FfiStruct<T: ?Sized> {
    data: u8,
    more_data: u32,
    tail: T,
}

Adding a DynSized trait would solve this as well, by requiring tail fields to be bound by it.

Despite being unsized, pointers to extern types are thin and can be casted from/to integers. However it is not possible to write a null<T>() -> *const T function which works with extern types, as I've explained here : Tracking issue for RFC 1861: Extern types #43467 (comment)
Trait objects cannot be built from extern types. I intend to support it eventually, although how this interacts with DynSized/size_of_val is still unclear.
The definition of c_void is unmodified

rust-highfive · 2017-09-03T18:57:24Z

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @arielb1 (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

plietar · 2017-09-04T03:00:49Z

I also have an implementation of DynSized which goes with this at https://github.com/plietar/rust/tree/dynsized

arielb1 · 2017-09-04T08:45:52Z

src/librustc_trans/intrinsic.rs

                let (llsize, _) =
                    glue::size_and_align_of_dst(bcx, tp_ty, llargs[1]);
                llsize
            } else {
-                let lltp_ty = type_of::type_of(ccx, tp_ty);
-                C_uint(ccx, machine::llsize_of_alloc(ccx, lltp_ty))
+                C_null(llret_ty)


What is that supposed to be? Did you mean C_uint(ccx, 0)?

Yes, although I'm not really familiar with LLVM IR and I expected the two to be equivalent. I originally took this from the discriminant_value intrinsic.

arielb1 · 2017-09-04T09:03:29Z

src/librustc_trans/mir/lvalue.rs

-                return (bcx.struct_gep(
-                        ptr_val, adt::struct_llfields_index(st, ix)), alignment);
-            }
+            bcx.ccx.shared().type_is_sized(fty)


You don't need to do an alignment fixup if you have a foreign vtable. Actually, you only need to do an alignment fixup if the vtable type is Trait.

Foreign tails were handled by the !self.has_extra() a few lines down. I've added a case to the match just beneath, especially since I'm pretty sure the !self.has_extra() branch should never be taken.

I think it's detritus from old trans, but with your patch it's taken by Foo<Foo<Opaque>>. I'm fine with it because you're trying to forbid that.

arielb1 · 2017-09-04T09:12:42Z

src/test/compile-fail/extern-types-not-sync-send.rs

@@ -0,0 +1,28 @@
+// Copyright 2017 The Rust Project Developers. See the COPYRIGHT


Also add a test that they are !Sized?

arielb1 · 2017-09-04T09:13:27Z

src/test/run-pass/extern-types-pointer-cast.rs

@@ -0,0 +1,30 @@
+// Copyright 2017 The Rust Project Developers. See the COPYRIGHT


It would be better to use a pointer with an "interesting" bit pattern rather than 0, to make sure we are not truncating anything.

Done. I've had to use cfg(target_pointer_width) to make sure all bits have something interesting.

arielb1 · 2017-09-04T09:22:21Z

src/test/run-pass/extern-types-size_of_val.rs

+
+    assert_eq!(size_of_val(x), 0);
+    assert_eq!(align_of_val(x), 1);
+}


More tests I think we need:

test that pointers to extern types have the same size and align as a thin pointer.

check that pointers to extern types have the right ABI - maybe add a test to https://github.com/rust-lang/rust/tree/e22a3cf7e12b622b143694942504c66a544dc4c5/src/test/run-make/extern-fn-struct-passing-abi

check that structs with extern tails work - e.g. struct Foo<T: ?Sized>(u8, T); where T = <foreign type>/Foo<<foreign type>>

arielb1 · 2017-09-04T11:41:24Z

Basically LGTM modulo the 2 nits and the tests.

plietar · 2017-09-04T13:36:34Z

@arielb1 I've addressed your comments.

check that structs with extern tails work - e.g. struct Foo<T: ?Sized>(u8, T); where T = /Foo<>

I've changed most of the tests to try both extern types and struct with foreign tails.

I've however not put any test which tries to get the address of the foreign tail, since the alignment of the field is unknown. While it is currently allowed (with an assumed alignement of 1), it shouldn't be once DynSized is implemented.

arielb1 · 2017-09-04T15:37:45Z

@bors r+

bors · 2017-09-04T15:37:47Z

📌 Commit 5814c88 has been approved by arielb1

bors · 2017-09-04T18:36:22Z

⌛ Testing commit 5814c880fcf31abd4895295d14f6d994b2badf34 with merge 9dbee87171690a1188497154b292f6403992db03...

kennytm · 2017-09-04T19:06:58Z

src/libsyntax/feature_gate.rs

@@ -382,6 +382,9 @@ declare_features! (

    // allow '|' at beginning of match arms (RFC 1925)
    (active, match_beginning_vert, "1.21.0", Some(44101)),
+
+    // extern types
+    (active, extern_types, "1.21.0", Some(43467)),


Shouldn't this be 1.22.0?

bors · 2017-09-04T19:43:01Z

💔 Test failed - status-appveyor

kennytm · 2017-09-04T19:53:37Z

dist x86_64-pc-windows-msvc failed, needs to update rustfmt. Legit.

[01:03:51] error[E0004]: non-exhaustive patterns: `Ty` not covered
[01:03:51]     --> C:\Users\appveyor\.cargo\registry\src\github.com-1ecc6299db9ec823\rustfmt-nightly-0.2.5\src\items.rs:2808:39
[01:03:51]      |
[01:03:51] 2808 |         let item_str = try_opt!(match self.node {
[01:03:51]      |                                       ^^^^^^^^^ pattern `Ty` not covered
[01:03:51] 
[01:03:51] error: aborting due to previous error
[01:03:51] 
[01:03:51] error: Could not compile `rustfmt-nightly`.

See rust-lang/rust#44295

plietar · 2017-09-05T01:26:39Z

Opened PR in rustfmt : rust-lang/rustfmt#1946

plietar · 2017-09-07T13:51:02Z

ping @arielb1, I've updated rustfmt

arielb1 · 2017-09-07T16:04:23Z

@bors r+

bors · 2017-09-07T16:04:24Z

📌 Commit d60d340 has been approved by arielb1

plietar · 2017-10-27T21:44:57Z

Hey,
Super sorry, I've been really busy and forgot about this. I've rebased now.
Now that rustbuild has support for broken rustfmt/rls this should be able to land.

kennytm · 2017-10-28T07:45:49Z

r? @arielb1

arielb1 · 2017-10-28T13:25:41Z

At last!
@bors r+

bors · 2017-10-28T13:25:42Z

📌 Commit 1e9e319 has been approved by arielb1

bors · 2017-10-28T13:34:19Z

⌛ Testing commit 1e9e319 with merge dce604a...

@eddyb

Implement RFC 1861: Extern types A few notes : - Type parameters are not supported. This was an unresolved question from the RFC. It is not clear how useful this feature is, and how variance should be treated. This can be added in a future PR. - `size_of_val` / `align_of_val` can be called with extern types, and respectively return 0 and 1. This differs from the RFC, which specified that they should panic, but after discussion with @eddyb on IRC this seems like a better solution. If/when a `DynSized` trait is added, this will be disallowed statically. - Auto traits are not implemented by default, since the contents of extern types is unknown. This means extern types are `!Sync`, `!Send` and `!Freeze`. This seems like the correct behaviour to me. Manual `unsafe impl Sync for Foo` is still possible. - This PR allows extern type to be used as the tail of a struct, as described by the RFC : ```rust extern { type OpaqueTail; } #[repr(C)] struct FfiStruct { data: u8, more_data: u32, tail: OpaqueTail, } ``` However this is undesirable, as the alignment of `tail` is unknown (the current PR assumes an alignment of 1). Unfortunately we can't prevent it in the general case as the tail could be a type parameter : ```rust #[repr(C)] struct FfiStruct<T: ?Sized> { data: u8, more_data: u32, tail: T, } ``` Adding a `DynSized` trait would solve this as well, by requiring tail fields to be bound by it. - Despite being unsized, pointers to extern types are thin and can be casted from/to integers. However it is not possible to write a `null<T>() -> *const T` function which works with extern types, as I've explained here : #43467 (comment) - Trait objects cannot be built from extern types. I intend to support it eventually, although how this interacts with `DynSized`/`size_of_val` is still unclear. - The definition of `c_void` is unmodified

bors · 2017-10-28T16:15:14Z

☀️ Test successful - status-appveyor, status-travis
Approved by: arielb1
Pushing dce604a to master...

eddyb · 2017-10-28T16:36:05Z

Oh no, this shouldn't have merged before #45225. Oh well, time to rebase now.

See rust-lang/rust#44295

The DynSized trait is implemented by all types which have a size and alignment known at runtime. This includes every type other than extern types, introduced in RFC 1861 and implemented in rust-lang#44295, which are completely opaque. The main motivation for this trait is to prevent the use of !DynSized types as struct tails. Consider for example the following types : ```rust extern { type foo; } struct A<T: ?Sized> { a_x: u8 a_y: T } struct B<T: ?Sized> { b_x: u8 b_y: T } ``` Before this change, the type `A<B<foo>>` is considered well-formed. However, the alignment of `B<foo>` and thus the offset of the `a_y` field depends on the alignment of `foo`, which is unknown. By introducing this new trait, struct tails are now required to implement `DynSized`, such that their alignment is known. The trait is an implicit bound, making `A<B<foo>>` ill-formed. Just like the `Sized` trait, the default bound can be opted-out by using `?DynSized`.

rust-highfive assigned arielb1 Sep 3, 2017

plietar mentioned this pull request Sep 3, 2017

Tracking issue for RFC 1861: Extern types #43467

Open

3 tasks

arielb1 reviewed Sep 4, 2017

View reviewed changes

plietar force-pushed the extern-types branch from ea9220e to fe7dc7d Compare September 4, 2017 13:28

kennytm reviewed Sep 4, 2017

View reviewed changes

plietar added a commit to plietar/rustfmt that referenced this pull request Sep 5, 2017

Add suport for extern types.

ccc864e

See rust-lang/rust#44295

plietar added a commit to plietar/rustfmt that referenced this pull request Sep 5, 2017

Add suport for extern types.

0620f4b

See rust-lang/rust#44295

plietar mentioned this pull request Sep 5, 2017

Add suport for extern types. rust-lang/rustfmt#1946

Closed

arielb1 added the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label Sep 5, 2017

plietar force-pushed the extern-types branch from 5814c88 to d60d340 Compare September 7, 2017 12:17

Paul Liétar added 3 commits October 27, 2017 23:01

Undo changes to core::ptr

398947d

fix rebase

5341d64

Move type_has_metadata to trans_utils

1e9e319

plietar force-pushed the extern-types branch from f71ec23 to 1e9e319 Compare October 27, 2017 21:36

kennytm added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Oct 28, 2017

kennytm added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 28, 2017

bors merged commit 1e9e319 into rust-lang:master Oct 28, 2017

This was referenced Oct 28, 2017

[Syntax] Implement auto trait syntax #45247

Merged

Move collector to librustc_mir::monomorphize #45525

Merged

dtolnay mentioned this pull request Oct 29, 2017

Does not compile on 1.23.0-nightly (269cf5026 2017-10-28) rust-lang/rustfmt#2096

Closed

nrc pushed a commit to rust-lang/rustfmt that referenced this pull request Oct 29, 2017

Add suport for extern types.

c1e8979

See rust-lang/rust#44295

est31 mentioned this pull request Nov 8, 2017

ICE on nightly 2017-11-07: casting *const i64 to *mut _ #45872

Closed

crumblingstatue mentioned this pull request Dec 11, 2017

Consider adding option for utilizing extern_types rust-lang/rust-bindgen#1191

Open

Deewiant mentioned this pull request Dec 22, 2017

unexpected compiler panic in cast.rs #46365

Closed

edwin0cheng mentioned this pull request Apr 27, 2020

Unknown types in web-sys dependency rust-lang/rust-analyzer#4150

Closed

This was referenced Dec 17, 2020

Described opaque type pattern has issues rust-lang/nomicon#250

Closed

Re-investigate alignment and size skade/ffi-opaque#9

Open

dead-claudia mentioned this pull request Apr 2, 2024

RFC: Trait for !Sized thin pointers rust-lang/rfcs#3536

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement RFC 1861: Extern types #44295

Implement RFC 1861: Extern types #44295

plietar commented Sep 3, 2017

rust-highfive commented Sep 3, 2017

plietar commented Sep 4, 2017

arielb1 Sep 4, 2017

plietar Sep 4, 2017

arielb1 Sep 4, 2017

plietar Sep 4, 2017

arielb1 Sep 4, 2017

arielb1 Sep 4, 2017

plietar Sep 4, 2017

arielb1 Sep 4, 2017

plietar Sep 4, 2017

arielb1 Sep 4, 2017

arielb1 commented Sep 4, 2017

plietar commented Sep 4, 2017

arielb1 commented Sep 4, 2017

bors commented Sep 4, 2017

bors commented Sep 4, 2017

kennytm Sep 4, 2017

bors commented Sep 4, 2017

kennytm commented Sep 4, 2017

plietar commented Sep 5, 2017

plietar commented Sep 7, 2017 •

edited

Loading

arielb1 commented Sep 7, 2017

bors commented Sep 7, 2017

plietar commented Oct 27, 2017

kennytm commented Oct 28, 2017

arielb1 commented Oct 28, 2017

bors commented Oct 28, 2017

bors commented Oct 28, 2017

bors commented Oct 28, 2017

eddyb commented Oct 28, 2017

		@@ -0,0 +1,28 @@
		// Copyright 2017 The Rust Project Developers. See the COPYRIGHT

		@@ -0,0 +1,30 @@
		// Copyright 2017 The Rust Project Developers. See the COPYRIGHT

Implement RFC 1861: Extern types #44295

Implement RFC 1861: Extern types #44295

Conversation

plietar commented Sep 3, 2017

rust-highfive commented Sep 3, 2017

plietar commented Sep 4, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

arielb1 commented Sep 4, 2017

plietar commented Sep 4, 2017

arielb1 commented Sep 4, 2017

bors commented Sep 4, 2017

bors commented Sep 4, 2017

Choose a reason for hiding this comment

bors commented Sep 4, 2017

kennytm commented Sep 4, 2017

plietar commented Sep 5, 2017

plietar commented Sep 7, 2017 • edited Loading

arielb1 commented Sep 7, 2017

bors commented Sep 7, 2017

plietar commented Oct 27, 2017

kennytm commented Oct 28, 2017

arielb1 commented Oct 28, 2017

bors commented Oct 28, 2017

bors commented Oct 28, 2017

bors commented Oct 28, 2017

eddyb commented Oct 28, 2017

plietar commented Sep 7, 2017 •

edited

Loading