Tracking Issue for `#![feature(offset_of)]` #106655

WaffleLapkin · 2023-01-09T22:56:35Z

Feature gates tracked by this issue:

#![feature(offset_of)]
~~#![feature(offset_of_enum)]~~ -> moved to Tracking Issue for enum access in offset_of #120141

This is a tracking issue for the offset_of! macro which evaluates to a constant
containing the offset in bytes of a field inside some type (rust-lang/rfcs#3308).

Public API

// core::mem

pub macro offset_of($Container:ty, $field:tt $(,)?) {
    // ...implementation defined...
}

Steps / History

Implementation: Add offset_of! macro (RFC 3308) #106934
Split off offset_of_enum: Feature gate enums in offset_of #117537
Final comment period (FCP)¹: old, new
Stabilization PR: Stabilize single-field offset_of #118799

Possible future extensions / work

Support for static alignment DSTs (struct Example(u32, [u8]))
Support for enums: Support enum variants in offset_of! #114208
Improving diagnostics around the macro errors

Unresolved Questions

None yet.

https://std-dev-guide.rust-lang.org/feature-lifecycle/stabilization.html ↩

The text was updated successfully, but these errors were encountered:

Add offset_of! macro (RFC 3308) Implements rust-lang/rfcs#3308 (tracking issue rust-lang#106655) by adding the built in macro `core::mem::offset_of`. Two of the future possibilities are also implemented: * Nested field accesses (without array indexing) * DST support (for `Sized` fields) I wrote this a few months ago, before the RFC merged. Now that it's merged, I decided to rebase and finish it. cc `@thomcc` (RFC author)

This macro is used to compute the offset of a field in a struct. This commit enables two unstable features that are necessary for using the macro in a constant. However, this is not a problem as the macro will become available from the Rust standard library soon [1]. The unstable features can be disabled again once that happens. The macro in this patch does not support sub-fields. That is, you cannot write `offset_of!(MyStruct, field.sub_field)` to get the offset of `sub_field` with `field`'s type being a struct with a field called `sub_field`. This is because `field` might be a `Box<SubStruct>`, which means that you would be trying to compute the offset to something in an entirely different allocation. There's no easy way to fix the current macro to support subfields, but the version being added to the standard library should support it, so the limitation is temporary and not a big deal. Link: rust-lang/rust#106655 [1] Signed-off-by: Wedson Almeida Filho <[email protected]> Co-developed-by: Alice Ryhl <[email protected]> Signed-off-by: Alice Ryhl <[email protected]>

est31 · 2023-05-04T15:43:35Z

Some things that could get tests:

offset_of!(Self, ...) works, if and only if it's inside an impl block.
the return type of offset_of!() is usize: some code that tries to use it as usize -> works. some code that tries to use it as u8/isize -> fails.

Edit: #111665

Add offset_of! macro (RFC 3308) Implements rust-lang/rfcs#3308 (tracking issue rust-lang#106655) by adding the built in macro `core::mem::offset_of`. Two of the future possibilities are also implemented: * Nested field accesses (without array indexing) * DST support (for `Sized` fields) I wrote this a few months ago, before the RFC merged. Now that it's merged, I decided to rebase and finish it. cc `@thomcc` (RFC author)

Implement builtin # syntax and use it for offset_of!(...) Add `builtin #` syntax to the parser, as well as a generic infrastructure to support both item and expression position builtin syntaxes. The PR also uses this infrastructure for the implementation of the `offset_of!` macro, added by rust-lang#106934. cc `@petrochenkov` `@DrMeepster` cc rust-lang#110680 `builtin #` tracking issue cc rust-lang#106655 `offset_of!` tracking issue

This macro is used to compute the offset of a field in a struct. This commit enables two unstable features that are necessary for using the macro in a constant. However, this is not a problem as the macro will become available from the Rust standard library soon [1]. The unstable features can be disabled again once that happens. The macro in this patch does not support sub-fields. That is, you cannot write `offset_of!(MyStruct, field.sub_field)` to get the offset of `sub_field` with `field`'s type being a struct with a field called `sub_field`. This is because `field` might be a `Box<SubStruct>`, which means that you would be trying to compute the offset to something in an entirely different allocation. There's no easy way to fix the current macro to support subfields, but the version being added to the standard library should support it, so the limitation is temporary and not a big deal. Link: rust-lang/rust#106655 [1] Signed-off-by: Wedson Almeida Filho <[email protected]> Co-developed-by: Alice Ryhl <[email protected]> Signed-off-by: Alice Ryhl <[email protected]>

This macro is used to compute the offset of a field in a struct. This commit enables two unstable features that are necessary for using the macro in a constant. However, this is not a problem as the macro will become available from the Rust standard library soon [1]. The unstable features can be disabled again once that happens. The macro in this patch does not support sub-fields. That is, you cannot write `offset_of!(MyStruct, field.sub_field)` to get the offset of `sub_field` with `field`'s type being a struct with a field called `sub_field`. This is because `field` might be a `Box<SubStruct>`, which means that you would be trying to compute the offset to something in an entirely different allocation. There's no easy way to fix the current macro to support subfields, but the version being added to the standard library should support it, so the limitation is temporary and not a big deal. Link: rust-lang/rust#106655 [1] Signed-off-by: Alice Ryhl <[email protected]> Co-Developed-by: Wedson Almeida Filho <[email protected]> Signed-off-by: Wedson Almeida Filho <[email protected]>

Amanieu · 2023-05-14T08:10:19Z

Small nit on the docs: I don't think "stable" is the right word to use for describing the output on non-repr(C) structs since this causes confusion with Rust's concept of stable APIs.

@rfcbot fcp merge

rfcbot · 2023-05-14T08:10:20Z

Team member @Amanieu has proposed to merge this. The next step is review by the rest of the tagged team members:

Concerns:

~~docs~~ resolved by Tracking Issue for #![feature(offset_of)] #106655 (comment)
~~implementation quality~~ resolved by Tracking Issue for #![feature(offset_of)] #106655 (comment)
syntax (Tracking Issue for #![feature(offset_of)] #106655 (comment))

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

m-ou-se · 2023-05-14T08:13:27Z

Looks like it has an issue: 0.0 doesn't work right now for nested tuple fields.

rfcbot · 2023-05-14T08:27:34Z

🔔 This is now entering its final comment period, as per the review above. 🔔

WaffleLapkin · 2023-05-15T13:31:50Z

@Amanieu do you suggest to stabilize offset_of? Could you clarify what is the FCP for? It seems a bit too soon, as this feature was just recently implemented.

m-ou-se · 2023-05-15T14:28:28Z

This FCP is to confirm that libs-api is happy with committing to this macro existing forever with this interface. That seems refreshingly uncontroversial in this case.

Whether the implementation is robust/bug free enough is a bit of a separate question, and it might be worth delaying stabilization for that.

If you think there is value in having more time for the interface/api itself, please share your thoughts on that!

WaffleLapkin · 2023-05-15T14:34:03Z

Oh I see! I don't think we are committing to the interface until stabilization, but yes, the interface seems uncontroversial 👍🏻

est31 · 2023-05-16T03:17:15Z

One nice project before stabilization would be to make a PR to memoffset to use offset_of internally behind a cfg or feature flag, then test that together with some users of memoffset. Maybe it could be integrated into a crater run in some fashion.

BurntSushi · 2023-05-16T16:55:34Z

I wanted to double check that the lang team was okay with us making the call on an API like this, and indeed, they did respond on the RFC saying they were comfortable with it being in libs-api's hands: rust-lang/rfcs#3308 (comment)

est31 · 2023-12-06T18:07:26Z

The current-on-nightly syntax has precedent in c's offsetof (standard link), including the way dots work for nesting, so it has some familiarity I would presume.

But this doesn't mean that we can't do offset_of!(Type.field.field) if we really wanted to.

Regarding pattern syntax, I wonder how arrays would be represented, so what would be the pattern analog of c's offsetof(S, field.field[20].10)?

est31 · 2023-12-06T18:10:32Z

IDK maybe I made the wrong estimate and it's too early still for a stabilization of offset_of before the syntax discussion advances further. But my fear is that this can drag on for months and years.

GKFX · 2023-12-06T18:32:12Z

I am thinking that offset_of!(Type, self.a.b.0[n]) might be a good choice - it's a single expr which the compiler already knows how to parse, with no edge cases around 0.0 etc.

To summarise, the main options that I'm aware of and my views on them are:

Type.a.b - not a syntax currently used elsewhere so would need custom parsing, breaks the macro follow-set restrictions if matching the type with ty.
C-style - current implementation but has issues with consecutive tuple accesses and whitespace not matching the current metavariables. Needs some quite awkward parsing because of that.
Pattern-based - significantly more verbose, no obvious method for array indexing, but is a pre-existing syntax so no parser work
self.a.b - slightly more verbose version of C-style. Also should need no parser work.

the8472 · 2023-12-06T18:36:32Z

You're forgetting enums

est31 · 2023-12-06T19:11:27Z

@CryZe btw, patterns have come up in the syntax discussion on zulip.

Maybe it would be best to move this discussion there? Or a dedicated thread elswhere. This is a tracking issue after all :).

RalfJung · 2023-12-06T21:23:35Z

Regarding pattern syntax, I wonder how arrays would be represented, so what would be the pattern analog of c's offsetof(S, field.field[20].10)?

offset_of!(S.field.field[20].10). (C does not have .10 though, is this a Rust tuple field access?)

I don't think I understand what the pattern syntax would look like, could someone give some examples? offset_of!(Struct { field1, field2 }) makes no sense so surely it'd have to be a very restricted subset of patterns, and rather redundant if one has to always add the ..?

est31 · 2023-12-29T21:36:02Z

surely it'd have to be a very restricted subset of patterns, and rather redundant if one has to always add the ..?

Yeah the zulip thread proposes usage of @ for that.

C does not have .10 though, is this a Rust tuple field access?

Yes, sorry I forgot. replace that .10 with .ten :).

Make offset_of field parsing use metavariable which handles any spacing As discussed at and around comments rust-lang#106655 (comment) and rust-lang#106655 (comment), the current arguments to offset_of do not accept all the whitespace combinations: `0. 1.1.1` and `0.1.1. 1` are currently treated specially in `tests/ui/offset-of/offset-of-tuple-nested.rs`. They also do not allow [forwarding individual fields as in](https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=444cdf0ec02b99e8fd5fd8d8ecb312ca) ```rust macro_rules! off { ($a:expr) => { offset_of!(m::S, 0. $a) } } ``` This PR replaces the macro arguments with `($Container:ty, $($fields:expr)+ $(,)?)` which does allow any arrangement of whitespace that I could come up with and the forwarding of fields example above. This also allows for array indexing in the future, which I think is the last future extension to the syntax suggested in the offset_of RFC. Tracking issue for offset_of: rust-lang#106655 `@rustbot` label F-offset_of `@est31`

Make offset_of field parsing use metavariable which handles any spacing As discussed at and around comments rust-lang#106655 (comment) and rust-lang#106655 (comment), the current arguments to offset_of do not accept all the whitespace combinations: `0. 1.1.1` and `0.1.1. 1` are currently treated specially in `tests/ui/offset-of/offset-of-tuple-nested.rs`. They also do not allow [forwarding individual fields as in](https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=444cdf0ec02b99e8fd5fd8d8ecb312ca) ```rust macro_rules! off { ($a:expr) => { offset_of!(m::S, 0. $a) } } ``` This PR replaces the macro arguments with `($Container:ty, $($fields:expr)+ $(,)?)` which does allow any arrangement of whitespace that I could come up with and the forwarding of fields example above. This also allows for array indexing in the future, which I think is the last future extension to the syntax suggested in the offset_of RFC. Tracking issue for offset_of: rust-lang#106655 ``@rustbot`` label F-offset_of ``@est31``

Rollup merge of rust-lang#119532 - GKFX:offset-of-parse-expr, r=est31 Make offset_of field parsing use metavariable which handles any spacing As discussed at and around comments rust-lang#106655 (comment) and rust-lang#106655 (comment), the current arguments to offset_of do not accept all the whitespace combinations: `0. 1.1.1` and `0.1.1. 1` are currently treated specially in `tests/ui/offset-of/offset-of-tuple-nested.rs`. They also do not allow [forwarding individual fields as in](https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=444cdf0ec02b99e8fd5fd8d8ecb312ca) ```rust macro_rules! off { ($a:expr) => { offset_of!(m::S, 0. $a) } } ``` This PR replaces the macro arguments with `($Container:ty, $($fields:expr)+ $(,)?)` which does allow any arrangement of whitespace that I could come up with and the forwarding of fields example above. This also allows for array indexing in the future, which I think is the last future extension to the syntax suggested in the offset_of RFC. Tracking issue for offset_of: rust-lang#106655 ``@rustbot`` label F-offset_of ``@est31``

Make offset_of field parsing use metavariable which handles any spacing As discussed at and around comments rust-lang/rust#106655 (comment) and rust-lang/rust#106655 (comment), the current arguments to offset_of do not accept all the whitespace combinations: `0. 1.1.1` and `0.1.1. 1` are currently treated specially in `tests/ui/offset-of/offset-of-tuple-nested.rs`. They also do not allow [forwarding individual fields as in](https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=444cdf0ec02b99e8fd5fd8d8ecb312ca) ```rust macro_rules! off { ($a:expr) => { offset_of!(m::S, 0. $a) } } ``` This PR replaces the macro arguments with `($Container:ty, $($fields:expr)+ $(,)?)` which does allow any arrangement of whitespace that I could come up with and the forwarding of fields example above. This also allows for array indexing in the future, which I think is the last future extension to the syntax suggested in the offset_of RFC. Tracking issue for offset_of: #106655 ``@rustbot`` label F-offset_of ``@est31``

…wesleywiser Stabilize single-field offset_of This PR stabilizes offset_of for a single field. There has been some further discussion at rust-lang#106655 about whether this is advisable; I'm opening the PR anyway so that the code is available.

Rollup merge of rust-lang#118799 - GKFX:stabilize-simple-offsetof, r=wesleywiser Stabilize single-field offset_of This PR stabilizes offset_of for a single field. There has been some further discussion at rust-lang#106655 about whether this is advisable; I'm opening the PR anyway so that the code is available.

est31 · 2024-01-20T21:45:19Z

As #118799 has been merged, non-nested offset_of!() is now stable on the master branch and will likely be part of the 1.77.0 stable release on March 21, 2024. For details of what was stabilized, see the stabilization report.

I'm closing this tracking issue as the still unstable aspects of the offset_of macro now have different tracking issues:

Thanks everyone involved in getting offset_of specified, implemented, tested and stabilized.

Stabilize single-field offset_of This PR stabilizes offset_of for a single field. There has been some further discussion at rust-lang/rust#106655 about whether this is advisable; I'm opening the PR anyway so that the code is available.

Feature gate enums in offset_of As requested at rust-lang/rust#106655 (comment), put enums in offset_of behind their own feature gate. `@rustbot` label F-offset_of

Remove option_payload_ptr; redundant to offset_of The `option_payload_ptr` intrinsic is no longer required as `offset_of` supports traversing enums (#114208). This PR removes it in order to dogfood offset_of (as suggested at rust-lang/rust#106655 (comment)). However, it will not build until those changes reach beta (which I think is within the next 8 days?) so I've opened it as a draft.

Feature gate enums in offset_of As requested at rust-lang/rust#106655 (comment), put enums in offset_of behind their own feature gate. `@rustbot` label F-offset_of

Remove option_payload_ptr; redundant to offset_of The `option_payload_ptr` intrinsic is no longer required as `offset_of` supports traversing enums (#114208). This PR removes it in order to dogfood offset_of (as suggested at rust-lang/rust#106655 (comment)). However, it will not build until those changes reach beta (which I think is within the next 8 days?) so I've opened it as a draft.

WaffleLapkin added T-libs-api Relevant to the library API team, which will review and decide on the PR/issue. C-tracking-issue Category: An issue tracking the progress of sth. like the implementation of an RFC F-offset_of `#![feature(offset_of)]` labels Jan 9, 2023

ojeda mentioned this issue Jan 9, 2023

core wanted features & bugfixes Rust-for-Linux/linux#514

Open

13 tasks

beepster4096 mentioned this issue Jan 16, 2023

Add offset_of! macro (RFC 3308) #106934

Merged

ojeda mentioned this issue Feb 7, 2023

Rust unstable features needed for the kernel Rust-for-Linux/linux#2

Open

90 tasks

PieKing1215 mentioned this issue Feb 16, 2023

Trying to initialize dynamically sized SSBO glium/glium#1918

Open

This comment was marked as resolved.

Sign in to view

est31 mentioned this issue May 4, 2023

Implement builtin # syntax and use it for offset_of!(...) #110694

Merged

rfcbot added proposed-final-comment-period Proposed to merge/close by relevant subteam, see T-<team> label. Will enter FCP once signed off. disposition-merge This issue / PR is in PFCP or FCP with a disposition to merge it. labels May 14, 2023

rfcbot added final-comment-period In the final comment period and will be merged soon unless new substantive objections are raised. and removed proposed-final-comment-period Proposed to merge/close by relevant subteam, see T-<team> label. Will enter FCP once signed off. labels May 14, 2023

GKFX mentioned this issue Dec 10, 2023

Stabilize single-field offset_of #118799

Merged

GKFX mentioned this issue Jan 2, 2024

Make offset_of field parsing use metavariable which handles any spacing #119532

Merged

est31 mentioned this issue Jan 20, 2024

Tracking Issue for nested field access in offset_of #120140

Closed

3 tasks

est31 closed this as completed Jan 20, 2024

rfcbot removed proposed-final-comment-period Proposed to merge/close by relevant subteam, see T-<team> label. Will enter FCP once signed off. disposition-merge This issue / PR is in PFCP or FCP with a disposition to merge it. labels Jan 20, 2024

Rixxc mentioned this issue Feb 6, 2024

refactor: replace lenses library with the zerocopy crate rosenpass/rosenpass#239

Merged

Swoorup mentioned this issue Feb 19, 2024

Create rust.yml Swoorup/wgsl-bindgen#2

Open

xiaguan mentioned this issue Apr 10, 2024

tracking: make foyer compatible with stable rust foyer-rs/foyer#310

Closed

17 tasks

acheroncrypto mentioned this issue Aug 22, 2024

Build Failed: use of unstable library feature '_memoffset__offset_of_impl' coral-xyz/anchor#3187

Open

Thell mentioned this issue Oct 23, 2024

update highs to v1.8.0 rust-or/highs-sys#29

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tracking Issue for `#![feature(offset_of)]` #106655

Tracking Issue for `#![feature(offset_of)]` #106655

WaffleLapkin commented Jan 9, 2023 •

edited by est31

Loading

This comment was marked as resolved.

est31 commented May 4, 2023 •

edited

Loading

Amanieu commented May 14, 2023

rfcbot commented May 14, 2023 •

edited

Loading

m-ou-se commented May 14, 2023

rfcbot commented May 14, 2023

WaffleLapkin commented May 15, 2023

m-ou-se commented May 15, 2023

WaffleLapkin commented May 15, 2023

est31 commented May 16, 2023

BurntSushi commented May 16, 2023

est31 commented Dec 6, 2023 •

edited

Loading

est31 commented Dec 6, 2023

GKFX commented Dec 6, 2023

the8472 commented Dec 6, 2023

est31 commented Dec 6, 2023

RalfJung commented Dec 6, 2023

est31 commented Dec 29, 2023

est31 commented Jan 20, 2024 •

edited

Loading

Tracking Issue for #![feature(offset_of)] #106655

Tracking Issue for #![feature(offset_of)] #106655

Comments

WaffleLapkin commented Jan 9, 2023 • edited by est31 Loading

Public API

Steps / History

Possible future extensions / work

Unresolved Questions

Footnotes

This comment was marked as resolved.

est31 commented May 4, 2023 • edited Loading

Amanieu commented May 14, 2023

rfcbot commented May 14, 2023 • edited Loading

m-ou-se commented May 14, 2023

rfcbot commented May 14, 2023

WaffleLapkin commented May 15, 2023

m-ou-se commented May 15, 2023

WaffleLapkin commented May 15, 2023

est31 commented May 16, 2023

BurntSushi commented May 16, 2023

est31 commented Dec 6, 2023 • edited Loading

est31 commented Dec 6, 2023

GKFX commented Dec 6, 2023

the8472 commented Dec 6, 2023

est31 commented Dec 6, 2023

RalfJung commented Dec 6, 2023

est31 commented Dec 29, 2023

est31 commented Jan 20, 2024 • edited Loading

Tracking Issue for `#![feature(offset_of)]` #106655

Tracking Issue for `#![feature(offset_of)]` #106655

WaffleLapkin commented Jan 9, 2023 •

edited by est31

Loading

est31 commented May 4, 2023 •

edited

Loading

rfcbot commented May 14, 2023 •

edited

Loading

est31 commented Dec 6, 2023 •

edited

Loading

est31 commented Jan 20, 2024 •

edited

Loading