Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support ref field declarations from source and metadata #60416

Merged
merged 31 commits into from
May 2, 2022

Conversation

cston
Copy link
Member

@cston cston commented Mar 28, 2022

Proposal: low-level-struct-improvements.md
Test plan: #59194

Included:

  • API: added IFieldSymbol.RefKind and IFieldSymbol.RefCustomModifiers
  • Parsing: allow ref and ref readonly modifiers for fields
  • Binding: allow ref fields as l-values and r-values
  • Code gen: allow ref fields as l-values and r-values
  • Metadata decoding/encoding: read/write ByReference and RefCustomModifiers
  • -langversion: report error for ref declarations and use-sites with -langversion:10 or below
  • SymbolDisplay: include ref modifiers and RefCustomModifiers from C#

Not included:

  • Use-site error using ref field from VB
  • Updates to val and ref escape calculations
  • ref auto-properties
  • Unit tests do not execute binaries from CI builds

@cston cston changed the base branch from main to features/ref-fields April 5, 2022 21:52
@cston cston marked this pull request as ready for review April 5, 2022 21:53
@cston cston requested review from a team as code owners April 5, 2022 21:53
@cston
Copy link
Member Author

cston commented Apr 5, 2022

cc @jaredpar, @AaronRobinsonMSFT

Copy link
Contributor

@RikkiGibson RikkiGibson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with some minor nits. Will review tests shortly.

Copy link
Contributor

@RikkiGibson RikkiGibson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had various comments on the tests, discussed offline but leaving the comments here to ensure we can go through and resolve.

Assert.Equal(RefKind.RefReadOnly, field.RefKind);
// Currently, source symbols cannot declare RefCustomModifiers. If that
// changes, update this test to verify retargeting of RefCutomModifiers.
Assert.Equal(new string[0], field.RefCustomModifiers.SelectAsArray(m => m.Modifier.ToTestDisplayString()));
Copy link
Contributor

@RikkiGibson RikkiGibson Apr 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: prefer Assert.Empty #Resolved

[Fact]
public void DefiniteAssignment_02()
{
// Should we report a warning when assigning a value rather than a ref in the
Copy link
Contributor

@RikkiGibson RikkiGibson Apr 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like this should have a PROTOTYPE marker or tracking issue. #Resolved

Copy link
Member Author

@cston cston Apr 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's an open question for language design in the test plan so I'll leave this as a regular comment.

// (20,21): error CS1510: A ref or out value must be an assignable variable
// F = ref GetValue();
Diagnostic(ErrorCode.ERR_RefLvalueExpected, "GetValue()").WithLocation(20, 21),
// (22,21): error CS8331: Cannot assign to method 'S<T>.GetRefReadonly()' because it is a readonly variable
Copy link
Contributor

@RikkiGibson RikkiGibson Apr 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like this message is expected but the language seems confusing. It feels like we should say something like "cannot take a writable ref to a readonly variable". Not suggesting you fix it in this PR or even this feature, but perhaps we can improve the message separately. #Resolved

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a PROTOTYPE comment.

@"ref struct S<T>
{
public ref T F;
public ref T F1() => ref F;
Copy link
Contributor

@RikkiGibson RikkiGibson Apr 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider using unique method names across the test to make it a little easier to relate the diagnostics to the source. #Resolved

IL_0000: ldarga.s V_0
IL_0002: ldfld ""ref T S<T>.F""
IL_0007: ldloca.s V_0
IL_0009: initobj ""T""
Copy link
Contributor

@RikkiGibson RikkiGibson Apr 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't understand why there's an initobj here. Consider including a PROTOTYPE comment. #Resolved

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The initobj is part of a if ((object)default(T) != null) { ... } test that is emitted for unconstrained T to test whether T is a struct (see CodeGenerator.EmitLoweredConditionalAccessExpression).

Console.WriteLine((i, s));
}
}";
CompileAndVerify(source, verify: Verification.Skipped, expectedOutput: IncludeExpectedOutput(@"(1, Hello world)"));
Copy link
Contributor

@RikkiGibson RikkiGibson Apr 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we verify the IL of the deconstruction assignment itself? #Resolved

// (5,27): error CS8145: Auto-implemented properties cannot return by reference
// public ref readonly T Q { get; }
Diagnostic(ErrorCode.ERR_AutoPropertyCannotBeRefReturning, "Q").WithArguments("S<T>.Q").WithLocation(5, 27),
// (8,9): error CS8373: The left-hand side of a ref assignment must be a ref local or parameter.
Copy link
Contributor

@RikkiGibson RikkiGibson Apr 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: we'll want to adjust this diagnostic language when we get the chance. Perhaps to simply say ref variable. #Resolved

}
}";
var comp = CreateCompilation(source);
// PROTOTYPE: Should not report ERR_RefReturnStructThis.
Copy link
Contributor

@RikkiGibson RikkiGibson Apr 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spec says (deep in the detailed design section):

When scoped is applied to an instance method the this parameter will have the type scoped ref T. By default this is redundant as scoped ref T is the default type of this. #Resolved

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Therefore I expected an error here, but no error once we are able to declare the unscoped version of the method.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, thanks. I've split this into two tests, one for a value field, and one for a ref field, and moved the PROTOTYPE comment to the ref case. scoped and unscoped will be handled in a later PR.

Comment on lines 638 to 640
s.F = ref tValue;
s.F = ref tRef;
s.F = ref tOut;
Copy link
Member

@jaredpar jaredpar Apr 11, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These three should all be errors as well. The field is readonly ref here which means the ref cannot be re-pointed but the value is mutable. Hence any ref assignment outside the constructor should be an error. #Resolved

@cston
Copy link
Member Author

cston commented Apr 27, 2022

        if (DeriveUseSiteInfoFromType(ref result, this.TypeWithAnnotations, AllowedRequiredModifierType.System_Runtime_CompilerServices_Volatile))

Added a PROTOTYPE comment.


In reply to: 1111564993


Refers to: src/Compilers/CSharp/Portable/Symbols/FieldSymbol.cs:351 in 6bfd88c. [](commit_id = 6bfd88c, deletion_comment = False)

@@ -796,7 +796,8 @@ private bool CheckFieldValueKind(SyntaxNode node, BoundFieldAccess fieldAccess,
// S has a mutable field x, then c.f.x is not a variable because c.f is not
// writable.

if (fieldSymbol.IsReadOnly)
if ((RequiresAssignableVariable(valueKind) && fieldSymbol.RefKind == RefKind.None) ||
RequiresRefAssignableVariable(valueKind))
Copy link
Contributor

@AlekseyTs AlekseyTs Apr 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RequiresRefAssignableVariable(valueKind)

Should this be combined with ref fields only? #Closed

@@ -1890,7 +1919,7 @@ private static ErrorCode GetStandardRValueRefEscapeError(uint escapeTo)
private static void ReportReadOnlyFieldError(FieldSymbol field, SyntaxNode node, BindValueKind kind, bool checkingReceiver, BindingDiagnosticBag diagnostics)
{
Debug.Assert((object)field != null);
Debug.Assert(RequiresAssignableVariable(kind));
Debug.Assert(RequiresAssignableVariable(kind) && field.RefKind == RefKind.None || RequiresRefAssignableVariable(kind));
Copy link
Contributor

@AlekseyTs AlekseyTs Apr 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RequiresRefAssignableVariable(kind)

Should this be combined with ref fields only? #Closed

this.Visit(fieldDefinition.GetType(Context));
}

public virtual void Visit(IFieldReference fieldReference)
{
this.Visit(fieldReference.RefCustomModifiers);
Copy link
Contributor

@AlekseyTs AlekseyTs Apr 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this.Visit(fieldReference.RefCustomModifiers);

This doesn't look right. We are not visiting type here. Why visit custom modifiers? #Closed

@AlekseyTs
Copy link
Contributor

AlekseyTs commented Apr 28, 2022

        AssertEx.Equal(new[] { "ref System.Int32 R<T>.F", "ref modopt(System.Int32) T R<T>.F", "ref modopt(System.Object) T R<T>.F" }, fields.ToTestDisplayStrings());

I am curious why private fields are imported, it doesn't look like we are requesting this explicitly


In reply to: 1111602390


In reply to: 1111602390


In reply to: 1111602390


Refers to: src/Compilers/CSharp/Test/Semantic/Semantics/RefFieldTests.cs:484 in 7c8b1e3. [](commit_id = 7c8b1e3, deletion_comment = False)

Dim type As TypeSymbol = Me.DecodeFieldSignature(signaturePointer, customModifiers)
Return FindFieldBySignature(_containingType, memberName, customModifiers, type)
Dim fieldInfo As FieldInfo(Of TypeSymbol) = Me.DecodeFieldSignature(signaturePointer)
Return FindFieldBySignature(_containingType, memberName, fieldInfo.CustomModifiers, fieldInfo.Type)
Copy link
Contributor

@AlekseyTs AlekseyTs Apr 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FindFieldBySignature(_containingType, memberName, fieldInfo.CustomModifiers, fieldInfo.Type)

It doesn't look right to ignore 'ref' here. Perhaps it is fine to not find a match for a ref field because VB doesn't support ref fields, but ignoring the presence of the 'ref' feels wrong. #Closed

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a test and PROTOTYPE comments for now.

@cston
Copy link
Member Author

cston commented Apr 28, 2022

        AssertEx.Equal(new[] { "ref System.Int32 R<T>.F", "ref modopt(System.Int32) T R<T>.F", "ref modopt(System.Object) T R<T>.F" }, fields.ToTestDisplayStrings());

PENamedTypeSymbol.CreateFields() imports private fields of ordinary structs.


In reply to: 1111602390


Refers to: src/Compilers/CSharp/Test/Semantic/Semantics/RefFieldTests.cs:484 in 7c8b1e3. [](commit_id = 7c8b1e3, deletion_comment = False)

Inherits BasicTestBase

<Fact>
Public Sub RefField()
Copy link
Contributor

@AlekseyTs AlekseyTs Apr 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RefField

I am not sure what this test is actually testing. Is the field actually imported since it is private? I would at least expect this test to cover PEFieldSymbol behavior, where custom modifiers go, etc. #Closed

@@ -322,6 +350,24 @@ private PEModuleSymbol ContainingPEModule
}
}

public override RefKind RefKind
Copy link
Contributor

@AlekseyTs AlekseyTs Apr 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RefKind

Do we have direct tests for the added APIs? #Closed

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, RefFieldTests tests RefKind and RefCustomModifiers for various FieldSymbol implementations.

@@ -287,6 +313,8 @@ private void EnsureSignatureIsLoaded()
type = TypeWithAnnotations.Create(new PointerTypeSymbol(TypeWithAnnotations.Create(fixedElementType)));
}

ImmutableInterlocked.InterlockedInitialize(ref _lazyRefCustomModifiers, CSharpCustomModifier.Convert(fieldInfo.RefCustomModifiers));
Copy link
Contributor

@AlekseyTs AlekseyTs Apr 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_lazyRefCustomModifiers

Do we have direct tests asserting where regular and ref modifiers go? #Closed

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes.

@AlekseyTs
Copy link
Contributor

AlekseyTs commented Apr 28, 2022

            if (customModifiersArray.IsEmpty && IsFixedBuffer(out fixedSize, out fixedElementType))

Can ref be combined with fixed buffer?


In reply to: 1111621176


In reply to: 1111621176


Refers to: src/Compilers/CSharp/Portable/Symbols/Metadata/PE/PEFieldSymbol.cs:309 in 7c8b1e3. [](commit_id = 7c8b1e3, deletion_comment = False)

// void M(readonly ref int p)
Diagnostic(ErrorCode.ERR_SyntaxError, ")").WithArguments("(", ")").WithLocation(4, 30));
Diagnostic(ErrorCode.ERR_SyntaxError, ")").WithArguments(",", ")").WithLocation(4, 30),
// (5,6): error CS1002: ; expected
Copy link
Contributor

@AlekseyTs AlekseyTs Apr 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// (5,6): error CS1002: ; expected

Consider adding a syntax tree test for this scenario. I am assuming we used to parse parameter as a method and now we parse it as a field. #Closed

@AlekseyTs
Copy link
Contributor

Done with review pass (commit 26)

@cston
Copy link
Member Author

cston commented Apr 28, 2022

            if (customModifiersArray.IsEmpty && IsFixedBuffer(out fixedSize, out fixedElementType))

Added tests and PROTOTYPE comments to disallow fixed ref declarations and use.


In reply to: 1111621176


Refers to: src/Compilers/CSharp/Portable/Symbols/Metadata/PE/PEFieldSymbol.cs:309 in 7c8b1e3. [](commit_id = 7c8b1e3, deletion_comment = False)

@cston
Copy link
Member Author

cston commented Apr 29, 2022

@AlekseyTs, @RikkiGibson, please review the latest changes. I believe I have responded to all feedback, thanks.

@@ -796,8 +796,7 @@ private bool CheckFieldValueKind(SyntaxNode node, BoundFieldAccess fieldAccess,
// S has a mutable field x, then c.f.x is not a variable because c.f is not
// writable.

if ((RequiresAssignableVariable(valueKind) && fieldSymbol.RefKind == RefKind.None) ||
RequiresRefAssignableVariable(valueKind))
if (fieldSymbol.RefKind == RefKind.None ? RequiresAssignableVariable(valueKind) : RequiresRefAssignableVariable(valueKind))
Copy link
Contributor

@RikkiGibson RikkiGibson Apr 29, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this just a simplification of the same logic that was there previously?

                if ((RequiresAssignableVariable(valueKind) && fieldSymbol.RefKind == RefKind.None) ||
                    RequiresRefAssignableVariable(valueKind))

#Resolved

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a response to #60416 (comment). But it is also simpler.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, my mistake. The original form got us in here when RequiredRefAssignableVariable is true and RefKind is None.

Copy link
Contributor

@AlekseyTs AlekseyTs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM (commit 31)

@cston
Copy link
Member Author

cston commented May 2, 2022

Thanks for the detailed reviews!

@cston cston merged commit 6b47535 into dotnet:features/ref-fields May 2, 2022
@cston cston deleted the ref-field-decl branch May 2, 2022 14:53
@hez2010
Copy link

hez2010 commented Jun 8, 2022

I think this PR doesn't cover below scenario:

ref struct Node
{
    ref Node _next; // should be ok, but actually error
}

@cston
Copy link
Member Author

cston commented Jun 10, 2022

Thanks @hez2010. I've added an item for "ref field of the containing type" to the test plan.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants