[Proposal] Attributes on Statements and Expressions

[AdamSobieski]

Introduction

It is here proposed that:

1. attributes could be placed on statements and expressions in method bodies, e.g., using a [body: ...] syntax
2. various kinds of attributes, abstract subclasses of System.Attribute, could provide general utility across .NET and, more specifically, enhance the usefulness of placing attributes on statements and expressions
3. a new attribute, MethodBodyAttribute can enable persisting attributes on statements and expressions in .NET assemblies.

1. Attributes on Statements and Expressions

Below are some examples of what attributes on method bodies’ statements and expressions could syntactically resemble. These syntax examples draw from the domain of JIT-hints, parallelization, and optimization.

public void Function()
{
    [body: SomeAttribute]
    var x = 0;

    var y = [body: AttributeHere] MethodInvocation();

    [body: BranchPrediction(true)]
    if(x > y)
    {
        
    }

    [body: LoopUnrollHint(10)]
    [body: OpenMP.Attribute]
    for(int x = 0; x < 100; ++x)
    {
        [body: Inline] MethodInvocation();
    }

    [body: SomeAttribute]
    using(var z = MethodInvocation())
    {
    
    }

    [body: SomeAttribute] // nested {} scope
    {
    
    }
}

2. Kinds of Attributes

Four kinds of attributes have thus far been considered: annotation, description, validation, and transformation.

In addition to potentially providing new functionalities across .NET, in general, e.g., decorators and aspect-oriented programming, these new kinds of attributes can enhance the utility of the capability to place attributes on statements and expressions.

The attribute kinds of describers, validators, and transformers each extend System.Attribute which providing an abstract Invoke method which receives as an input argument the attributed thing (an assembly, module, class, method, statement, or expression) and, as envisioned, can also be provided with a second argument, context (with which transformers could wrap the input argument or otherwise construct a new assembly, module, class, method, statement, or expression).

2.1. Annotation

Attributes could be utilized to annotate portions of source code.

2.2. Description

Attributes could be extended into a base class for describers in a manner resembling:

public abstract class Describer : Attribute
{
    protected Describer() { }

    public abstract Description Invoke(object value, object context);
}

As envisioned, the Invoke method on describer attributes would be called with the attributed thing provided as the argument for the value parameter. As envisioned, a context would also be provided. This method would produce either Semantic Web graphs or sets of predicate-calculus statements which each partially describe the attributed thing. The graphs or sets provided by multiple such attributes could be merged together.

2.3. Validation

Attributes could be extended into a base class for validators in a manner resembling:

public abstract class Validator : Attribute
{
    protected Validator() { }

    public abstract bool Invoke(object value, object context);
}

As envisioned, the Invoke method on validator attributes would be called with the attributed thing provided as the argument for the value parameter. As envisioned, a context would also be provided.

2.4. Transformation

Attributes could be extended into a base class for transformers in a manner resembling:

public abstract class Transformer : Attribute
{
    protected Transformer() { }

    public abstract object Invoke(object value, object context);
}

As envisioned, the Invoke method on transformer attributes would be called with the attributed thing provided as the argument for the value parameter. As envisioned, a context would also be provided.

Example uses of decoration or transformation include those uses of decorators and aspect-oriented programming while exceeding those uses as transformer attributes could be placed upon method bodies' statements and expressions.

2.5. Other Ideas

2.5.1. Reversible Transformations

public abstract class ReversibleTransformer : Transformer
{
    protected ReversibleTransformer() { }

    public abstract object Undo(object value, object context);
}

2.5.2. Code Analysis

Special types of attributes for code analysis could extend validators:

public abstract class CodeAnalysisValidator : Validator
{
    protected CodeAnalysisValidator() { }

    public sealed override bool Invoke(object value, object context)
    {
        foreach(var diagnostic in Analyze(value, context))
        {
            if(diagnostic.Severity == DiagnosticSeverity.Error || diagnostic.IsWarningAsError)
            {
                return false;
            }
        }
        return true;
    }

    public abstract IEnumerable<Diagnostic> Analyze(object value, object context);
}

3. Persisting Attributes on Statements and Expressions in .NET Assemblies

How can attributes on statements and expressions in method bodies be persisted in .NET assemblies?

The following approach shows that a new attribute, intended for use on methods, can describe, e.g., by IL offset, other wrapped attributes, with wrapped attributes’ types and constructor arguments passed to the wrapping attributes’ constructor.

[AttributeUsage(AttributeTargets.Method, AllowMultiple = true, Inherited = false)]
public abstract partial class MethodBodyAttributeAttribute : Attribute
{
    public MethodBodyAttributeAttribute
    (
        string fileName,
        int startLine,
        int startColumn,
        int endLine,
        int endColumn,
        int ilOffset,
        Type attributeType,
        params object?[]? ctorArguments
    )
    {
        // ...
    }

    public MethodBodyAttributeAttribute
    (
        int ilOffset,
        Type attributeType,
        params object?[]? ctorArguments
    )
    {
        // ...
    }
}

Above, the parameter attributeType is for the type of the attribute which is utilized in the method body, at an IL offset, and the parameter ctorArguments is for those arguments passed to the wrapped attribute’s constructor.

4. Uses and Examples

4.1 Previous Discussions

Drawing from a previous discussion thread (dotnet/csharplang#2478), subtopics and example uses of attaching attributes to portions of source code include, but are not limited to:

1. Descriptive metadata annotation
2. Functional metadata annotation
   a. Specifying desired capture behaviors for lambda expressions.
3. Analyzers
4. Editor support
   a. One can envision IDEs’ syntax coloring ordinary metadata and describer, validator, and transformer attributes differently.
   b. One can envision the capability to visualize whether a validator attribute is presently returning true or false for the attributed thing, e.g., the portion of source code, that it is placed upon.
   c. One can envision the capability to navigate to and view transformation output source code for transformer attributes.
5. Code generation and expansion
   a. Loop unrolling
   b. Parallelization
   c. Distributed computing
   d. Other
6. Optimizer hints
   a. Branch prediction hints
   b. Method invocation hints, e.g., caller-side inlining hint
   c. Parallelization hints
   d. Distributed computing hints
   e. Other
7. Other
   a. Variable declaration hints
   b. Other hints

Issue dotnet/csharplang#2478 indicates that metadata on things including statements and expressions, method body metadata, perhaps utilizing a [[...]] syntax, would not be persisted in assemblies. This thread explores that at least some metadata attributes on statements and expressions, perhaps utilizing a [body: ...] syntax, would be persisted in assemblies.

4.2. Uses of Decorators

A decorator is a kind of attribute which receives an input thing, e.g., an assembly, class, method, statement, or expression, and returns a modified, e.g., wrapped, result. Decorators are a kind of transformer attribute.

Some preliminary research into decorators, a language feature of Python and a proposed language feature of ECMAScript, indicates that their uses include the following (see also: https://wiki.python.org/moin/PythonDecoratorLibrary):

1. creating well-behaved decorators / "decorator decorator"
2. property definition
3. memoization
4. caching
5. retry
6. pseudo-currying
7. creating decorator with optional arguments
8. controllable DIY debug
9. easy adding methods to a class instance
10. counting function calls
11. alternate counting function calls
12. generating deprecation warnings
13. smart deprecation warnings (with valid filenames, line numbers, etc.)
14. ignoring deprecation warnings
15. enable/disable decorators
16. easy dump of function arguments
17. pre- and post-conditions
18. profiling and coverage analysis
19. line tracing individual functions
20. synchronization
21. type enforcement (accepts/returns)
22. CGI method wrapper
23. state machine implementaion
24. C++/Java-keyword-like function decorators
25. different decorator forms
26. unimplemented function replacement
27. redirect Console.Out printing to logging.
28. access control
29. events rising and handling
30. singleton
31. asynchronous call
32. class method decorator using instance
33. another retrying decorator
34. lazy thunkify
35. aggregative decorators for generator functions
36. function timeout
37. collect data difference caused by decorated function

Many of these uses of decorators are also uses of transformer attributes on methods and on statements and expressions therein.

4.3. Uses of Aspect-oriented Programming

AOP is, according to Wikipedia, "a programming paradigm that aims to increase modularity by allowing the separation of crosscutting concerns." It deals with functionality that occurs in multiple parts of the system and separates it from the core of the application, thus improving separation of concerns while avoiding duplication of code and coupling (see also: https://learn.microsoft.com/en-us/archive/msdn-magazine/2014/february/aspect-oriented-programming-aspect-oriented-programming-with-the-realproxy-class).

Some uses include:

1. the application must have an authentication system, used before any query or update.
2. the data must be validated before it’s written to the database.
3. the application must have auditing and logging for sensible operations.
4. the application must maintain a debugging log to check if operations are OK.
5. some operations must have their performance measured to see if they’re in the desired range.

According to a StackOverflow discussion (https://stackoverflow.com/questions/849994/what-are-your-real-world-uses-for-aspect-oriented-programming), some general uses of AOP include:

1. logging
2. caching
3. security
4. exception reporting
5. performance counters

These uses of AOP are also uses of transformer attributes on methods and on statements and expressions therein.

4.4. Interoperation with Preprocessing Directives

#if TRACE
[TransformForTrace]
#endif
public void Function()
{
    // ...
}

or

#if COLLECT_TIMING
[TransformForCollectTiming]
#endif
public void Function()
{
    // ...
}

Conclusion

Thank you. I look forward to discussing this proposal with you!

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[stephentoub]

Duplicate of dotnet/csharplang#2478

[AdamSobieski]

Duplicate of dotnet/csharplang#2478

@stephentoub, hello. I do not agree that this proposal is a duplicate of dotnet/csharplang#2478. In my opinion, this proposal, which cites and references your 2015 proposal, differs from it in a number of important ways.

Firstly, your 2015 proposal neither seeks to nor addresses any means of persisting attributes on statements and expressions in .NET assemblies. In your proposal, you “propose we introduce an additional [[…]] syntax” and “[…] remains exactly as it is today, whereas [[…]] represents attribution that is source-only and will never be emitted to IL.” This proposal, however, does not propose “source-only attributes” and, instead, provides a solution for persisting attributes on statements and expressions in .NET assemblies as presented in its section 3.

Secondly, this proposal broaches a number of new subtypes of attributes: describers, validators, and transformers, which, together, would increase the usefulness of the .NET attribute system, in general, and would increase the usefulness of placing them on statements and expressions. Transformers, for instance, can be said to resemble the feature known as “decorators” in other languages.

[stephentoub]

It's a duplicate in that it's about exploring ways to put attributes in method bodies where you can't today.

Regardless, your proposal is about the C# language. The csharplang repo is the place for that, not dotnet/runtime.

[AdamSobieski]

I disagree that this issue is a duplicate of dotnet/csharplang#2478.

I have been discussing these topics in the C# language's discussion forum for roughly a week (dotnet/csharplang#7008) and it was recommended to me, there, that I propose it, here, to see if runtime is interested and has uses for these technologies.

[stephentoub]

The very first thing this states is:

It is here proposed that: attributes could be placed on statements and expressions in method bodies, e.g., using a [body: ...] syntax

That is a language issue. This is unactionable in this repo.

[CyrusNajmabadi]

The core issue here is that this appears to be a proposal without concrete, actionable, use cases that real world cases are requesting. In the corresponding C# issue you mentioned things like the runtime being able to use these attributes to better optimize code. However, for that to actually be a compelling use case for the language, we'd need to actually hear it from the runtime team that this is actually something they think they need and would be willing to use for this purpose. Absent that, the justifying use cases would appear to be more speculative, and are not something that we would be likely to design a whole language feature around.

If the runtime team does feel like there are particular needs that justify this sort of feature, then we'd def look into changing hte language for it. But absent desire on their part, it's unlikely to move forward in any meaningful way in either repo.

[AdamSobieski]

@stephentoub, I think that there is an awkward conflict of interest situation here.

Am I supposed to seek an alliance with the proposer of dotnet/csharplang#2478 indicating that an administrator is inadvertently being an obstacle our overlapping ideas to advance .NET and then, subsequently, alongside that proposer, petition the administrator who closed this issue that they should reconsider their decision? Should I, instead, argue to the administrator that the previous, similar issue isn't necessarily indicative of best or current practices some 8 years after it was posted? You are both the proposer of the previous issue and the administrator who closed this issue.

In any event, should I spin-off the new kinds of attributes broached in section § 2 into a new issue for the runtime repository? I would like to explore proposing that these new and useful classes should be explored by the community, refined and editioned, and then added to a .NET system library.

[stephentoub]

I think that there is an awkward conflict of interest situation here.

I'm sorry you feel there's a conflict of interest. I don't see such a conflict, nor am I somehow pushing the issue I opened. I'm happy to stop responding and allow someone else to chime in, but as I already noted multiple times, dotnet/runtime is not the repo for it; there is no code that can be written in dotnet/runtime that will enable the C# compiler to recognize, parse, and lend semantic value to attributes inside of method bodies.

I closed this as a duplicate of that issue because it's fundamentally about the same thing: being able to express attributes inside of method bodies. Whether the attributes are then subsequently erased or are persisted in metadata is the kind of decision that gets made as part of discussion and evaluation in C# language design once a feature has made it to that point. dotnet/csharplang#2478 has been open for seven years without further progress because there hasn't been widespread interest in being able to use attributes in method bodies, regardless of what happens to them after parsing and analysis. My expectation is that for the foreseeable future that level of interest won't change, and that issue (as a representation for being able to use "attributes everywhere") will eventually be closed.

we'd need to actually hear it from the runtime team that this is actually something they think they need and would be willing to use for this purpose

The use cases I'm aware of are the same ones called out in dotnet/csharplang#2478, as a more structured form of trivia that would enable the same kind of interaction with analyzers we have in metadata today but in other places, e.g. on lambdas to indicate what should be allowed to be captured, on arguments to indicate that an analyzer warning about boxing should be suppressed, etc. These are all things we could do today just with comments if we wanted, just as is the case in many cases for attributes used in signatures.

should I spin-off the new kinds of attributes broached in section § 2 into a new issue for the runtime repository? I would like to explore proposing that these new and useful classes should be explored by the community, refined and editioned, and then added to a .NET system library.

Do they provide value even without the ability to use them in method bodies? If they're independent of such a language feature, please feel free to do so.

[CyrusNajmabadi]

I'm happy to stop responding and allow someone else to chime in, but as I already noted multiple times, dotnet/runtime is not the repo for it; there is no code that can be written in dotnet/runtime that will enable the C# compiler to recognize, parse, and lend semantic value to attributes inside of method bodies.

Thanks for that perspective stephen. The request to come here was to help gauge if anything had changed much in the last seven years. e.g. if this still remained a nice-to-have, but ok-to-not-have, idea. Or if this had raised significantly as something the runtime team would very much want. And thus we would potentially work towards adding language support as well as necessary APIs in teh runtime for encoding/decoding this information.

Adam raised ideas that runtime needed this so that users/library/bcl authors could add things like inlining/branch-prediction/other-optimization attributes directly in-situ in the code, and that that would be an important use case for the runtime. Given the lack of interest in that sort of area, i'm going to assume that that's not currently a pressing area of interest and that the proposal remains similar to where it's always sat (interesting, but not rising to the level of important).

If things change, let us know! :)

[stephentoub]

add things like inlining/branch-prediction/other-optimization attributes directly in-situ in the code

When we've discussed that in the past, we haven't discussed it with attributes, rather with intrinsic methods, e.g. #6225

[CyrusNajmabadi]

When we've discussed that in the past, we haven't discussed it with attributes, rather with intrinsic methods, e.g. #6225

Perfect, thanks for the clarification!

[AdamSobieski]

Do they provide value even without the ability to use them in method bodies? If they're independent of such a language feature, please feel free to do so.

Thank you. Yes, they are separable and can provide value for various other attributable things. I'll write something for the runtime discussion area (in the next week or so) about these new kinds of attributes which each extend System.Attribute and provide an Invoke method and which receives, as an argument, the attributed thing.