[SUGGESTION] Treat a keyword as an identifier

[msadeqhe]

Preface

I suggest to use "keyword"$ syntax to treat keywords as identifiers in Cpp2.

Suggestion Detail

Some identifiers in Cpp2 are keywords in Cpp1 such as and, or, etc. These identifiers aren't valid identifiers in Cpp1. Therefore Cpp2 appends cpp2_ prefix to these identifiers during code generation for Cpp1 as discussed in this issue. For example identifiers and, or, etc in Cpp2 will respectively become identifiers cpp2_and, cpp2_or, etc in generated Cpp1 code.

On the other hand, some keywords in Cpp2 aren't keywords in Cpp1 such as type, next, etc. These keywords are valid identifiers in Cpp1. Therefore it's not possible to use such identifiers in Cpp2 when dealing with Cpp1 API:

// ERROR! `type` is a keyword, whereas it's an identifier in Cpp1.
v0: = type.name();

// ERROR! `next` is a keyword, whereas it's an identifier in Cpp1.
while next < 10 next next++ {
    //: statements...
}

I suggest to use a syntax such as "keyword"$ to access identifiers from Cpp1 API in which they are keywords in Cpp2:

// OK. `type` is an identifier.
v0: = "type"$.name();

// OK. `next` is an identifier.
while "next"$ < 10 next "next"$++ {
    //: statements...
}

I think syntax "keyword"$ is good enough because Cpp2 will have similar syntax for reflections and code generations as described in this page of Wiki. And semantically this is related to them.

Your Questions

Will your feature suggestion eliminate X% of security vulnerabilities of a given kind in current C++ code?

No.

Will your feature suggestion automate or eliminate X% of current C++ guidance literature?

No.

Considered Alternatives

I considered to use syntax @keyword but it would resemble meta-class functions. Another possible syntax is @"keyword", which doesn't resemble meta-class functions in addition to its difference from capture syntax.

Also we can use a prefix such as cpp1_. For example:

// OK. `cpp1_type` will become `type` in Cpp1.
v0: = cpp1_type.name();

// OK. `cpp1_next` will become `next` in Cpp1.
while cpp1_next < 10 next cpp1_next++ {
    //: statements...
}

In this way, cpp1_ prefix in addition to cpp2_ prefix should be reserved for Cpp2 compiler, and user-defined identifiers with those prefixes shouldn't be allowed.

By the way, treating a keyword as an identifier, isn't frequency needed to dedicate a new symbol to it.

EDIT 1: keyword<> is another alternative solution as described in this comment.

[JohelEGP]

Therefore it's not possible to use such identifiers in Cpp2 when dealing with Cpp1 API

I've actually found the Cpp2-specific keywords to be contextual, except for the <cstdint> shorthands (e.g., i8), and inspect. So this works: https://godbolt.org/z/vYPf6MG6h.

int next = 0; // Cpp1.
f: () = {
    while next < 10 next next++ {
    //: statements...
    }
    while 10 < next next next++ {
    //: statements...
    }
}
main: () = {
    :(is) = { [[assert: (is)is(is)]] }(:() = {});
}

Including the :(is) = { [[assert: (is)is(is)]] }(:() = {});.

[msadeqhe]

You're right 😬, that's very nice but it doesn't work in every situation, for example:

type: type = {}

main: () = {
    variable: type = 0;
}

[JohelEGP]

I prefer cpp2_type
over "type"$
to lower to Cpp1 as type.

[msadeqhe]

Also Cpp2 may have more keywords during language evolution in the future, therefore keywords such as result in this issue are already valid identifiers in Cpp1.

[JohelEGP]

For variable: type = (), you want type to be a Cpp1 identifier. In this case, it clashes with Cpp2 syntax itself. It can be solved with an indirection: std::type_identity_t<type>: https://godbolt.org/z/TGoqsebrW.

[JohelEGP]

This are the sets of Cpp2-specific keywords:

{i8,       i16,       i32,       i64,       u8,
 u16,      u32,       u64,       i8_fast,   i16_fast,
 i32_fast, i64_fast,  u8_fast,   u16_fast,  u32_fast,
 u64_fast, i8_least,  i16_least, i32_least, i64_least,
 u8_least, u16_least, u32_least, u64_least, inspect});
{next, copy, move, forward, pre, post, final,
 in, 
 as, is, type, 
 assert, throws,  
 implicit, out,   
 inout});            

I can't come up with another example like #392 (comment)'s.
A simpler alternative for variable: type = ()
is qualifying type: variable: ::type = () https://godbolt.org/z/W3v48se5K.

[msadeqhe]

Thanks. It's a good idea to use helper type std::type_identity_t<type> or write the qualified name ::type. But ::type doesn't work for local types:

main: () = {
    type: type = {}
    variable: ::type = ();
}

[JohelEGP]

Well, the only reason to name it type in that case, would be for some Cpp1 library that reflects the identifer, or a wicked Cpp2 one.

[MichaelCook]

Perhaps a backslash prefix would be a little more readable.

v0: = \type.name();
while \next < 10 next \next++ {
    //: statements...
}```

[JohelEGP]

Makes me think if raw string literals could just be backslash-escaped (see #302).

[JohelEGP]

Another place where backslash-escaping provides simpler solutions:

• my_type: @struct <T: type> type = { vs.
  my_type: @struct \<T> type = {
  to disambiguate a template-id's template argument list from a template-parameter-list.
  I think <T is C> is going to reduce my needs for this to 0 ([BUG] Non-type template parameter used as _type-id_ is not diagnosed #575 (comment)).
• x++ vs. x\++ to do a Cpp1 post-increment instead of a Cpp2 increment (Cpp1 pre-increment).
  Although this might be too subtle, and deserve a louder incantation.

[msadeqhe]

@MichaelCook, I like the idea to use backslash syntax \keyword.

Escape sequences in string literals start with a backslash to change the meaning of a character, why not just use it to change the meaning of a keyword and treat it as an identifier?

@JohelEGP, Good idea but in my opinion, raw (non-interpolated) string literals break the general capture syntax (thing)$ everywhere in the language. Also string literals without prefix or suffix will make it possible to have operator'' and operator"" or Tagged Template Strings or any other versatile syntax in Cpp2.

[msadeqhe]

@MichaelCook, I like the idea to use backslash syntax \keyword.

Because it only works on keywords, if Cpp2 has binary operator \, it wouldn't conflict with it:

// Treats `type` keyword as identifier.
x: \type = ();

// binary operator \
y: = var0 \ var1;

Also unary prefix and postfix operators \ wouldn't conflict with \keyword because they would be applied to identifiers.

[msadeqhe]

These are other use cases and reasons why it does matter in addition to C and Cpp1 language interop:

• When Cpp2 introduces a new keyword in the future, a simple find and replace from keyword to \keyword (or whatever else) helps programmers to migrate the source code of their programs and libraries.
• \keyword (or whatever else) is more readable than workarounds to treat a keyword as an identifier.
• It allows to use keywords for the name of interfaces and functions in APIs:

variable.\if(count < 10).\return();

[jcanizales]

Therefore it's not possible to use such identifiers in Cpp2 when dealing with Cpp1 API

Could you give an example C++ API that would not be usable from Cpp2? Naming variables is not such a case.

[MichaelCook]

How's this:

struct myapi_t {
    int type;
};

[JohelEGP]

See #392 (comment).
https://godbolt.org/z/GoK86hn53:

struct myapi_t {
    int type;
};
main: () -> int = myapi_t().type;

[msadeqhe]

Also Cpp1 type traits in standard library have a member type named type. For example:

std::is_integral<T>::type

[msadeqhe]

Also const doesn't work as a type name, although both Cpp1 and Cpp2 have const keyword:

const: type = {}

main: () = {
    x: const = "";
}

But it works with qualified name. By the way Cpp2 generates invalid Cpp1 code:

const: type = {}

main: () = {
    x: ::const = "";
}

[msadeqhe]

According to @JohelEGP's helpful comment:

...
I can't come up with another example like #392 (comment)'s.
A simpler alternative for variable: type = () is qualifying type: variable: ::type = () ...

If Cpp2 accepts all keywords as identifiers with qualified names, and if Cpp2 would allow a way to have qualified names for local declarations (e.g. _::identifier), and if Cpp2 could treat keywords as identifiers in expressions where there is an operator, I think it's going to solve the issue, because it's a general rule and easy to understand/follow:

// `while` is not a keyword, because it's left-hand-side of `:`.
while: type = {
    // `return` is not a keyword, because it's left-hand-side of `:`.
    return: () = {}
}

main: () = {
    // `do` is not a keyword, because it's left-hand-side of `:`.
    // `do` is a local type.
    do: type = { /* definition */ }
    // `_::do` is a qualified name.
    variable: _::do = /* definition */

    // `for` is not a keyword, because it's left-hand-side of `:`.
    // And `while` is an identifier.
    for: ::while = ();
    // `for` is an identifier becuase there is an operator dot.
    // And `return` is an identifier too.
    for.return();

    // Also to clarify, `for` may be required to be quialified, but it's a little restrict.
    _::for.return();

    // `for` is a keyword, because there isn't any operator between `for` and `args`.
    for args do (arg) { /* statements */ }

    // `return` is not a keyword, because it's left-hand-side of `:`.
    return: (param) -> ::while = ();

    // AMBIGUOUS!
    // When there is an ambiguous, they are keywords.
    // So `return` is a keyword here.
    return(/* something */);

    // This `return` is not a keyword, because its name is qualified.
    _::return(/* something */);

    // `if` and `for` are not keywords, because there is an operator between them.
    value: = if * for;

    // `do` is not a keyword, because it's already a qualified name.
    my_namespace::do();
}

In this way, keywords would be fully contextual (except for operator() becuase it means both for function call and grouping), and any new syntax won't be required to treat keywords as identifiers.

EDIT

This way has a problem if Cpp2 supports operator overloading for '' and "" in the future (of course perhaps Cpp2 never supports them but it's a possibility to consider):

// This is AMBIGUOUS:
//   Is `return` an object with `operator""`?
//   Or is `return` a keyword and returns string `"something"`?
return "something";

// This is OK. `return` is an object with `operator""`.
_::return "something";

It is similar to how operator() is ambiguous.

[msadeqhe]

To simplify the rule, maybe Cpp2 should require that every identifier to be qualified in which their name are equal to keywords:

// `_::` must be omitted for naming declarations.
while: type = /* definition */
do: _::while = /* definition */

// `_::` must be omitted for accessing members with :: or dot.
// Because these are already qualified names.
_::do::type.as.forward;

_::return(args);

// OK. Keywords cannot be qualified names.
value: = _::if * _::for;

// `inspect` is not a keyword, because it's already a qualified name.
my_namespace::inspect();

So they must be always qualified either namespace::keyword for identifiers inside a namespace or ::keyword for global identifiers or _::keyword for local identifiers (if Cpp2 supports something similar).

Instead of "keyword"$ or @"keyword" or \keyword, it's going to be namespace::keyword or ::keyword or _::keyword. Those are alternative ways to achieve the same thing...

[msadeqhe]

Instead of "keyword"$ or @"keyword" or \keyword, it's going to be namespace::keyword or ::keyword or _::keyword. Those are alternative ways to achieve the same thing...

Another option to consider is keyword<>.

Types, functions and variables can be templates already. So keyword<> would mean a type, function or variable.

if: (condition: bool, forward yes_value: int, forward no_value: int) -> forward int = {
/* Or force the template notation <> within declaration:
if: <> (condition: bool, forward yes_value: int, forward no_value: int) -> forward int = {
*/
    if condition {
        return yes_value;
    }
    else {
        return no_value;
    }
}

if: <T> (condition: bool, forward yes_value: T, forward no_value: T) -> forward T = {
    if condition {
        return yes_value;
    }
    else {
        return no_value;
    }
}

main: () = {
    // if<> is not a keyword here.
    x: = if<>(2 * 2 == 4, 1, 0);

    // if<bool> is not a keyword here.
    y: = if<bool>(2 * 2 == 4, true, false);

    // if<> is not a keyword here.
    z: = (2 * 2 == 4).if<>(1, 0);
}

But keywords cannot be templates, so template arguments after keywords, mean they are not keywords.

This requires empty template argument list as described in this bug.

[msadeqhe]

But keywords cannot be templates, so template arguments after keywords, mean they are not keywords.

Also it depends if Cpp2 would have similar Cpp1-style static_cast<X>(arg), dynamic_cast<X>(arg) and etc (keywords which look like templates) or not.

Or would Cpp2 have stand-alone <something> as an expression (without identifier before it)?

if <something> { ... }

That's suggested in this issue.

[msadeqhe]

Well, I learned that I can treat a keyword as an identifier indirectly while reading Design note: Capture:

if_name: = "if";

// Type "if"
if_name$: type = {
    test: (this) = {
        print("Something");
    }
}

for_name: = "for";

// Function "for"
for_name$: () = {
    print("Something");
}

// It creates an instance of type "if"
x: if_name$ = ();

// It creates an instance of type "if", and calls member function "test"
y: = if_name$().test();

// It calls function "for"
a: = for_name$();

But I don't know if the syntax is available outside code generation.

[msadeqhe]

Or simply we may have a semi-direct but ugly syntax like this:

// Type "if"
(:="if")$: type = {
    test: (this) = {
        print("Something");
    }
}

// Function "for"
(:="for")$: () = {
    print("Something");
}

// It creates an instance of type "if"
x: (:="if")$ = ();

// It creates an instance of type "if", and calls member function "test"
y: = (:="if")$().test();

// It calls function "for"
a: = (:="for")$();

In general: (:="keyword")$

[msadeqhe]

Of course if Cpp2 supports unnamed variable without explicit type, in general (:="keyword")$ can also be used for identifiers with spaces and more. For example we can declare the following variables in which they are invalid today:

(:="my variable name")$: int = 10;
(:="[email protected]")$: int = (:="my variable name")$ + 2;

[JohelEGP]

I don't think that'll work.
It's the same as directly using the keyword as an identifier.
Because eventually the strings will be parsed as Cpp2.

[msadeqhe]

It's capturing an unnamed variable:

unnamed_variable: = "[email protected]";
x: = unnamed_variable$;

It's equal to:

x: = (: = "[email protected]")$;

[SebastianTroy]

Why would this feature be desirable? On 11 September 2023 14:33:53 Johel Ernesto Guerrero Peña ***@***.***> wrote: I don't think that'll work. It's the same as directly using the keyword as an identifier. Because eventually the strings will be parsed as Cpp2. — Reply to this email directly, view it on GitHub<#642 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AALUZQOTS7DGWHJMSA7W5ATXZ4HLJANCNFSM6AAAAAA4S7N35I>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[Eljay-Adobe]

The general language design technique is called stropping.

Might be of interest to see how other languages have solved the problem. See link to wikipedia article.

[hsutter]

Might be of interest to see how other languages have solved the problem

That reminds me of our experience in C++/CLI: We found this to be important, and chose to add the feature as __identifier(keyword) in the C++/CLI standard. IIRC part of the rationale was to make it visible. Reference: ECMA-372, section 9.1.1

[hsutter]

tl;dr

Thanks! Yes, I'll implement this... but not for Cpp1 compatibility (I don't think it's needed? Cpp2 is already 100% compatible with Cpp1 libraries), but rather from cross-language interop with other non-C++ languages.

I don't think we need this feature for compatibility with Cpp1 code -- does anyone have counterexamples?

Sorry if I missed it, but has anyone come up with an example of a need to turn a Cpp2 keyword into an identifier in order to use an existing Cpp1 library?

• Cpp2 keywords are contextual, they are not globally reserved words. For example, std::is_integral<T>::type works fine even though it uses the word type; for example, my_type_alias: type == std::is_integral<int>::type; works.
• The only two new Cpp2 keywords that currently are in the internal list of Keywords are is and as, and that's an internal implementation detail, it doesn't reserve them. They appear to work fine -- see above comment.

But I'm concerned about abuse of the proposed feature: It's not necessary, but would be actively harmful, to let people write Cpp1 reserved words like if as identifiers.

Disclaimer: Yes, in Cpp2 I do take back a few Cpp1 reserved words as identifiers, notably new, class, struct, enum, union, and the logical and bitwise alternative names. But I try to be disciplined about that, and do it mainly when I'm taking what in Cpp1 is a language feature and providing it in Cpp2 as a library (e.g., a safe allocator object, or type metafunction name) and so I think it's justifiable that the feature be able to keep its nice name. But it's not intended to be an open-ended land rush on Cpp1 reserved words.

But we do have prior art, that that prior art shows it's needed for compatibility with non-C++ code

As I mentioned above, in C++/CLI we added this feature as __identifier(keyword). But the reason we needed it was to enable writing code in (extended) C++ that would be compatible with using existing libraries written in other languages that might use C++ reserved words as identifier. A canonical example, cited at the above link, is __identifier(template).

The comparable use case will be to enable writing Cpp2 code that's compatible with libraries written in non-C++ languages, and I know that's coming, so I'll add the requested feature. For example, one of the planned uses of metafunctions is to use code generation to author a type in another language, say my_type: @java_interface type = { /*...*/ }. This has many legitimate use cases, such as automatically exposing a C++ type natively to C# or Java or Swift. Now imagine a metafunction that generates a type that implements an interface defined in some other language's major frameworks that has a method named template... and now we do need __identifier__template.

I really was serious when I've said (since 2017 in P0707 and metaclass (now called type metafunction) talks that I really never want to design C++/CLI and C++/CX and C++ AMP again as language extensions. I would have done those as libraries had it been possible then, but C++ wasn't powerful enough then (and still isn't now) to get the usability we felt was needed, so we had to resort to doing them as language extensions implemented in an extended C++ compiler. A major goal of Cpp2, and notably of type metafunctions including reflection and source code generation, is to be able to develop such technologies as libraries implemented in compile-time code.

Spelling

My current preference is to spell the escape hatch as __identifier__keyword. It's simple because that will already lex as an identifier, and so I just need to add one line to have the lexer remove the __identifier__ prefix. It's simple to implement (I just did), and I like that __identifier__ is ugly because unusual things should stand out.

Thanks!

[JohelEGP]

It's simple to implement (I just did)

For reference (because Discussions aren't link-history friendly)
the commit is 1043622.
And this Discussion needs to be manually closed because commits can't close them either.