Token name - reserved words

[kevinmpowell]

Are there any reserved words that should not be used in token names?

Examples: 'default', 'int'

{
  default: {
    value: '#000000'
  },
  int: {
    value: 50
  }
}

Will these (and others) cause problems when converted to various languages and platforms?

Resolution: #61 (comment)

[jina]

Hmm. We use default quite a bit at work! 😛

[blackfalcon]

The purpose of reserved words is to keep users from colliding with words that have a specific meaning to the language. Is there a language taxonomy for this project? Like @jina pointed out, people use default.

I am curious why you picked default as an example. Or int, what would that collide with here?

[kevinmpowell]

I've seen default create collisions in the Swift language on iOS.
See: https://stackoverflow.com/questions/49352629/why-cant-i-put-an-enum-case-in-single-quotes-which-is-of-type-int

[jina]

oh interesting. for some reason that's not been an issue for our ios tokens yet. I'll bring it up with our team just in case. :)

[c1rrus]

There's a bunch of property names that have a special meaning in our spec, so I believe these necessarily need to be reserved words: value, type, description, extensions...

It might be a good idea to have a section somewhere that lists them all for convenience.

It's likely that future spec iterations will add more (we've already been talking about things like deprecated, version & author), but I don't know if we should "landgrab" some others that we think we might need in future.

If we do, we may unnecessarily limit what people can name their tokens and groups (e.g. it seems default is already being used as a token name by some teams).

If we don't, we may may find it hard to add more properties in future spec versions.

So, how about this as an alternative: What if all spec properties started with a certain symbol. Taking a bit of inspiration from JSON schema, we could use a $ prefix.

So, value becomes $value, type becomes $type and so on. An example file might then look like this:

{
  "$description": "The description of this group",
  "token name": {
    "$value": "100px",
    "$type": "dimension"
  },
  "another token": { 
    "$value": "#abcdef",
    "$type": "color"
  }
}

Pros:

• Easier to avoid naming clashes in the future
• Perhaps makes it easier to see which properties are spec things and which are author-defined token and group names

Cons.

• Might be a bit trickier to work with in code (e.g. in JavaScript you'd always need to access things like token['$value'] intead of token.value Thanks to @TravisSpomer for pointing out that JS vars can begin with '$'. However, other programming languages like Python do mandate that variables must begin with a letter or number)
• Is a fairly big change from the current draft spec, which has already been reviewed by several design tool vendors

Thoughts?

[jina]

Turns out the way we are exporting our tokens and for the code gen we have, this isn't an issue for us at work. Phew!

[ivnmaksimovic]

@c1rrus

There's a bunch of property names that have a special meaning in our spec, so I believe these necessarily need to be reserved words: value, type, description, extensions...

This could be avoided, as you suggested, with some prefix like $, so only $value is special and value could be a valid token name. But as you also pointed out accessing things like token['$value'] instead of token.value is not so nice.

Another way would be to add just one special keyword tokens.

{
  "description": "The description of this group",
  "value": "just a custom property"
  "tokens": {
    "any word can be a token name": {
        "value": "100px",
        "type": "dimension"
    },
    "value": { 
        "value": "#abcdef",
        "type": "color",
        "description": "this token is used for item value"
    },
    "description": {
        "value": "#ffffff",
        "type": "color",
        "description": "this token is used for description texts"
    }
  }
}

Root level tokens would also not need any exception if we threat mysystem.tokens just as we treat tokens.

{
    "token name": {
        "value": "100px",
        "type": "dimension"
    },
    "value": { 
        "value": "#abcdef",
        "type": "color"
    },
    "description": { 
        "value": "#ffffff",
        "type": "color"
    }
  }

I wrote a maybe too big example in #55 (comment)

[kevinmpowell]

I'm not keen on the $ prefix or the addition of a 'tokens' node. Both decrease readability IMO, and the $ creates the access issues @c1rrus noted.

I think documenting our current reserved words and having a rule in the spec that a token cannot be named:

• "value"
• "description"
• "extensions"
• "type"

is reasonable. If we need to add new reserved keys in the future we can circulate those in the community to gauge the level of collision. It does mean future versions of the spec that add new reserved keys would be breaking changes, which is a bummer, but my gut says we're over engineering this if we're trying to avoid those collisions already.

[ChucKN0risK]

I agree with @kevinmpowell here 👍

[ivnmaksimovic]

Additional tokens node is just one suggestion, but it fully solves the problem of reserved words and collisions, both today and in future. At least I didn't find a case when this would not work. I would also argue that having tokens under tokens node increases readability, not decrease.

Prefixes are also solving the problem and the only con I see is ease of access if prefix is $. Some other prefix might avoid that issue. But which one? Is this worth discussing more? Maybe the readability is decreased, but with "the right" prefix, it might also increase readability it and make it easy to differentiate tokens from groups and other properties.

Current proposal with 4 reserved keywords is ok today, but I am pretty sure that it will not be enough as the project matures. Then the choice will be either breaking change, which is IMO a huge deal in projects like this, or some hacky solutions where everything is for example added to extensions just to avoid breaking change.

Given the impact of these decisions, I would like to ask you for a bit more discussion, both around possible prefixes, additional node, and impact of breaking change if a new reserved word is added later.

[TravisSpomer]

I've been working on building tooling for tokens at Microsoft for a couple of years now, and one of my biggest gripes with Style Dictionary today is how they used value instead of $value. If you're writing a tool for token authoring, what do you do if someone tries to name one value? Put up a message in red that says "Sorry, you're not allowed to use that word?" Keeping type as a reserved word feels even worse since "type" is a common word that designers use when referring to text styles.

I'm very much in favor of prefixed keywords like $value and $type. I'm not sure I follow why it's harder to use those from JavaScript code—$ is an allowed character in JavaScript identifiers.

const tokens = { red: { $value: "#ff0000ff" } }
console.log(tokens.red.$value)

[ivnmaksimovic]

Funny how we all just accepted that $value has access issue without even checking. @TravisSpomer is correct. We would not need to access things like token['$value'] as token.$value is perfectly valid syntax.

So what would actually be downside of this except maybe personal preference? My personal preference is still the idea of having tokens or maybe $tokens node for tokens because of groups and group properties.

But let's not make decisions based on personal preferences. And let's not just try to confirm that initial idea with reserved words is good. I think the modern term for it would be confirmation bias ;)

[nesquarx]

Reserved words are definitely good. Just less common words are better, hence the $.

…

On Thu, Dec 2, 2021 at 7:51 PM Ivan Maksimovic ***@***.***> wrote: Funny how we all just accepted that $value has access issue without even checking. @TravisSpomer <https://github.com/TravisSpomer> is correct. We would not need to access things like token['$value'] as token.$value is perfectly valid syntax. So what would actually be downside of this except maybe personal preference? My personal preference is still the idea of having tokens or maybe $tokens node for tokens because of groups and group properties. But let's not make decisions based on personal preferences. And let's not just try to confirm that initial idea with reserved words is good. I think the modern term for it would be confirmation bias ;) — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#61 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEKS36GCR4IYPGB3LXLU7E3UO56HHANCNFSM5DXWWQCA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[c1rrus]

You're quite right @TravisSpomer, thanks for pointing out my mistake about things in JS not being allowed to begin with $ (now that I think about it, I used to use jQuery back in the day so I knew this was OK😝). I've updated my comment aboved accordingly. The same might not be true in other programming languages though and we shouldn't assume that every tool that wants to read or write design token files will be written in JavaScript.

I do agree this topic needs further consideration and debate.

@ivnmaksimovic's suggestion of nesting a group's children under a tokens property and thereby allowing the top-level group object to be reserved for only format-specific properties is quite elegant. However, if you don't use any special group properties, then it might look a bit verbose.

{
  "group-1": {
    // "special" group properties 
    "description": "Only the coolest tokens get to be in this group!",
    "type": "dimension",

    // this group's items:
    "tokens": {
      "token-1": { "value": "2rem" },
      "token-2": { "value": "99px" },
      "nested-group-1": {
        "description": "A nested group",
        "tokens": {
          "deep-token-1": { "value": "1rem" },
          // ...
        }
      },
      // ...
    }
  },
  
  // So far, so good. But when your group doesn't use any special properties
  // you still need put the items inside a tokens prop.
  "group-2": {
    "tokens": {
      "token-A": { "value": "#ff0000", "type": "color" },
      "token-B": { "value": "#00ff00", "type": "color" },
      "nested-group-A": {
        "tokens": {
          "deep-token-A": { "value": "#00ff00", "type": "color" },
          // ...
        }
      },
      // ...
    }
  }
}

What if we flip the idea though? Rather than nesting the group's items, we nest the special properties instead. Let's imagine we have a special metadata property for groups and descriptions, default types, etc. all go in there. The example above could then be written as:

{
  "group-1": {
    // "special" group properties 
    "metadata": {
      "description": "Only the coolest tokens get to be in this group!",
      "type": "dimension"
   },

    // this group's items (now at the top level):
    "token-1": { "value": "2rem" },
    "token-2": { "value": "99px" },
    "nested-group-1": {
        "metadata": {
          "description": "A nested group"
        },
        "deep-token-1": { "value": "1rem" },
        // ...
      }
    // ...
  },
  
  // Groups that don't use any special properties remain the same as they are
  // in the current spec draft
  "group-2": {
    "token-A": { "value": "#ff0000", "type": "color" },
    "token-B": { "value": "#00ff00", "type": "color" },
    "nested-group-A": {
      "deep-token-A": { "value": "#00ff00", "type": "color" },
      // ...
    }
    // ...
  }
}

We'd still have the same benefits:

• We only need one reserved word (e.g. metadata) that group items can't use as their name
• No risk of naming clashes with group items if we want to add more special properties in the future

What do you think?

[gearsandcode]

I certainly prefer the idea of nesting the special properties. That seems to align with Style Dictionary's CTI convention out of the box. This allows you to filter on the special properties without needing to loop through the properties. I like grouping them in something like metadata that can be ignored by default during processing. You can then easily output something like .format where the output is shallow and the metadata is excluded and .raw.format where the metadata is included.

[kevinmpowell]

I support @c1rrus' proposal to use a metadata key to group all "non-token name" properties.
This would mean having a rule in the spec that a token cannot be named:

• "metadata"

But this also means we wouldn't need to expand the list of reserved words beyond "metadata"

[dbanksdesign]

I like having metadata for non-token stuff. We still need other reserved words like value and type though right?

[ChucKN0risK]

I also like keeping things simple and having metadata for non-token stuff.

+1 to reserve value and type as well.

[c1rrus]

With the changes in PR #89 only metadata and value need to be reserved words, in the sense that tokens or groups cannot be named like that.

Since type (and other properties like description) only have special meaning for groups if they are within metadata, there's no need to forbid using them as group / token names.

The reason value needs to be a reserved word, is because it's presence is how parsers can differentiate between tokens and groups. I.e. if an object has a value property, it must be a token, not a group.

[donnavitan]

Ahoy, late to the chat. I'm trying to figure out what counts towards "non-token name" properties when avoiding reserved words.

[c1rrus]

FYI: There's been some good debate over on PR #89 which I had started with the intent of updating the spec to use the metadata approach outlined above.

The main concern is the inconsistency between groups (where "special" properties are nested under metadata) and tokens (where they are top-level properties) that this would introduce. The worry is it might make the file format a little confusing for folks to learn or read.

The format editors discussed this at our last meeting and agreed this needs some more thought before we decide how to proceed.

In light of that, I'm starting to wonder if it's worth revisiting the $ prefix approach I proposed earlier in this thread (@kaelig pointed out on our call that HTML does something similar - it requires custom elements always have a hyphen in their name and therefore all unhyphenated words are reserved words that could be used for future HTML elements) . Rather than having several reserved words, we'd instead have a reserved prefix. Tokens and groups would be forbidden from using that prefix for their names. If we choose a special character like $ then hopefully that restriction on naming won't be too bad for most teams. This approach is:

• Future proof: Future spec versions can add as many new $... properties as they want and there's no risk of naming clashes with token and group names
• Consistent: Assuming we apply the same prefix to all token properties, then groups and tokens share a familiar syntax.

What do you think? Is this perhaps a better alternative to the .metadata route we've been going down?

[ilikescience]

I'm all for using a prefix instead of metadata! I have a very slight and personal aesthetic preference for using _value instead of $value because the $ looks like an S. Very slight and very personal :)

[nesquarx]

Prefixes also improve human readability.

…

On Fri, 14 Jan, 2022, 12:28 am Matthew Ström, ***@***.***> wrote: I'm all for using a prefix instead of metadata! I have a very slight and personal aesthetic preference for using _value instead of $value because the $ looks like an S. Very slight and very personal :) — Reply to this email directly, view it on GitHub <#61 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEKS36AISKXZYDK7HHE3DDLUV4OHHANCNFSM5DXWWQCA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you commented.Message ID: ***@***.***>

[ivnmaksimovic]

I don't see any flows with reserved prefix approach and I agree that it improves human readability. Unlike other proposals where user would have to learn more about the format, prefixes would serve as a hint that those are special properties. I also agree that $ is not the nicest prefix, but maybe not a big deal.

[TravisSpomer]

$ has my vote primarily because it's what's used in JSON Schema to indicate "special stuff," so it seems like there's precedent.

[c1rrus]

Does anyone have objections to going down the prefix route?

If not, I think we should press ahead and update the spec draft accordingly.

All that remains is to choose a prefix. I don't have a strong preference myself.

As @TravisSpomer noted, $ is similar to what JSON Schema uses for some of its keywords. That may or may not be a reason for our format to use the same character.

JSON-LD on the other hand uses @ as a prefix for "special" things, so that might be another alternative to consider.

And then there's the _ as @ilikescience has proposed.

How about we do a vote? please react with the emoji that corresponds to your preferred prefix:

Prefix	Emoji
$ (dollar sign)	😄
@ (at sign)	❤️
_ (underscore)	🚀
Other - suggest in a comment	👀

[ilikescience]

@c1rrus great idea on the poll! Let's close this puppy out 😄

[ivnmaksimovic]

If it is matters at all, quite a few languages like C++, C#, Java, Python allows variable name to start only with a-z, A-Z and "_". (result of quick search. I haven't really checked that).

[kevinmpowell]

Editor decision 1/25/22: moving forward with $ as a prefix for reserved words at the group and token level.