Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Native modes and theming support #210

Open
jjcm opened this issue Mar 22, 2023 · 59 comments
Open

Native modes and theming support #210

jjcm opened this issue Mar 22, 2023 · 59 comments

Comments

@jjcm
Copy link

jjcm commented Mar 22, 2023

We're currently working on native support for tokens internally here at Figma. In our eyes there are two core use cases that stem from customer requests for design tokens:

  1. Token aliasing (i.e. danger-bg -> red-300)
  2. Theming

Currently the spec does not support theming, which at the moment is a blocker for us for full adoption. I'd like to start a thread here on what native mode support would look like for this format. Major shout out to @drwpow for trailblazing some of this with Cobalt-UI, to @jkeiser for turning this into a proper proposal, and to @connorjsmith for comments and critiques.

Here's the proposal we ended up with:

Overview

Modes represent alternative sets of values for a collection of design tokens. For example, one might wish to have a different value for the “background” and “foreground” tokens depending on whether they are in “light” mode or “dark” mode.

This proposal allows the user to define a set of modes that apply to all tokens in the design tokens file, allowing them to have distinct values for each mode.

Herein we’ll use this example:

{
  "$name": "Figma UI Colors",
  "$modes": {
    "light":      {}, // no fallback
    "dark":       {}, // no fallback
    "super-dark": { "$fallback": "dark" }
  },
  "bg": {
    "$type": "color",
    "brand": {
      "$value": "{colors.blue.300}", // light mode falls back to this
      "$modes": {
        "dark": "{colors.blue.500}" // super-dark mode falls back to this
      }
    }
  },
  "fg": {
    "$type": "color",
    "brand": {
      "$modes": {
        "light": "{colors.black}",
        "dark": "{colors.white}",
        "super-dark": "{colors.gray}"
      }
    }
  }
}

In this example, the values for bg and fg for each mode would be:

  light dark super-dark
bg {colors.blue.300} {colors.blue.500} {colors.blue.700}
fg {colors.black} {colors.white} {colors.white}

Defining Modes

A design tokens file may optionally define a set of named modes at the top of the file.

{
  "$name": "Figma UI Colors",
  "$modes": {
    "light": {},
    "dark": {},
    "super-dark": { "$fallback": "dark" }
  },
  // tokens ...
}

The $modes definition is an object at the top level of the design tokens file.

  • $modes should be placed before the first token or token group, to make efficient file import possible.
  • Mode names are case sensitive: "``light``" and "``LIGHT``" are different modes.
  • Mode names have the same restrictions as token and group names: they must not start with $, and must not contain {, . or } characters.
  • If $modes is empty {}, it is treated the same as if it were not specified (namely, that no modes are defined).

Fallbacks

Each mode may optionally define a $fallback mode, which will be used to determine the value of tokens which do not define a value for the given mode.

  • The lack of a $fallback value implies that mode will fall back to a token’s default $value.
        "dark":       {}, // no fallback
  • $fallback value must be the name of another mode in the same file.
        "super-dark": { "$fallback": "dark" }
  • Fallbacks must not form a cycle.
        "dark": { "$fallback": "super-dark" },
        "super-dark": { "$fallback": "dark" } // ERROR: cycle

Token Values

Design token files may specify different values for each mode, for each token.

    "brand": {
      "$value": "{colors.blue.300}", // light mode falls back to this
      "$modes": {
        "dark": "{colors.blue.500}" // super-dark mode falls back to this
      }
    }
  • A token may optionally define $value, which determines its default value.
  • If no modes are defined, $value must be defined and represents the token’s value.
  • A token may optionally define $modes, which is an object defining its value for specific modes.
  • A token’s $modes may only define values for modes defined in the same file. "$modes": {"daaaaark":"#000000"} is an error if there is no "daaaaark" mode.
  • "$modes": {} is equivalent to not specifying $modes at all.

NOTE: this relaxes the requirement that $value is required when modes exist.

Value Resolution

If modes are defined, all tokens must have values for all modes, taking into account fallback and default rules. This means that either $value or $modes (or both) must be defined for all tokens.

The value of a token for mode "m" is as follows:

  1. If the token defines a value for "m" , that value is used for the mode.
  2. Otherwise, if the mode defines a $fallback, the token’s value for the fallback mode is used. The same rules are applied for the fallback mode, so if an explicit value is not defined for the fallback mode, its fallback is used, and so on.
  3. Otherwise, if $value is defined, then that value is used for the mode.
  4. Otherwise, the token is undefined for the mode, which is an error.
@NateBaldwinDesign
Copy link

I like this approach. I am curious if mode nesting in token definitions would be supported or explicitly disabled?

For example, you have light and dark mode, but for each of those you also have increased and decreased contrast modes. Modes in this case could not be defined in isolation as there is some dependency (light + increased contrast, dark + increased contrast).

Example contrast modes:

{
  "$name": "Figma UI Colors",
  "$modes": {
    "light": {},
    "dark": {},
    "increased-contrast": {}
    "decreased-contrast": {}
  },
  // tokens ...
}

Examples with nesting:

  "text-primary": {
    "$type": "color",
    "brand": {
      "$modes": {
        "light": {
          "value":  "{colors.gray.800}",
          "increase-contrast": "{colors.gray.900}",
"decreased-contrast": "{colors.gray.700}"
        },
        "dark": {
          "value":  "{colors.gray.200}",
          "increase-contrast": "{colors.gray.100}",
"decreased-contrast": "{colors.gray.300}"
        }
      }
    }
  }

Alternatively, disallowing nesting for modes may provide a forcing function for aliasing (one level handles light/dark, another level handles contrast). Either I think are ok but should be considered.

Similarly, it may be necessary to define which modes are relative to one another, or what token types they can support. Ie, "light", "dark", and "darkest" are enumerations for a property (eg, "colorScheme"), whereas "increase-contrast", "decreased-contrast" or even "forced-colors" would be enumerations for a different property (eg. "contrastModes"). That way we can enforce that you cannot nest/combine options of the same mode property (eg, "light": {"dark": "$token"}} is disallowed).

@nesquarx
Copy link

nesquarx commented Mar 23, 2023 via email

@romainmenke
Copy link
Contributor

romainmenke commented Mar 23, 2023

This proposal is missing a critical bit.
Since modes are user provided content translation tools can not generate code based on modes.

All possible modes must be defined by the specification for translation tools to work.


see : #169

@connorjsmith
Copy link

Today for theming we just add a new theme file with the tokens defined in the theme file overriding said tokens in the base file. This allows product teams to create custom themes without messing with the core token file. With the modes proposal, I do not see a mechanism for defining a mode outside the main token definitions, or would it work the same way? Define the same tokens a second time with just the additional mode value?

Depends on the heuristic we'd want to use for what constitutes an "extension" of the base theme (either via some sort of $extends, or just via name matching), but I'd think that defining more modes would be something like this

File 1

{
  "$name": "Figma UI Colors",
  "$modes": {
    "light": {},
    "dark": {},
    "increased-contrast": {}
    "decreased-contrast": {}
  },
  // tokens ...
}

File 2

{
  "$name": "Figma UI Colors Extended",
  "$extends": "Figma UI Colors", // Alternatively tools just batch accept files and try to combine them blindly
  "$modes": {
    "extra-dark-mode": {"$fallback": "dark-mode"}
  },
  // tokens (can overwrite values for light and dark mode, as well as define new values for the added extra-dark-mode)
}

@connorjsmith
Copy link

This proposal is missing a critical bit. Since modes are user provided content translation tools can not generate code based on modes.

All possible modes must be defined by the specification for translation tools to work.

see : #169

Could you expand on this? This proposal defines all modes upfront inside $modes, or am I misunderstanding the problem?

A minimal example would be super helpful, thanks in advance!

@romainmenke
Copy link
Contributor

romainmenke commented Mar 23, 2023

A minimal example would be super helpful, thanks in advance!

The json in the first post is an excellent example.

How would a translation tool process that file?
How do end users "access" the mode specific values?

Taking CSS as an example. What would the generated CSS code be for that file?

@romainmenke
Copy link
Contributor

Also see : #204

@connorjsmith
Copy link

connorjsmith commented Mar 23, 2023

I would expect the following generated CSS

/* figma-ui-colors_light.css */
:root {
  --bg-brand: #1010FF; /* colors.blue.300 */
  --fg-brand: #000000; /* colors.black */
}

/* figma-ui-colors_dark.css */
:root {
  --bg-brand: #1010FF; /* colors.blue.500 */
  --fg-brand: #FFFFFF; /* colors.white */
}

/* figma-ui-colors_super-dark.css */
:root {
  --bg-brand: #0000FF; /* colors.blue.700 */
  --fg-brand: #A0A0A0; /* colors.gray */
}

Usage:

@import url("figma-ui-colors_light.css") prefers-color-scheme: light
@import url("figma-ui-colors_dark.css") prefers-color-scheme: dark

Those variables would be split up over 3 files, one for each mode. Alternatively a generator could specify all modes in a single file

:root {
  --bg-brand: #1010FF; /* colors.blue.300, from $value */
}

:root[mode="light"] {
  --bg-brand: #1010FF; /* colors.blue.300 */
  --fg-brand: #000000; /* colors.black */
}

:root[mode="dark"] {
  --bg-brand: #1010FF; /* colors.blue.500 */
  --fg-brand: #FFFFFF; /* colors.white */
}

:root[mode="super-dark"] {
  --bg-brand: #0000FF; /* colors.blue.700 */
  --fg-brand: #A0A0A0; /* colors.gray */
}

Of course tools could probably optimize the above to utilize css var override cascades

Another alternative would be for tools to append the mode to token names, should a "multi mode" or override use case be required

  --bg-brand-light: #1010FF; /* colors.blue.300 */
  --fg-brand-light: #000000; /* colors.black */

  --bg-brand-dark: #1010FF; /* colors.blue.500 */
  --fg-brand-dark: #FFFFFF; /* colors.white */

  --bg-brand-super-dark: #0000FF; /* colors.blue.700 */
  --fg-brand-super-dark: #A0A0A0; /* colors.gray */

Are there other gaps that a generator tool wouldn't be able to create based on the proposed JSON structure?

@connorjsmith
Copy link

connorjsmith commented Mar 23, 2023

Looking at the postcss plugin, I'd expect this to be possible with an added mode() extension. The mapping could be done in postcss config as well, similar to the is option

Using new mode() option

@design-tokens url('./figma-ui-colors.json') format('style-dictionary3') mode('light');
@design-tokens url('./figma-ui-colors.json') when('brand-2') format('style-dictionary3') mode('dark');

.foo {
  color: design-token('fg.brand');
}

Using postcss config

postcssDesignTokens({ 
  modes: { 
    'figma-ui-colors.json': 'dark'
  } 
})

@romainmenke
Copy link
Contributor

romainmenke commented Mar 23, 2023

I think there is some confusion here :)
Is this a proposal for "theming" or for "conditional/contextual values"?


Theming :

  • one design system
  • multiple sets of values
  • multiple outputs
  • static result

Examples :

  • a customizable design system that is used by 3rd parties
  • having variants of a design in multiple color hues (a blue theme, a red theme, ...)
  • ...

Conditional values :

  • one design system
  • multiple sets of values
  • one output
  • dynamic result
  • native API's

Examples :

  • dark mode
  • light mode
  • high contrast mode
  • large screens
  • small screens
  • ...

It might sometimes be possible to build a dynamic result by combining multiple themes on a single page or screen but these are always custom mechanics that don't leverage native API's.

Given the mentions of dark mode and high contrast mode I assumed this was a proposal for conditional values.

Can you clarify?


If this is a proposal for theming and isn't intended for dark mode / light mode then we would just wire this up behind is in the PostCSS plugin.

For a tokens file with modes a and b defined within a single file :

@design-tokens url('./figma-ui-colors.json') format('style-dictionary3');

.foo {
  color: design-token('fg.brand');
}

output A :

postcssDesignTokens({ 
  is: ['a']
})

output B :

postcssDesignTokens({ 
  is: ['b']
})

But that is only interesting for us to do when this feature isn't intended for dark / light mode and other conditional values.


This proposal doesn't make it possible for us to support this :

CSS author writes :

.foo {
  color: token('foo');
}

We generate :

.foo {
  color: red;
}

@media (prefers-color-scheme: dark) {
  .foo {
    color: pink;
  }
}

We can not generate this output because there isn't any link between a mode with user defined name "dark" and the concept of a functional dark mode.

@TravisSpomer
Copy link

That kind of link can, however, be supplied to the code generator as configuration. Every code generator tool is going to require some amount of configuration to make the output fit well with your codebase; a per-output-platform mapping of mode names to "the native thing" could be part of that configuration.

{
  "light": "",
  "dark": "@media (prefers-color-scheme: dark)"
}

@TravisSpomer
Copy link

Put another way, even if this spec defined a full set of what the allowed themes/modes were, and they were light and dark, a CSS code generator would still need to know how to map those themes to code. One project wants @media (prefers-color-scheme: dark), another makes dark mode the default and uses @media (prefers-color-scheme: light), and another uses [theme-dark] and switches attributes with JavaScript.

@romainmenke
Copy link
Contributor

[theme-dark] That is not a native API for dark mode and shouldn't be taken into consideration. Anyone can do any custom thing.

One project wants @media (prefers-color-scheme: dark), another makes dark mode the default and uses @media (prefers-color-scheme: light)

That would just work.
Your base set of values wouldn't be wrapped in any conditional at rules.
Your conditional set of values would be wrapped in whatever conditional matches.

If a designers decides to do dark as default then the code generator would produce :

.foo {
  color: pink;
}

@media (prefers-color-scheme: light) {
  .foo {
    color: red;
  }
}

But I don't want to focus to much on conditional values and the benefits of being to generate those without first confirming if the original proposal was for theming or conditional values.

@connorjsmith
Copy link

Sorry for the confusion! Yes this is purely about theming/modes, not with determining how/when those modes should be applied.

@jjcm
Copy link
Author

jjcm commented Mar 23, 2023

Since modes are user provided content translation tools can not generate code based on modes.

All possible modes must be defined by the specification for translation tools to work.

I heavily disagree with this. Any list that we come up with of valid modes will never satisfy the demands of end users. Let's say for example we have these modes as valid ones:

  • Light
  • Dark

But now a user wants to add midnight mode (i.e. similar to dark, but all near-blacks are set to black to preserve battery life on OLED mobile devices, a fairly common pattern). Are we saying that midnight modes are an invalid use case for tokens?

Maybe we add it, so now our list is:

  • Light
  • Dark
  • Midnight

But now a user wants to add high contrast. Another user wants to add colorblind mode. Lets say somehow we hypothetically come up with a list of modes that encapsulate all possible visual modes (which personally I don't believe is possible, but lets say hypothetically we did) and we ended up with something like,

  • Light
  • Dark
  • Midnight
  • Deuteranopia
  • Protanopia
  • Tritanopia
  • High Contrast
  • Low Contrast
  • ...etc

But a large organization wants different themes per product brand, one of which has a light appearance with a red brand color, and another of which has a dark appearance with a green brand color. Are we saying that's an invalid use of tokens?

Modes need to be user defined, not an enum.

The output of these modes by translation tools should attempt their best-guess, but there are hundreds of ways people represent theming on the web today - there's no standard for it (aside from prefers-color-scheme, which we already know doesn't satisfy the needs of users).

@jjcm
Copy link
Author

jjcm commented Mar 23, 2023

That kind of link can, however, be supplied to the code generator as configuration. Every code generator tool is going to require some amount of configuration to make the output fit well with your codebase; a per-output-platform mapping of mode names to "the native thing" could be part of that configuration.

{
  "light": "",
  "dark": "@media (prefers-color-scheme: dark)"
}

I like this, though this also feels like something that might belong more in $extensions rather than the core format, though I'm curious what others think here. IMO the core format should be apathetic to the output format.

@romainmenke
Copy link
Contributor

@jjcm #210 (comment)

Please see my comment here : #210 (comment)


Why not have both generic themes and conditional/contextual values?

The core concept of this design tokens specification is to be a bridge between different formats. Part of that work imo is mapping concepts like native API's for dark mode.


IMO the core format should be apathetic to the output format.

I disagree with this.

The format should try to be platform agnostic but it must be sympathetic to how it will be used.

If a small change to the format can make a large difference on the output for all platforms then that seems like a worthwhile thing to me.

@mkeftz
Copy link

mkeftz commented Mar 23, 2023

I really like this proposal and think building theming into the format will improve interoperability.

At Interplay, we use a very similar data structure internaly and get users to import/export specific themes to the token group format. The only difference is we use $values rather than $modes, and $inherit rather than $fallback

One suggestions I have is making $fallback an array instead of a single value.

Then users could define themes/modes for compact, jumbo, dark and light. Then they can combine these to create e.g. dark-mobile, light-mobile, etc. The value of the tokens would be determined by the order of the inherited themes.

This would save a lot of duplicate entry for values across themes.

{
  "$name": "UI Colors",
  "$modes": {
    "light": { }, 
    "dark": { },
    "compact": { },
    "jumbo": { }, 
    "dark-compact": { "$fallback": ["dark", "compact"] }
  },
  "font-size": {
    "$type": "dimension",
    "small": {
      "$value": "1rem",
      "$modes": {
        "compact": "0.5rem",
        "jumbo": "1.5rem",
      }
    }
  },
  "backgrounds": {
    "$type": "color",
    "base": {
      "$value": "{colors.white}",
      "$modes": {
        "light": "{colors.white}",
        "dark": "{colors.black}",
      }
    }
  }
}

The values for these tokens in the dark-compact theme would be
font-size.small: 0.5rem
backgrounds.base: {colors.black}

@jjcm
Copy link
Author

jjcm commented Mar 23, 2023

@mkeftz interesting proposal with the array, totally see the use case.

One thing I'm curious about though is if others expect to define their dimension modes along with their color modes. Having compact be a valid theme for UI Colors feels somewhat weird to me. I'd expect each of these to be in separate files given the lack of overlap in most situations. Would be curious to see what others think.

@connorjsmith
Copy link

Perhaps mobile/desktop and compact/large would be a more common combo? E.g. certain buttons would likely want their mobile sizings (for us fat fingered individuals), but non-interactive densities would want to stay compact (with the occasional explicit mobile-compact value where those two would overlap).

Mainly spitballing though, I'd also be curious to see how other people would plan on using that!

@gossi
Copy link

gossi commented Mar 28, 2023

This topic requires a lot of research, or basically the one I did for the past years and started to put into blog posts. I'm currently nearing ~10k words (so there is a substantial backup for what's to come). I hope to start releasing this series in late april.
With the spoiler out of the way that there is more to come, I think this topic is very important and I can share parts I have ready already.

Features / User Preferences

Let's start this from user preferences, this is what a user might wanna choose:

user-preferences

And the idea is, whatever the user is about to choose, will receive the respective value for that token. As theme authors we would call them features. A theme supports feature color-contrast or color-scheme, etc.
Not all of these can be "System", as these may be product related options. See next section for more differentiation.

By the way, here is github:

Bildschirmfoto 2023-03-28 um 18 15 21

They don't have a "skin", but support all the other features from the user preferences menu above (in terms of color). You can even have dark appearance set when your system is light o_O.

Behavior

There is a behavior involved in here:

  • Adaptive: User chooses "System" and passes the choosing on to the Operating System
  • Mode: User explicitely chooses a particular preference

In CSS:

:root {
  --background: white;
}

/* Adaptaive */
@media (prefers-colors-scheme: dark) {
  :root {
    --background: black;
  }
}

/* Mode */
[theme-color-scheme="dark"] {
  :root {
    --background: black;
  }
}

The important thing: The behavior goes independent from the storage of values in the tokens file! This goes into the next step when values are extract from the tokens file and put into CSS or whatever else format

References as Solutions in Development

In development we use references as solutions to this problem, here is one for colors:

References as Solutions

With that configuration above that is:

intent-action-base-background = 4 x 4 x 3 x 3 = 144

so the token can take 144 permutations in this example - if there was a value for all of them (which in reality I wouldn't expect).

A Format to Structure Permutations

Modes is the wrong word to this - as this is something a user would opt into (see Raskin, J. (2008). The Humane Interface: New Directions for Designing Interactive Systems (10th printing). Crawfordsville, Indiana: Addison-Wesley.) For example nowadays it would be dark color-scheme and not dark mode (that goes back to the old days, when it really was a mode - it still is but has moved to OS level).

I'm also not sure yet, what would be the best format to support this. It however needs to be defined on the token itself. It needs to be stored alongside the feature configuration. For example:

{
  "$name": "intent-action-base-background",
  "$value": [
    {
      "$value": "white"
    },
    {
      "$value": "darkgrey",
      "color-scheme": "dark"
    },
    {
      "$value": "black",
      "color-scheme": "dark",
      "color-contrast": "high"
    }
  ]
}

The finding of tokens would be programmatical. Those with the highest match win:

const token = tokens.find(t => t.name === 'intent-action-base-background');
const value = token.value.find({ 
  colorScheme: 'dark', 
  colorContrast: 'high' 
});

That is in fact no different than how tools currently work, as danny wrote in https://dbanks.design/blog/dark-mode-with-style-dictionary (Single Token Method)

An alternative to stick with the object approach would be (DON'T DO THIS):

{
  "$name": "intent-action-base-background",
  "$value": {
    "": "white",
    "dark-": "darkgrey",
    "dark-high": "black"
  }
}

where features become the key within $value (This is how one needs to do that in figma right now, because styles don't support multiple values... hint hint :D). However, I'm heavily against this, as it would eliminate structural information and requires parsing.

In terms of scalability a format is required that can potentially scale up to ~150 permutations per token but we also know there will be maybe permutations based on 2 features with 2 permutations - as companies will already started to support color scheme and soon color contrast is to come (maybe because it only has two options? or because it has media query backup?). But once this wave is over, we probably will see adoption for chroma - Yes I'm very hypothetical here.

On the other hand the format shall be practical, this is why all (?) ideas so far use an object. This could easily end up in a nightmare nested tree as @NateBaldwinDesign showed (given the github use-case above) and practicability is gone.

I'm still with the array and it's configuration, but happy to read about better ideas.


Also, if I rushed over some of the topics, then please ask for more detailed explanation.

@jjcm
Copy link
Author

jjcm commented Mar 29, 2023

so the token can take 144 permutations in this example - if there was a value for all of them (which in reality I wouldn't expect).

Definitely agree! That's one of the reasons why we are pushing for fallback definitions for each mode, as well as overrides being optional. You shouldn't have to define all permutations.

Modes is the wrong word to this

Modes is the wrong word if you're only supporting color. What complicates things is that design tokens are used for many use cases, color just being one of them. We originally were using theme and color-scheme, but we found it didn't apply for other use cases of tokens. Here are the use cases we came up with that we felt the terminology should support:

  • visual themes (i.e. light/dark)
  • product/brand themes (i.e. docs/sheets/slides)
  • accessible themes (high contrast, colorblind modes)
  • dimensions and platform size differences (mobile/desktop/tablet)
  • UI string translations / alternatives (i.e. english/spanish/german)
  • and likely more in the future! What happens with depth is a factor for VR/AR?

In the end we found that color-scheme simply didn't work outside of color (i.e. a mobile color scheme doesn't convey that you're talking about differences in padding), anything that's specific like that also encounters issues of not being scalable for future use cases.
theme was great for color as well, but it didn't work well for things such as translations (i.e. a "German theme" feels like you're going to have a bavarian/oktoberfest vibe, not having strings translated to German). We ran a survey among a few different disciplines and users, and found that mode was the least-bad option for a generic term. It was the 2nd choice among the 3 cohorts we tested: visual designers, content designers, and those working explicitly on dimensionality - not their first choice, but one that still made sense for their use case.


The permutation structure you suggest is interesting - treating each permutation as a flag rather than an individual grouping. I'm not a huge fan of the readability of it (though I'm not sure if that matters for an interop format), but I see the value. Would be curious on the needs of this vs explicit modes. One thing to call out is with this approach you'd need some very explicit logic around missed finds, as with this approach it's possible for no correct value to exist.

As an example, if you had:

{
  "$name": "intent-action-base-background",
  "$value": [
    {
      "$value": "white"
    },
    {
      "$value": "darkgrey",
      "color-scheme": "dark"
    },
    {
      "$value": "black",
      "color-scheme": "dark",
      "color-contrast": "high"
    }
  ]
}

And you queried for:

const token = tokens.find(t => t.name === 'intent-action-base-background');
const value = token.value.find({ 
  colorContrast: 'high' 
});

Would you return black or white in this case? There's no explicit definition for color-contrast: high without dark mode defined. Do we then return the dark mode value? Or do we return the light value?

If invalid queries like this aren't allowed, validations would be non-trivial as they'd be exponential in nature to test. A flag based approach is definitely going to need more thought.

@c1rrus
Copy link
Member

c1rrus commented Mar 30, 2023

Thanks for sharing this thorough proposal @jjcm, and thanks everyone else for all the insightful comments here!

Overall, I really like this proposal. I think it would provide the format with an elegant mechanism for providing alternate token values for any number of use-cases. As Jake pointed out in the previous comment, this could have applications way beyond light and dark color schemes.

Of course, since the modes are author-defined (i.e. there wouldn't be a pre-defined set of permitted mode names in the spec), tools can't automatically infer any semantics from the mode names or apply specialised functionality based on the selected mode.

If I understood correctly, I think this is essentially the issue @romainmenke raised in his comments. For example, a translation tool could not "know“ that a particular mode represents the colors that should be used when the user has set their OS to dark mode and therefore could not automatically output an appropriate @media query in CSS.

However, I think the $modes proposal nonetheless enables a lot of use-cases that are so far impossible to represent using the current DTCG format draft. I'd therefore be in favour of adding it to the spec.

I do think some of the finer details need ironing out first though...

Does $value really need to be optional?

The OP proposal relaxes the requirement for every token to have a $value since could instead just have $modes. While this certainly could be done, I wonder whether it might keep things a bit more consistent and simple if we keep $value being mandatory.

The way I see it, every DTCG file has an implied "default" mode, which is what you get if you didn't use $modes at all. So why not lean into that and encourage authors to only use $modes for additional modes they might need.

Taking the initial example from the OP, we might decide that "light" is our default and therefore only use $modes to create the additional "dark" and "super-dark" modes. The example could then be rewritten as:

{
  "$modes": {
    // no longer specify light
    "dark":       {}, // no fallback
    "super-dark": { "$fallback": "dark" }
  },
  "bg": {
    "$type": "color",
    "brand": {
      "$value": "{colors.blue.300}", // light mode falls back to this
      "$modes": {
        "dark": "{colors.blue.500}" // super-dark mode falls back to this
      }
    }
  },
  "fg": {
    "$type": "color",
    "brand": {
      "$value": "{colors.black}", // use $value rather than $modes.light here
      "$modes": {
        "dark": "{colors.white}",
        "super-dark": "{colors.gray}"
      }
    }
  }
}

I think doing it this way and retaining the "every token must have a $value" rule is preferable for the following reasons:

  • Avoids situations where an author omits $value but forgets to specify $modes values for every mode. In that case there'd be no $value to fall back on, so the token becomes... invalid?
  • Could make the learning curve for the format lower since there's less exceptions to learn. I think "every token must have a $value" is simpler than "every token must have a $value, unless it has a $modes property". Especially if you consider $modes an advanced feature that newcomers might not learn about right away.
  • Keeps parser logic a (tiny bit) simpler too for the same reason as above.

I think the only thing we might lose would be the ability to assign a name the default mode. In the above example, users would need to know that "no mode means light mode". However, we could solve that by having a means to provide optional metadata about the default mode. For example, we could have an optional $defaultMode property that could be added to the top-level $modes declaration like so:

{
  "$modes": {
    "$defaultMode": {
      "$name": "light"
    },
    "dark":       {}, // no fallback
    "super-dark": { "$fallback": "dark" }
  },
 // ...
}

Could modes have additional metadata?

For example $description? I could imagine this might be useful for tools that have the ability to list out all declared modes to the user.

{
  "$modes": {
    "$defaultMode": {
      "$name": "light",
      "$description": "A color scheme to be used when the user has expressed a preference for a light UI"
    },
    "dark": {
      "$description": "A color scheme to be used when the user has expressed a preference for a dark UI"
    },
    "super-dark": {
      "$description": "A even darker version of the dark color scheme which uses pure black backgrounds to preserve energy on devices with OLED displays",
      "$fallback": "dark" 
    }
  },
 // ...
}

It's conceivable that future spec iterations might add more properties for other bits of useful metadata for tokens and groups. When doing so, we could also consider whether they might be useful for mode declarations and, if so, permit them to be used here.

Should modes have $extensions?

Sort of related to the previous point, maybe permitting $extensions on mode declarations could be useful too. The use-case would be extensions that relate to a mode itself rather than a token or group.

This kind of thing could perhaps enable tools to add some additional semantics per mode, as @TravisSpomer was suggesting in response to @romainmenke.

E.g.:

{
  "$modes": {
    "$defaultMode": {
      "$name": "light",
      "$extensions": {
        "org.postcss.design-tokens.media-query": "(prefers-color-scheme: light)" 
      }
    },
    "dark": {
      "$extensions": {
        "org.postcss.design-tokens.media-query": "(prefers-color-scheme: dark)" 
      }
    },
    "super-dark": {
      "$extensions": {
        "org.postcss.design-tokens.media-query": "(prefers-color-scheme: dark) and (prefers-contrast: more)" 
      },
      "$fallback": "dark" 
    }
  },
 // ...
}

Obviously, the standard $extensions caveat of being prorietary and thus not universally supported by tools applies. But this could still provide an "escape hatch" to enable tool-specific functionality. Also, if particular extensions gain a lot of adoption, it provides us with a cowpath to pave in future spec iterations.

@gossi
Copy link

gossi commented Mar 30, 2023

Oh, so wonderful thoughts in your answer :)

PS. While typing my answer @c1rrus also posted (I will read his post after I posted mine). This is a response to @jjcm

I think, permutation (as odd as it sounds) is the word I used most for to describe those varying token values. It is not as much as opinionated as mode - wdyt? Perhaps not necessary, as values might be best as an array?! 😅


I thought about validation and fallback a bit more after posting here. Here is a bit more of what's in my head.

Do we want to have a fallback for the sake of having a fallback, because we want to have it or because it must be there? I think we are going with fallback, because we want to have it to avoid dealing with the complexity that awaits us. And is fallback even a correct value then?

About features, these are defined by your product/theme/design system - they may or may not live within the token file. The bonus of having it is to that you can validate the file by itself, which I'd actually in favor of having - or this can be set as reference:

{
  "$reference": "path/to/features.tokens"
}

I think, having validation is important. I'm author of theemo and am currently working on v2 which actually is about letting designers define their features and then provide a UI for letting them define values for a token based on features - sounds familiar?

Here is the challenge: Let's say you defined two features, color-scheme (light/dark) and color-contrast (low/high) and you have a color token with a given value. Now you want this token to support color-scheme. Your UI splits into two input fields one for light one for dark and copies over the value from before into the two new input fields. Let's say the value from before was set for light color-scheme, the designer now will choose a value for dark color-scheme.
The token stays valid as long as there are two values present. The original value before this token support color-scheme feature is no longer relevant (=> no fallback needed, the fallback became the light color-scheme value). The UI can validate if the token has support for all values based on the features the token shall support.

Next up: Let's make this token support color-contrast feature. We will be presented with a 2 x 2 input matrix. Same drill again for the designer. However, if low contrast was considered the default, then the designer would only change the values for high color-contrast.
The values for low color-contrast can even be omitted. This is the "none" case in the picture above. The fallback here are the two color-scheme values without color-contrast.

That is a token is valid as long as permutations of a supported feature are matched with permutations of other supported features - hell, this is a heck to explain in words. I'm having truth tables in my mind here, where you can cluster groups.

What's important to note here is, that features will have a default, which would ease building UIs and help validate (from my research before, I never find a case where there is no default, this will always ever be provided by someone - the OS at last). Let's say our default for color-scheme is light and revisit the process from above. The designer would choose to support color-scheme for a token, the previous value is copied into light color-scheme (the default) and leaves the designer to fill out only the dark value. Same for color-contrast set to low as default, when supporting color-contrast on the token, the UI would only show the option to the designer to fill out values for high color-contrast.

Which brings us back once more to fallbacks: The fallbacks are the default values of a feature.

That is a fallback for the entire value set is wrong, think this: If you have a fallback value (that was set for light color-scheme and low color-contrast) and as a user you want to experience a product in dark color-scheme and high color-contrast. If there is a value given for dark color-scheme but low color-contrast and the fallback value - which one to serve to users?

Which brings us to your question:

Would you return black or white in this case? There's no explicit definition for color-contrast: high without dark mode defined. Do we then return the dark mode value? Or do we return the light value?

I think there are two options here to answer this:

  1. That's an invalid token, as the request you are having cannot be answered - because explicitely matching your query does not work
  2. Assume the default for colorScheme and combine it with the query for colorContrast to return the appropriate value

When authoring (build time) we are in need of defining a default (this is truly a mode * here). Whereas in experiencing a product (runtime) as a user, a preference is given (by the the default from the product, by the platform (browser) or latest by the OS). With that preference present the correct token value can be found.

* maybe this was the case for calling it mode?


I have been playing around with typing this to work on theemo - as typing is a good way to prototype this and figure out problems. In the next branch (I gave it a push right now), there are types in @theemo/core - I've been playing with only the features the web offers through media queries (color-scheme and color-contrast), but potentially could be more.

A theme in theemo would be to it's current state defined as:

{
  "name": "super theemo",
  "features": {
    "color-scheme": {
      "behavior": "adaptive",
      "default": "light"
    },
    "color-contrast": {
      "behavior": "mode",
      "default": "low"
    }
  } 
}

behavior is for the implementation on the product (ignore, see post above) but default will be used for constructing the UI for designers to pick a proper value as well as the algorithm to find the correct token value.

@jeromefarnum
Copy link

jeromefarnum commented Apr 1, 2023

Could modes have additional metadata?

For example $description? I could imagine this might be useful for tools that have the ability to list out all declared modes to the user.

Should modes have $extensions?

Sort of related to the previous point, maybe permitting $extensions on mode declarations could be useful too. The use-case would be extensions that relate to a mode itself rather than a token or group.

This kind of thing could perhaps enable tools to add some additional semantics per mode, as @TravisSpomer was suggesting in response to @romainmenke.

Both of these suggestions seem like sensible considerations to add, as both fit existing patterns within the spec.

The use of $extensions to handle generator considerations seems like a particularly appropriate approach, as how tokens are output may have different solutions even within a particular platform or framework. Consider how one might handle modes and themes in CSS, there are several strategies one could employ. Just off the top of my head, one could:

  • Generate a single CSS file containing all permutations; each permutation wrapped within an appropriate media query
  • Generate a single CSS file containing all permutations; each wrapped within a rule and use some matchMedia logic to toggle rules as needed
  • Generate CSS files for each permutation and use the media attribute of the link or style elements to responsively load files
  • Some combination of all of the above

Moreso, one might desire to optimize the token output any number of ways, perhaps choosing to output a set of default/base tokens that are stable across all permutations paired with streamlined sets per permutation that only contain the unique tokes for that permutation OR simply render all tokens per permutation. This choice may even be different per targeted platform, technology, or framework.

It seems prudent to me, that the DTCG spec should aim to provide the means for a designer/team to model the relationship of the tokens to modes, but leave the business of how tokens are translated to generator tools. Leveraging the $extensions field to provide instructions for generator tools accomplishes that very well.

@romainmenke
Copy link
Contributor

romainmenke commented Apr 1, 2023

It seems prudent to me, that the DTCG spec should aim to provide the means for a designer/team to model the relationship of the tokens to modes, but leave the business of how tokens are translated to generator tools. Leveraging the $extensions field to provide instructions for generator tools accomplishes that very well.

As a translation tool implementer I can safely and surely say that we will never go beyond the specification. If there is an expectation of some behavior in translation tools, it needs to be specified. We don't see the point of building tools for a specification and then having to invent critical implementation details.

I think it is dangerous to consider $extensions as an escape hatch that can be used to fill in holes in this specification. We have zero interest in keeping track of what other tools are dumping in $extensions and building logic around that.

$extensions is only realistically useful to store data to be used by a single tool or multiple tools by a single vendor. It isn't suitable as a basis for complex features that must work in all tools.

I am still hoping that this format looks at the whole picture, design tools and translation tools.

@kevinmpowell
Copy link
Contributor

Linking to #187 to group all theming related discussions together.

@ddamato
Copy link

ddamato commented Apr 9, 2023

Hi folks, long time no visit. @kevinmpowell pointed me in the direction of this thread to weigh in.

Note

tl;dr, I recommend different files for different contexts (themes) and I have no idea how the group could create a schema for token aliasing because humans have opinions on what's best for them.

One of the things I'm concerned with here depth that this is introducing, and not a good depth in my opinion. Both in terms of variation and in terms of diving into an object for values. The former is difficult for humans, the latter is challenging to code. Specifically about coding to a complex object spec; diving into trees several levels deep with variations of keys looking for values sounds like an engineering interview question I have in my nightmares.

What I'd like to propose is aligned with the way I believe humans think of token assignment. We aren't thinking about light and dark mode simultaneously, we think of them one at a time. Therefore, the first step to my recommendation is to first focus on solving the theme layer; which I believe should be a nearly flat structure of semantic token to value assignments. No variation of light and dark, because each file is meant to relate to a single context. Here's a very minimal example of what could be considered a "light" theme based on the spec today:

{
  "ux-surface-backgroundColor": {
    "$type": "color",
    "$value": "white"
  }
}

As a human thinking about how I want to assign color semantically, I scope myself to a single context (ie., mode/business-need/experiment) at a time. If I'm working on a light theme, I'm not trying to find all of the light theme values in a single file, I'm working in the file that is meant to convey the light theme. Anything else is just noise. Granted, I recognize that the expectation is to not work with token files directly but I believe there's an opportunity for simplicity here. Defining each theme file as a single context helps focus the curation exercise. Having everything available at once is triggering Hick's Law. Even with tools, this would be visually daunting. Speaking from experience here, if we were to attempt to put all tokens across all brands in the same file at GoDaddy (with numerous reseller brands), we're talking about a number around 24,000 token assignments in a single file or hidden within the depths of nested UI.

Furthermore, this supports the ability to have a scoped theme within a larger page. You can be sure that anything placed within that scope (ie., inverted hero) will be covered either by the page level theme, or the one expected to be applied in scope. There's more about that in this post, where I recommend that variations of "on dark" as token names are not scalable.

What I've mentioned above covers semantic token to value assignment; the tokens that will eventually apply directly to UI elements. What it doesn't cover is additional tokens that I recognize would be helpful for brand organization. In my view this layer is wildly unique among teams, brands, and organizations because it is often a reflection of the personalities of the people who maintain these layers.

I ask myself why people need additional layers all the time. In reality there's nothing stopping someone from just assigning blue as the value to a semantic token. Again, speaking as someone who has been making these kinds of value assignments for years, creating a color.blue.500 token just means I'm copying (or assigning through some tool) the color.blue.500 token over and over across them theme layer in the same place I would be copying blue. In reality color.blue.500 shouldn't change because we don't easily know what this means for the assignments at the theme layer. Seeing the change of color.blue.500 in a sample of UIs doesn't cover what it means for all UI as a semantic tier does.

The only reason I can think of that these other tiers could exist is to be able to speak about the token conversationally. It's clearly more helpful to say color.blue.500 over #1d9bf0, there's even some categorization built into the name. It feels helpful from a human perspective.

Above lies the challenge, as I imagine it'll be impossible to propose avoiding additional tiers. It'll be more of a challenge to define what these tiers look like and support all of the variations the humans may dream up (I can see the marketing team coming in there, wanting a color called make-it-pop-pink).

From there, it's a matter of importing this tree into the theme file:

import colors from './colors.json';

// OR, and probably more desirable for multibrand

import { colors } from './my-brand-styles.json';

// For the "light" theme
const tokens = {
  'ux-action-backgroundColor' : {
    '$type': 'color',
    '$value': colors.blue['500']
  }
}

export default tokens;

I'm admittedly torn having an opinion at the additional tier layers (aka token aliasing). On the one hand as a specification meant to cover tokens, it should have some well-defined schema where systems can share, read, and manipulate maintaining expectations. However, on the other hand, I find these additional tiers mostly useless personally. Which probably answers the question for the group about whether to include them or not. Clearly they should be included for the greater design community, I just can't imagine how we're going to cover everything people could want to do here in a schema without it being a dynamic dumping ground or tied up in ego.

I'll also admit that I lean into the semantic layer hard. I believe that UI designers are really theme authors; people interested in what the values of the UI are. And that UX designers (folks interested in the user experience) could submit wireframes which are wired semantically (this is a button, this is a card) to point to the semantic tokens to be informed by the theme author. This presupposes that UX designers don't have opinions about what color their design should be, and I know that's absolute crazy talk.

@Sidnioulz
Copy link

My main concern is also what @nesquarx mentioned, about scalability. If I'm adding a theme with only a small bit of overrides, after the fact.. it might be easier to have this in a separate file, so I can easily see what a theme overrides on the core.

@connorjsmith 's comment addresses my concern in that regard:

Depends on the heuristic we'd want to use for what constitutes an "extension" of the base theme (either via some sort of $extends, or just via name matching), but I'd think that defining more modes would be something like this

I also think that is very closely related to my proposal of an $extends property, some sort of simple metadata that defines a contract between token files or token groups even.

For us, having distinct files would be unmanageable. We define hundreds and hundreds of tokens, currently with 18 different modes applied. Getting the full picture of what a token's value can be would be difficult, and we find that once the token system is set up, we update tokens rather than updating modes. Having a single file where a token showcases all its values make that easier to manage.

I'd love to see both formats supported by tools, though I believe what tools like Style Dictionary currently do (multiple files with a value per token per file) is too inconvenient for long-term maintenance workflows.

@SorsOps
Copy link

SorsOps commented May 18, 2023

TL;DR

We dont think the $modes example is scalable. We're showing a counter-proposal that extracts the theme definitions outside of the actual token files using a concept we call "Resolvers". This concept allows multidimensionality and granular controls over the final output.

The resolver supports multiple dimensions for token resolution and can handle an arbitrary number of dimensions. It keeps the token specification simple and externalizes the logic for handling additional dimensions outside of the token files themselves.

The proposed resolver introduces the concept of modifiers, which act as "libraries" that are referenced by the source set and can be used to define different dimensions for token resolution. Modifiers can have different values, and their selection is dependent on the input. The resolver also supports the use of contexts to pass values as inputs for context-sensitive design.

Overall, the proposed resolver offers a more flexible and scalable approach to multidimensional token resolution in design systems. It simplifies the token specification by keeping theme definitions outside of it, improves performance, and enables context-sensitive design.

Take a look at our POC site which shows the tokens in action and provides a few examples that can be played with.

This post is a snapshot of our public proposal for resolvers in Tokens Studio available here

We will take all the feedback we receive here to evolve the spec according to requirements

Parameters

It's important to setup some parameters to tailor the discussion of a proposal around:

  1. As @lukasoppermann mentions, files should still be simple enough to be edited by hand. Tooling as it currently stands has not reached the point of maturity where the system is fully automated. Additionally for a new spec it should be simple enough for new users to dip their toes in and test the viability of the solution.

  2. Scalability. This proposal is currently addressing adding support for modes and themes which are two additional dimensions for token resolution. In a real world case for larger systems there might be significantly more dimensions, such as the brand, surface, etc, so whatever solution we provide should be able to handle an arbitrary amount of dimensions.

  3. Simplicity. Referring back to point 1, we don't want to complicate the existing spec for tokens. If we do we end up polluting the spec with responsibilities outside of its core intent of just providing a simple and extensible way to define tokens. Thus we should externalize the logic concerned with handling these additional dimensions outside of the actual token files themselves.

Using the above parameters, the proposal is that we create standalone resolver files which represent the different ways in which a token can be resolved.

Background

This approach is based off the existing work in the Tokens Studio Plugin which currently supports multidimensional tokens through the use of its $themes.json file that it exports with the following shape

[
  {
    //An arbitrary generated id as a hex string
    "id": "12323422b00f1594532b34551306745622567a",
    "name": "brand",
    "selectedTokenSets": {
      // A set that is used to resolve references
      "tokens/core": "source",
      // A set expected to be used in the output
      "tokens/semantic": "enabled"
    }
  },
]

Using this, they are able to support multiple dimensions by using token sets as sources, which means they are able to be referenced for aliases, but do not contribute to the final token values and enabled sets which are then flattened in order to get the final values of the resulting token set once all transformations have been applied.

An example of this in action would be the following where the brand X has different combinations of themes by selecting a combination of differents sets to form that multidimensional theme :

[
  {
    //An arbitrary generated id as a hex string
    "id": "2f440c32b00f1594532bf5b051306724e22136a",
    "name": "Brand X | Light theme",
    "selectedTokenSets": {
      "brand/x/foundation/color/light": "source",
      "foundation/color/appearance": "source",
      "foundation/dimensions": "source",
      "mode/light": "enabled",
      "semantic/actions": "enabled",
    }
  },
  {
    "id": "aa7e80632359bcfc4e09761f8d8f235d02eb41d7",
    "name": "Brand X | Dark theme",
    "selectedTokenSets": {
      "brand/x/foundation/color/dark": "source",
      "foundation/color/appearance": "source",
      "foundation/dimensions": "source",
      "mode/dark": "enabled",
      "semantic/actions": "enabled",
    }
  }
]

The token sets used are typically larger than what is shown and there are cases where there are 40+ sets

There are a number of problems to this approach though. This is currently being applied to create a single addressable space for tokens and is not granular. If you wanted to create a much smaller set that represented each component for example, you could not as the output of the set resolution is a single named token set that is then used to reference the tokens.

Button-Breakdown

The references to the tokens used in this styling are tied to a single addressable token space

Whilst you could add more token sets as enabled if you wanted a button component by adding comp/button under comp/body for example, unless you are following strict naming, your button set might clash with other named tokens in the final generated token set. It is very likely that you would want to define dimensionality on a per component basis if your design system were evolving as well, so that new designers who were onboarding were not overwhelmed by the existing complexity of the system and can focus on isolated components.

In this we find the next property for the resolver that we would need to specify which is specificity. Different components might want to resolve in an isolated manner
with different modifiers that might not be global

@jjcm mentions this as well. Modes are based on how the sets want to be consumed by a user, and are not a property of the set themselves, hence why it also makes sense to use standalone files outside of the token spec to define these and apply them independently.

{
      /*
      * Optional name of a resolver
      */
      "name": "Preset resolver",
      /*
      * Optional description
      */
      "description": "This handles the preset from Figma tokens",
      /*
      * A series of sets. The values of the tokens within these sets will
      correspond with the names of the outputted tokens.

      The order of these tokens is important. If keys for tokens are defined within them, the last token will be
      the effective value used
      */
     "sets": [{
          // An optional override of the name of the set. Can be used when tracing the resolution logic or when using `include` modifiers. Read further to see an include modifier in action
          "name": "first",
          // A reference to the tokens. This could vary depending on whether the resolution is occuring through the file system or in memory
          // In this example we assume through the file system through a relative file called core.json
          "values": ["foundation.json"]
        },
        {
            "values": ["semantic.json"]
        },
        {
            "values": ["button.json"]
        },
      ],
      /*
      * These modifiers act as "libraries" that are "imported" 
         and referencable by themselves and the source set
      */
      "modifiers": [
        {
          "name": "theme",
          //Default value of the modifier
          "default": "light",
            //Optional parameter to rename the set prior to resolution
          "alias":"theme",
          //Identifies the modifier type. In this case it is an enumerated value with named key value pairs
          "type":"enumerated",
          "values": [
            {
              "name": "light",
              "values": ["light.json"]
            },
            {
              "name": "dark",
              "values": ["dark.json"]
            }
          ]
        },
        {
          "name": "core",
          // Potential optional parameter to hide this modifier in software that visualizes the resolver
          "hidden": true,
          "default": "core",
          "type":"enumerated",
          "values": [
            {
              "name":"core",
              "values": ["core.json"]
            }
          ]
        }
      ]
    }

Note in the above example json specification since we do not allow arbitrary values, we have left out using the $ prefix on properties as is used in the token spec.

For visual thinkers the following is in effect

resolvers

For the type property there are multiple possible values

  • enumerated. This is a switch where the selection of the value is dependent on the input. If no value is provided, the default case is chosen. If no default is provided, no set is included.

  • include. This uses the same approach as the enumerated type however if an input matches the include, then it will replace the set with the same name as in the sets array. If no match occurs, then it is replaced with an empty set

Example
{
    //...
     "sets": [{
          //This will be populated by the modifier
          "name": "theme",
          "values": []
        },
      ],
      "modifiers": [
        {
          "name": "theme",
          "default": "light",
          "type":"include",
          "values": [
            {
              "name":"light",
              "values": ["themes/light.json"]
            },
             {
              "name":"dark",
              "values": ["themes/dark.json"]
            }
          ]
        }
      ]
    };

Modifiers can have the same name attribute set. This helps to support more complex cases than just a simple switch. Since names being shared results in the same input property being read, multiple effects can occur due to the same input.

Motivation

Why this approach over the current $themes approach?

  1. Having named enumerated possibilities indicates to documentation systems the different possible dimensions. Eg a button resolver could be identified as supporting 3 different brands, 2 modes and 4 themes for example. This is something that cannot be inferred currently. As @romainmenke mentions all possible permutations must able to be detected for code generation tools. By explicitly defining the possible values this should go aways to helping with this.

A point to mention alongside this is the detection of orthogonality surrounding the modifiers. Certain modifiers might override all others, eg disabled which might be a boolean. This might not potentially interact with any other modifier and as such is orthogonal to them and does not result in a value in the permutation matrix

  1. Lower cognitive load. As mentioned keeping all of this within a single global namespace with $themes means a designer needs to keep track of all the different tokens that are currently being used. If they could instead focus purely on logic related to a component, surface, etc by reducing the scope of tokens involved they can intuit the relations between tokens and sets easier.

  2. Faster resolution. For enterprise systems that have potentially thousands of tokens with multiple combinations, getting the final values becomes very slow assuming O(n) resolution time multiplied by O(m) for each dimension. By reducing the amount of tokens in scope, this should be significantly faster. Also since the relationships for which sets consume which other sets are within the resolver, a system could also know which resolvers to cache and which to re-evaluate

  3. Context sensitive design. For cases where context sensitive design is needed we could use contexts to pass through values as inputs to the resolver. An example would be surface logic. A button might change its values in response to the surface its placed on. In a Figma environment this could be implemented by storing a context object at different points of the document tree and then merging it together with a component input to react to whatever it is being placed into. Multiple different points of context could exist within the tree. For example the top level node might set the brand, while a frame within it might set the theme, and within it the surface. These values could be merged together in addition to say a buttons input to determine the final tokens that are in scope

Why is this preferred over embedding $modes?

As mentioned before, a single dimension is not sufficient. Even if we were to add $themes in addition to a $mode it would be a matter of time until a design system needed to support additional logic or logic that depends on other values.

Pros

  1. The spec for a resolver can still be kept fairly simple and should still be possible to be handwritten by a human.

  2. We can support an arbitrary amount of modifiers as well as keep the execution time quick even if permutating all possible values by keeping the amount of tokens in scope small by making resolvers that operate only on specific sets. @gossi mentions that larger design systems will likely end up with permutations around ~144.

  3. The token spec does not need to be extended as we are not embedding the resolution logic with the data.

  4. By removing fallbacks from the proposal we don't have to worry about edge cases like @jjcm mentioned with the following.

{
  "$name": "intent-action-base-background",
  "$value": [
    {
      "$value": "white"
    },
    {
      "$value": "darkgrey",
      "color-scheme": "dark"
    },
    {
      "$value": "black",
      "color-scheme": "dark",
      "color-contrast": "high"
    }
  ]
}
const token = tokens.find(t => t.name === 'intent-action-base-background');
const value = token.value.find({ 
  colorContrast: 'high' 
});

As this should be an error if token references are not resolved fully during the resolution.

Resolution logic

The resolution logic is fairly simple. Let's assume that a resolution request comes into a tool programmed in javascript

//Any acceptable resolver definition
const resolver = //...

const inputs :{
    theme:'dark'
    //Additional inputs
}

//async as sourcing tokens from arbitrary storage locations is likely not synchronous 
await resolve(resolver, inputs);

The resolve function would perform a number of steps

  1. Validate the inputs. Input types such as the enumerated type have specific named values that are acceptable. If these are not matched by the provided input, throw an error and fail.

  2. Read the input sets and flatten them into a single new set

  3. Select the appropriate modifiers and read the token files

  4. Perform optional aliasing on the modifier sets if required

  5. Replace input sets if using include modifiers

  6. Flatten the modifiers into a single new set

  7. Resolve the aliases of the input set using values from the modifier set if a value is not found in the input set.

The output of the resolution is a new token set. This token set can then be consumed by any export tools if need be as it is just a simple token set.

Example calculation

We want to resolve the provided resolver:

Resolver
{
  "sets": [
    {
        "values": ["foundation.json","semantic.json","button.json"]
    },
  ],
  "modifiers": [
    {
      "name": "theme",
      "default": "light",
      "alias":"theme",
      "type":"enumerated",
      "values": [
        {
          "name": "light",
          "values": ["light.json"]
        },
        {
          "name": "dark",
          "values": ["dark.json"]
        }
      ]
    }
  ]
}

with the input

{
    "theme":"light"
}

We first validate that there is a modifier called theme that has an acceptable input value of light, then load the necessary sets.

Let's assume the following tokens structures for the example

foundation.json

{
    "gray": {
        "$value": "coolgray",
        "$type": "color"
    },
    "padding": {
        "$value": "4px",
        "$type": "dimension"
    }
}

semantic.json

{
    "primary":{
        "$value": "{theme.accent}"
    }
}

button.json

{
    "padding": {
        "$value": "8px",
        "$type": "dimension"
    }
}

We first squash all the above tokens into a single set that will be resolved with the modifier

Name Set Value Overriden
gray foundation coolgray
padding foundation 4px
primary semantic {theme.accent}
padding button 8px

This results in :

{
    "gray": {
        "$value": "coolgray",
        "$type": "color"
    },
    "primary": {
        "$value": "{theme.accent}"
    },
    "padding": {
        "$value": "8px",
        "$type": "dimension"
    }
}

Now the modifiers are :

light.json

{
    "accent": {
        "$value": "lightblue", // for our light theme we want a lighter shade of blue
        "$type": "color"
    }
}

dark.json

{
    "accent": {
        "$value": "darkblue", // for our dark theme we want a darker shade of blue
        "$type": "color"
    }
}

Note in actual system that implement resolver logic we would not need to load both light and dark json files, only what is specified in the modifier. We show both here for illustrative purposes.

Assuming light was picked as the theme, we would first alias the light set using theme and flatten the values, resulting in :

{
    //These tokens are referenced by the input values and thus we will see them reflect in the output
    "theme":{
        "accent":{
            "$value": "lightblue", // {theme.accent} = lightblue in light.json
            "$type":"color"
        }
    }
    // If there are additional tokens defined here that are not referenced by input values they will not be part of the output
}

Now resolution would occur. We iterate through the tokens within our input set till we find any that require reference resolution and resolve them first using any values found in the input set, falling back to the modifier set as necessary. In this case primary contains a reference to theme.accent. The key theme.accent does not exist within the input set so we look at the modifier set, and find it. We then perform a replacement within the input set resulting in :

{
    "primary": {
        "$value": "lightblue",
        "$type": "color"
    },
    "gray": {
        "$value": "coolgray",
        "$type": "color"
    },
    "padding":{
        "$value": "8px",
        "$type": "dimension"
    }
}

Note if theme.accent had itself been a reference, we would recursively resolve the reference using the same logic.

Usage with export tools

For large design systems that might have multiple brands, the resolution of their tokens as they are finally consumed by the frontend code might be permuting multiple modifiers whilst holding other constant. Eg in an example webapp, the brand foo is known up front and will be the only such value, however the app supports multiple themes and modes.

Either multiple resolvers could be defined on a per brand level or the value of the brand in a single resolver could be held constant whilst evaluating the combinations of the other modifiers.

This would likely be an architectural descision depending on whether the brands have anything in common or not or otherwise require seperate resolvers for governance reasons.

@connorjsmith makes a valid point of the final tools such as style-dictionary deciding the final form of the tokens as a list of css variables, as well as optimizing the final form. Two resolved sets, with light and dark mode set respectively for example, could then be analyzed to optimize the final form of the tokens and removing redundant values .

Resolution aliasing

Consider the following set called size.json.

{
  "sm": {
    "$value": "1px",
    "$type": "dimension"
  },
  "lg": {
    "$value": "10px",
    "$type": "dimension"
  }
}

Let us assume that we want this file to be namespaced so that we can reference this in one of the sets to be resolved. Altering the file directly is not a good solution as it might introduce naming brittleness, as well as potentially have a number of other restrictions like being read-only, owned by someone else, etc.

This also speaks to an assumption we have never mentioned before. It is being assumed right now that everyone is in complete control of their sets, but if we want to support someone referencing another token set(s) eg Material, they should be able to just reference the values without having to modify them, similar to how we import modules in programming.

Rather we could dynamically namespace this when its loaded into the system through an alias like foo to result in

{
    "foo":{
        "sm": {
            "$value": "1px",
            "$type": "dimension"
        },
         "lg": {
            "$value": "10px",
            "$type": "dimension"
        }
    }
}

This allows us to consume other peoples token libraries without directly modifying their files.

Real world use case

The GitHub Primer token sets can be used as an example of applying the resolver to a large system.

The theme specifier here shows a case where we have two dimensions, the light and dark mode, in tandem with a visual impairment dimension that supports:

Light

  • General
  • Tritanopia
  • Standard Colorblindness
  • High contrast

Dark

  • General
  • Dimmed
  • Tritanopia
  • Colorblind
  • High contrast

Note the addition of dimmed. This will be relevant to show where architectural choices might affect how resolvers are used.

Starting with the light mode, we see that there are common sets purely used to reference that should not be used for the output.

These are the src/tokens/base/color/light/light.json5 set and the src/tokens/base/color/light/light.high-contrast.json5 which is used for light-high-contrast. Since the src/tokens/base/color/light/light.json5 is used for all values, this could be set as a include in the modifiers

Note In this case we are assuming that this resolver exists in the src/tokens folder

Resolver
{
  "sets":[{
    //The current solution expects to get actual values from the light and dark base sets as the output. As this is dynamic depending on the dimension. We include an empty set here which will be overriden by the modifier
    "name":"theme",
    "values": []
  }],
  "modifiers":[
    //This is a common set that is used for all resolutions, but is not expected in the output so we use it purely for context
    {
      "name":"base",
      "type":"enumerated",
      "default":"common",
      "values": [
        {
          "name":"common",
          "values": ["src/tokens/functional/color/scales.json5"]
        }
      ]
    },
    {
      "name":"theme",
      "type":"include",
      "default":"light",
      "values": [
        {
          "name":"light",
          "values": ["base/color/light/light.json5"]
        },
         {
          "name":"dark",
          "values": ["base/color/dark/dark.json5"]
        }
      ]
    },
    //An example of a repeated modifier. The above performs a different action to the the below defined modifier with this providing a context source to read from
    {

      "name":"theme",
      "type":"enumerated",
      "default":"light",
      "values": [
        {
          "name":"light",
         "values": ["src/tokens/functional/shadow/light.json5`,`src/tokens/functional/border/light.json5"]
        },
         {
          "name":"dark",
          "values": ["src/tokens/functional/shadow/dark.json5`,`src/tokens/functional/border/dark.json5"]
        }
      ]
    },
    //etc, we add visual modifiers for each of the common visual modifiers
  ]
}

Since the high contrast set is included, but only if it is using this modifier, we use a include modifier

Updated resolver
{
  "modifiers":[
    //...
    {

      "name":"visual",
      "type":"enumerated",
      "default":"general",
      "values": [
        {
          "name":"general",
          //...
        }
        //...
      ]
    },
+   {
+       "name":"visual",
+        "type":"include",
+        "default":"",
+        "values": [
+          {
+            "name":"high-contrast",
+            "values": ["base/color/light/light.json5"]
+          }
+      ]
+    }
 ]
}

Orthogonality

Within the possible dimensions we identify a concept of orthogonality to determine if one dimensions is independent of another. In laymen's terms, can a modifier change freely without needing to change how resolution of another modifier works. The following diagrams illustrate this point.

In this first diagram, two independent sets are used to resolve the final value. Any change to the Mode choice does not affect the theme set used.

flowchart TD
    Mode(Mode) --> Light
    Mode -->Dark
    Theme(Theme)-->Samurai
    Theme-->Knight
    Light-->Final[[Final]]
    Samurai-->Final
Loading

In this diagram, the choice of mode combined with the theme changes which set is used for final resolution. If the mode modifier is changed, the choice of potential candidates for theme changes as well

flowchart TD
    Mode(Mode) --> Light
    Mode -->Dark
    Light(Light)-->SamuraiL[Samurai:Light]
    Light-->KnightL[Knight:Light]
    Dark(Dark)-->SamuraiD[Samurai:Dark]
    Dark-->KnightD[Samurai:Dark]
    SamuraiL-->Final[[Final]]
Loading

In the above example we purposefully chose values such that the we had balanced options where each mode has the same options to choose from, lets now see an example where this becomes unbalanced

flowchart TD
    Mode(Mode) --> Light
    Mode -->Dark
    Light(Light)-->SamuraiL[Samurai:Light]
    Light-->KnightL[Knight:Light]
    Light-->PirateL[Pirate:Light]
    Dark(Dark)-->SamuraiD[Samurai:Dark]
    SamuraiL-->Final[[Final]]
Loading

In this case the choice for resolution becomes contextual. ONLY if the chosen mode is Light do you have the option of choosing the Pirate theme and if you do chose both Dark and Pirate you have made an invalid selection.

This is a basic example using only 2 modifiers but once other modifiers such as "Brand", "Surface", etc are included into the mix, more logic would be necessary to both handle the resolution of this, and also to express what combinations are allowed. This adds complexity to the overall solution.

Future extensions

We have attempted to keep the resolver structure as basic as possible outside of some minor additions like the name and description that could be useful.

As part of extensions for the spec, we might want to have a resolver potentially refer to another resolver so that certain calculated sets might be blackboxed. This might take the form of using a different type in the modifiers object to specifiy a resolver as opposed to an enumerated or include type which might reference a resolver.

This is only really useful if we cannot create precomputed token sets, but might have its uses otherwise. An example of such might be to relay context to another resolver to simulate properties such as surface logic.

In the example shown, if the visual modifier could not be normalized to be completely orthogonal to the mode, via the use of a semantic layer which could handle the mapping, ie

graph LR
  A["In-Scope Visual Tokens"]--> B
  B["Semantic Layer"] --> C["Output"]
Loading

and the choice of sets is dynamic because of it, eg src/tokens/functional/color/light/overrides/light.protanopia-deuteranopia.json5 vs src/tokens/functional/color/dark/overrides/dark.protanopia-deuteranopia.json5,
then the output would depend upon the mode modifier for the visual modifier. To prevent overloading the complexity of one resolver for the different dimensions, one resolver would just read the output of another resolver either directly or through a precomputed set as previously mentioned. The alternative would be to change the resolver specification to express the acceptable combinations through the use of a tree structure that could be pruned. Looking back at our example of a non orthogonal modifier structure :

flowchart TD
    Mode(Mode) --> Light
    Mode -->Dark
    Light(Light)-->SamuraiL[Samurai:Light]
    Light-->KnightL[Knight:Light]
    Light-->PirateL[Pirate:Light]
    Dark(Dark)-->SamuraiD[Samurai:Dark]
    SamuraiL-->Final[[Final]]
Loading

Assuming this is valid and enforceable, this also reduces the space to only 4 sets as opposed to the 3 x 2 =6 permutations for each mode with each theme. For design systems that want to use a resolver spec as a source of truth, they could use it to construct a graph showing the different possible variations of a component, making it explicit which combinations are valid and invalid for the different dimensions. This could likely be useful in cases where the choice of the brand dimensions heavily affects the possible values of the other modifiers as brands might offer completely different combinations. This should likely be addressed by the use of seperate resolvers if they are heterogenous in their modifier usage, but regardless this could be more efficient for export tools.

As @romainmenke mentioned there will likely need to be some form of bridge connecting the possible values of a token and then providing some form of metadata to it to then be used in the various platforms, likely some form of output with conditional rules. Seeing as this is likely going to be dynamic as it needs to be evaluated at runtime (eg screen size) its outside the scope of what the resolver on its own might do without some additional system that could be embedded into a front end to do this.

A modifier that is not acceptable for this form of resolving is an example such as component state. A component can exist in multiple different potential states, eg Focused, Hover and in Error state, and each of these states might have different levels of precedence, such as the red color of the background always being shown.

Lastly the Primer examples shows an interesting point of using glob based selection of the sets. This could be useful in solving the issue shown above with regards to the file paths of the visual afflictions being dependent on the mode. If we potentially allow string interpolation based on the modifer inputs that could simplify the structuring and not require refactoring of semantic layers to enable this approach

@ddamato
Copy link

ddamato commented May 18, 2023

For us, having distinct files would be unmanageable. We define hundreds and hundreds of tokens, currently with 18 different modes applied. Getting the full picture of what a token's value can be would be difficult, and we find that once the token system is set up, we update tokens rather than updating modes. Having a single file where a token showcases all its values make that easier to manage.

I'd love to see both formats supported by tools, though I believe what tools like Style Dictionary currently do (multiple files with a value per token per file) is too inconvenient for long-term maintenance workflows.

@Sidnioulz could you describe your curation process a bit further?

Also what purpose does seeing a token's value across themes provide? From my perspective, token values curated collectively by theme reduces context switching across themes because we commonly select values for tokens related to other existing values in the same theme. Seeing the token value across themes out of context of the larger theme doesn't seem helpful but I'm interested to know how it could be?

@robinscholz
Copy link

@SorsOps Really interesting proposal, with a lot of flexibility I think.

We are actually in the midst of setting up a flexible white label design system, which should be able to support an arbitrary amount of themes/modifiers. In order to do this, we‘ve come up with a custom style-dictionary implementation that can take a config built around the idea of what is called sets in your proposal.

Our implementation is a work in progress and much simpler than the one above, but brings two key differences:

  • Instead of explicitly naming filenames, we work with glob patterns to be able to reduce the amount of LOC while keeping a human readable format.

  • We are also able to filter the output based on a glob pattern that matches the name of each token. We implemented this, since we ran into situations where a designer might need a specific token (i.e. font-size for an icon font), that an engineer might not use (icons implemented as svg with a …-dimension-height token). This reduces bundle size.

As far as I understand your example, it is possible to granularly combine tokens from various inputs, but it isn’t possible to further filter them. This would mean the input token structure still influences the outcome, since you‘d need to carefully group certain tokens in certain files. I might be wrong though, so please feel free to clarify if thats the case!

  • We are able to modify each token names to get rid of arbitrary token group names such as default, which Figma Tokens still requires since token names can’t share group names.

For reference, here is the (very simple) TS definition of the current config implementation:

type ConfigToken = {
  path: string // Used as a glob pattern
  ignore?: string | string[] // Used as (a) glob ignore pattern(s)
  selector?: string // CSS Selector
  filename?: { // Filename modifier, used to create output folder structures
    prefix?: string
    suffix?: string
    replace?: {
      from: string
      to: string
    }[]
  }
  tokens?: {
    ignore: string[] // Mulitmatch pattern to ignore certain tokens
  }
}

type ConfigTokenGroup = {
  selector?: string // Optional CSS Selector for all entries in the group
  files: ConfigToken[]
  filename?: {  // Optional filename modifier for all entries in the group
    prefix?: string
    suffix?: string
    replace?: {
      from: string
      to: string
    }[]
  }
}

type ConfigOptions = {
  basePath?: strin
  fileType?: string
  log?: boolean
}

type TokenTransformConfig = {
  options: ConfigOptions
  tokens: {
    include?: ConfigToken[] | ConfigTokenGroup
    source?: ConfigToken[] | ConfigTokenGroup[]
  }
}

Taking cues from style-dictionary, tokens within include are always imported as values that might be referenced, whereas everything in source is parsed and transformed separately.

I'm unsure how any of this would fit into the proposal, but I do think it might be generally useful.

@SorsOps
Copy link

SorsOps commented Jun 1, 2023

Hey @robinscholz.

Instead of explicitly naming filenames, we work with glob patterns to be able to reduce the amount of LOC while keeping a human readable format.

I saw in Primer they use the same. I understand the use case, but it would make it non deterministic to not know which files are necessary for the resolution, also assuming that the glob returns values in the same order which might cause a problem.

We are also able to filter the output based on a glob pattern that matches the name of each token. We implemented this, since we ran into situations where a designer might need a specific token (i.e. font-size for an icon font), that an engineer might not use (icons implemented as svg with a …-dimension-height token). This reduces bundle size.

I know what you mean. Trying to reduce the emitted tokens is ideal. This could be done through a post resolution step to choose the appropriate tokens prior to placing it in the style dictionary. I think this would be considered an edge case as we would either need to encapsulate the output and make it a single set output or potentially break it down into further resolvers for the pieces?

I think the takeaway is that we would not want to tie the support specifically to style dictionary as we would need to keep separation of concerns. Style dictionary has a very specific role handling transformation for platforms and rules for the output . I think this should be kept seperate from just the resolution step so we can keep simple tools for those stages

@c1rrus c1rrus closed this as completed Jun 28, 2023
@c1rrus c1rrus reopened this Jun 28, 2023
@c1rrus
Copy link
Member

c1rrus commented Jun 28, 2023

Gah! Sorry y'all I did not mean to close the issue just now. That was just my fat fingers on my phone!

@c1rrus
Copy link
Member

c1rrus commented Jun 28, 2023

Thanks so much for sharing the resolver proposal, @SorsOps. It's super interesting. I also had a brief play with the online tool you shared and it looks pretty sweet.

As you say, this moves the problem of how to handle tokens with variable values (for brands, themes, color schemes, information densities, etc. etc.) out of the scope of the DTCG format, which could help keep the spec simple for now. (aside: I do think there are some concepts within that resolver spec that might be nice to absorb into the spec in the future though)

However, it does raise at least one requirement for the DTCG format:

  • A single DTCG file must be allowed to contain references that cannot be resolved within that file

Otherwise, the files used as modifiers would not be able to contain aliases to tokens provided by the sets.

Currently the DTCG spec does not forbid this, but it doesn't explicitly allow it either. If we were to agree that we wanted to allow this, then I think we should explicitly state that in the spec to avoid differing interpretations (e.g. one tool assuming references must be resolvable within the same file and therefore rejecting files where that's not true).

(Btw, I think this current ambiguity in the spec was highlighted before in another issue, but I can't find it right now)

Thing is, do we want to allow that?

Personally, I quite like the idea that every DTCG file is self-contained. I think that can make them easier for humans to reason about. It could also make them more portable - e.g. the use-case mentioned above about referencing tokens from another DS's tokens. If you can cherry pick any .tokens.json file safe in the knowledge that it's not going to contain references that might not resolve, then I think that becomes easier.

What this thread is making clear is that there is a strong demand for being able to organise tokens into separate files. Forgetting about modes/themes/whatever for moment, that's desirable even if it's just to divide up a large collection of design tokens into more readable and manageable chunks.

I've got some ideas for how we could add that facility within the DTCG spec. However, I'll write it up as a separate issue and link to it here once I have.

@romainmenke
Copy link
Contributor

Btw, I think this current ambiguity in the spec was highlighted before in another issue, but I can't find it right now

this one?

#123 (comment)

@c1rrus
Copy link
Member

c1rrus commented Jun 28, 2023

Yup, that's the one. Thanks for fishing it out, @romainmenke!

@wuweiweiwu
Copy link

re: composite tokens and multiple themes (which i believe token studio supports currently)

it doesn't seem like the current shape of the spec supports being in multiple "modes" at the same time

mainly thinking in terms of composite tokens each mode overrides a specific property? Maybe it's not a valid use case though

@equinusocio
Copy link

equinusocio commented Jul 13, 2023

Sticking to the first post/request and the original issue, we should ignore tools/figma/plugins/third-party implementations and requests since the spec must be agnostic and must work for everyone.

Now, as the author of the original issue, in my opinion, theming and aliasing are the same thing. What I mean, once you can create an alias, you can create as many themes as you want, the only difference is the token name. There is no need to add a "specific way" to do theming... and "modes" is really a misleading name and a concept that IMHO should not be bound to the token itself. Design token should be simple, and hold a single raw value.

I'm posting here this really explanatory screenshot:

Following this important screenshot and the theming concept, we can achieve this with the current spec just with aliasing.

colors.tokens.json

{
  "$name": "Color Tokens",
  "red": {
    "$type": "color",
    "$value": "#ff0000"
  },
  "blue": {
    "$type": "color",
    "$value": "#0000ff"
  }
}

light-theme.json

{
  "$name": "Light Theme",
  "global": {
    "background": {
      "$type": "color",
      "$value": "{colors.red}"
    },
    "foreground": {
      "$type": "color",
      "$value": "{colors.blue}"
    }
  }
}

light-theme.json is just a set of semantic-named tokens that is transformed by the author for required platforms (transformation is a required step but outside the spec). For the web, for example, it could just become:

light-theme.css

:root {
 --global-background: #ff0000;
 --global-foreground: #0000ff; 
}

The only issue here is interoperability and alias resolving, no modes, no "theming" feature, or whatever. But since tokens must be always compiled/transformed, cross-file resolving should not be a big issue (but that's not my field)

The fact of having "modes" bound to the token is only a technical requirement for the Figma team and should not be the focus of the spec since it also violated one of the DT principles:

Design tokens don't change across implementations. They are constants.

Supposing we really want to bind everything to the token and single file, authors will end up with infinite JSON if they have to put everything in a single file. Also, tokens transformation could be a pain.

Warning Personal opinion ahead

Expand personal opinion

Personally, I don't like how this issue is now focused to satisfied only Figma technical requirements, instead of focusing on the spec, simplicity, and platform interoperability. If you really want to shape the spec around specific companies' requirements and put theming/transformation inside the spec, I'll probably never follow the spec and stick to Style Dictionary whereas as an author I can have control over token transformation over the platforms and where design tokens respect the core principles:

  • Design tokens don't change across implementations. They are constants.
  • Design tokens are platform-agnostic raw data; you must transform them into specific formats used by destination platforms, like the web, iOS, and Android.
  • Design tokens are non-semantic. They hold an absolute value.
  • Design tokens are the single source of truth to keep consistency across the system and platforms

(btw, principles we would violate if we add modes since the value of the token will change across the systems/platforms.

@mryechkin
Copy link

@equinusocio hard agree on all points 💯

@jjcm
Copy link
Author

jjcm commented Aug 15, 2023

@equinusocio agree that you can represent a single theme with just aliasing, however approaching it that way does have its issues, with the most major one being no guaranteed interoperability between themes. If you switch from light-theme.css to dark-theme.css, how does it work? How do I as a developer know it will have all of the tokens defined? Is dark-theme.css a set of overrides for light-theme.css, or is it entirely self contained? The issue with ambiguity here is it means people will approach this in two separate ways, which ends up being bad for the standard.

To that point, @c1rrus brings up a great point here:

Personally, I quite like the idea that every DTCG file is self-contained. I think that can make them easier for humans to reason about. It could also make them more portable - e.g. the use-case mentioned above about referencing tokens from another DS's tokens. If you can cherry pick any .tokens.json file safe in the knowledge that it's not going to contain references that might not resolve, then I think that becomes easier.

A self-contained aspect is quite nice for understandability. Not necessary by any means, but it is a nice to have. It means tools can be a bit more proactive around what they expect the file to have. In a multi-file scenario, load order and parsing becomes a bit more tricky. It's still workable for sure, but tricky.

@SorsOps's proposal feels like a bit of a hybrid of both my proposal and a @equinusocio's (simple files with no theming). I'm OK with @SorsOps's proposal, but I will +1 @c1rrus's points about cross-file dependencies and the issue of

A single DTCG file must be allowed to contain references that cannot be resolved within that file

While the spec doesn't have an opinion about this now, if we do pursue something like @SorsOps's approach, it'd be critical to explicitly state this in the spec. That's boils down to the heart of the issue and really my perspective here. Our request isn't "make the spec work for Figma" - this should be a spec that works for everyone, not one specific tool. But equally, we also shouldn't "ignore tools/figma/plugins/third-party implementations", as doing so will fracture the spec into multiple implementations. Themes are an extremely common use case for tokens, and my worry here is the spec not having an opinion will lead to many different approaches to a solution (which we already see today).

At the end of the day I care less about the exact structure/format of how we represent theming. Our requirements are just to have a solution for this, not to have a specific solution. What I don't want to end up having is there being 8 different ways people approach this, or worse still Figma having undue weight here where people align on whatever way we do this in the interim (as it wont have been done with regard to other tools' needs in mind).

@dev-nicolaos
Copy link

Reading this thread has been enlightening. I didn't fully appreciate the scope of use cases for design tokens and the huge variety in those use cases scale.

I'm currently in the process of setting up a pipeline to ingest, transform and consume tokens for my company. In the first day of researching/testing tools I saw three different tools use three different ways to represent "modes/themes" (at least two of those approaches have been discussed here). @jjcm's comment sums up my feelings well:

At the end of the day I care less about the exact structure/format of how we represent theming. Our requirements are just to have a solution for this, not to have a specific solution. What I don't want to end up having is there being 8 different ways people approach this,

I know this is a long (and probably exhausting) discussion, but I want to encourage the invested parties to continue pushing towards a consensus. All approaches have drawbacks, but with the amount of thought put in here it seems likely that whatever comes out will be better than if the spec becomes less relevant because its not meeting the needs of the industry and everybody tries to solve it on their own.

@equinusocio
Copy link

equinusocio commented Dec 17, 2023

@equinusocio agree that you can represent a single theme with just aliasing, however approaching it that way does have its issues, with the most major one being no guaranteed interoperability between themes. If you switch from light-theme.css to dark-theme.css, how does it work? How do I as a developer know it will have all of the tokens defined? Is dark-theme.css a set of overrides for light-theme.css, or is it entirely self contained? The issue with ambiguity here is it means people will approach this in two separate ways, which ends up being bad for the standard.

To that point, @c1rrus brings up a great point here:

Personally, I quite like the idea that every DTCG file is self-contained. I think that can make them easier for humans to reason about. It could also make them more portable - e.g. the use-case mentioned above about referencing tokens from another DS's tokens. If you can cherry pick any .tokens.json file safe in the knowledge that it's not going to contain references that might not resolve, then I think that becomes easier.

A self-contained aspect is quite nice for understandability. Not necessary by any means, but it is a nice to have. It means tools can be a bit more proactive around what they expect the file to have. In a multi-file scenario, load order and parsing becomes a bit more tricky. It's still workable for sure, but tricky.

@SorsOps's proposal feels like a bit of a hybrid of both my proposal and a @equinusocio's (simple files with no theming). I'm OK with @SorsOps's proposal, but I will +1 @c1rrus's points about cross-file dependencies and the issue of

A single DTCG file must be allowed to contain references that cannot be resolved within that file

While the spec doesn't have an opinion about this now, if we do pursue something like @SorsOps's approach, it'd be critical to explicitly state this in the spec. That's boils down to the heart of the issue and really my perspective here. Our request isn't "make the spec work for Figma" - this should be a spec that works for everyone, not one specific tool. But equally, we also shouldn't "ignore tools/figma/plugins/third-party implementations", as doing so will fracture the spec into multiple implementations. Themes are an extremely common use case for tokens, and my worry here is the spec not having an opinion will lead to many different approaches to a solution (which we already see today).

At the end of the day I care less about the exact structure/format of how we represent theming. Our requirements are just to have a solution for this, not to have a specific solution. What I don't want to end up having is there being 8 different ways people approach this, or worse still Figma having undue weight here where people align on whatever way we do this in the interim (as it wont have been done with regard to other tools' needs in mind).

Theme switching should not be part of the spec. How to handle themes really depends on the supported platforms, how tokens are transformed and consumed, if there are themes, etc.

The spec should allow strong syntax and aliasing for raw tokens, the token transformation (which is mandatory if you work with multiple platforms) is up to the authors

In my example, considering the web platform, I have two identical css files (because as the author I wrote the token and I know they are the same, with the same keys but different values). Swapping is not an issue and is not part of the spec anyway.

@universse
Copy link

universse commented Dec 22, 2023

This is my very first community involvement here so hopefully it adds something useful to the discussion.

Just a little background, I'm writing a token management and code generation tool that enables fully themeable design system. The focus is pretty much on the web ecosystem right now and I hope to expand it to other platforms in the future.

Let's first focus on conditional token values within a product. As mentioned by many others, these conditions comprise modes and system settings.

In my proposed format, these conditions are also defined as design tokens. They can then be used to generate PostCSS custom media & custom selectors, SASS mixins, Tailwind variants, TypeScript type definitions etc.

Unlike other design tokens which can be nested arbitrarily in groups, condition tokens are defined in a more structured manner.

Each alternative token value is itself a design token, defined using the keyword $set. They can refer to defined condition tokens using the keyword $condition.

{
  condition: {
    color_scheme: {
      light: {
        $value: '[data-color-scheme="light"]',
      },
      dark: {
        $value: '[data-color-scheme="dark"]',
      },
    },
    contrast: {
      standard: {
        $value: '[data-contrast="standard"]',
      },
      more: {
        $value: '[data-contrast="more"]',
      },
    },
    motion_pref: {
      none: {
        $value: '@media (prefers-reduced-motion: no-preference)',
      },
      reduced: {
        $value: '@media (prefers-reduced-motion: reduce)',
      },
    },
  },
  color: {
    primary: {
      $set: [
        {
          $condition: {
            color_scheme: 'light',
            contrast: 'standard',
          },
          $value: '{color.purple.80}',
        },
        {
          $condition: {
            color_scheme: 'light',
            contrast: 'high',
          },
          $value: '{color.purple.90}',
        },
        {
          $condition: {
            color_scheme: 'dark',
            contrast: 'standard',
          },
          $value: '{color.purple.20}',
        },
        {
          $condition: {
            color_scheme: 'dark',
            contrast: 'high',
          },
          $value: '{color.purple.10}',
        },
      ],
    },
  },
}

Translated to CSS:

:root[data-color-scheme='light'][data-contrast='standard'] {
  --color-primary: var(--purple-80);
}

:root[data-color-scheme='light'][data-contrast='more'] {
  --color-primary: var(--purple-90);
}

:root[data-color-scheme='dark'][data-contrast='standard'] {
  --color-primary: var(--purple-20);
}

:root[data-color-scheme='dark'][data-contrast='more'] {
  --color-primary: var(--purple-10);
}

The proposal can be extended to support component variants, a concept that has been adopted by various CSS libraries. There are parallels between the 2 concepts - conditions are visual variations of a product while component variants are visual variations of a component. One is global, the other local.

app conditions
  - color scheme mode:
    - light
    - dark
  - contrast mode:
    - standard
    - more

button variants
  - intent:
    - primary
    - secondary
  - style:
    - filled
    - outline

Just like a semantic token can have alternative values under different conditions, a component token can have alternative values for different component variants.

In CSS, it looks like this:

.button[data-intent='primary'][data-style='filled'] {
  --button-background-color: var(--color-primary);
  --button-border-color: transparent;
}

.button[data-intent='primary'][data-style='outline'] {
  --button-background-color: transparent;
  --button-border-color: var(--color-primary);
}

.button[data-intent='secondary'][data-style='filled'] {
  --button-background-color: var(--color-secondary);
  --button-border-color: transparent;
}

.button[data-intent='secondary'][data-style='outline'] {
  --button-background-color: transparent;
  --button-border-color: var(--color-secondary);
}

Similar to conditions, component variants are defined as design tokens. A component token can have multiple values defined using the keyword $variant.

{
  // component tokens are specified under "component" group
  component: {
    button: {
      // define variants
      $variant: {
        intent: {
          primary: {
            $value: '[data-intent="primary"]',
          },
          secondary: {
            $value: '[data-intent="secondary"]',
          },
        },
        style: {
          filled: {
            $value: '[data-style="filled"]',
          },
          outline: {
            $value: '[data-style="outline"]',
          },
        },
      },
      background_color: {
        $set: [
          {
            $variant: { intent: 'primary', style: 'filled' },
            $value: '{color.primary}',
          },

          // ... other variations

          // combining with $condition
          {
            $condition: { contrast_pref: 'forced' }, // resolves to "@media (forced-colors: active)"
            $variant: { intent: 'primary', style: 'filled' },
            $value: 'ButtonText',
          },
        ],
      },
    },
  },
}

Adopting condition and variant thus creates more opportunities for design system automation. In the context of websites and web applications, since a component's visual styling can be represented almost entirely by CSS custom properties generated from design tokens, the component's core CSS can remain stable across products/frameworks.

/* stable, fully themeable component CSS */
.button {
  background-color: var(--button-background-color);
  border-color: var(--button-border-color);
}

/* generated from design tokens */
.button[data-intent='primary'][data-style='filled'] {
  --button-background-color: var(--color-primary);
  --button-border-color: transparent;
}

.button[data-intent='primary'][data-style='outline'] {
  --button-background-color: transparent;
  --button-border-color: var(--color-primary);
}

.button[data-intent='secondary'][data-style='filled'] {
  --button-background-color: var(--color-secondary);
  --button-border-color: transparent;
}

.button[data-intent='secondary'][data-style='outline'] {
  --button-background-color: transparent;
  --button-border-color: var(--color-secondary);
}

@media (forced-colors: active) {
  .button[data-intent='primary'][data-style='filled'] {
    --button-background-color: ButtonText;
  }
}

Theming across multiple products on multiple platforms can be achieved simply by combining different token sources.

Brand A web:
  - core-tokens
  - web-tokens
  - brand-a-tokens
  - brand-a-web-tokens

Brand B web:
  - core-tokens
  - web-tokens
  - brand-b-tokens
  - brand-b-web-tokens

As the major focus of the spec is platform independence, I understand it is controversial to have platform-specific concepts in the design token format. For a code generation tool, I argue this is necessary as it allows using platform capabilities to the fullest. Overlooking platform-specific concepts, design tokens cannot fully represent all the design decisions within the system. Those instead need to be handled via tooling and source code, thereby reducing some degree of visibility and control.

Last but not least, since the tool is fairly new, I would very much appreciate it if you could try it out and share your feedback. Thank you very much.

@jorenbroekema
Copy link

jorenbroekema commented Feb 19, 2024

Theme switching should not be part of the spec

Fully agree with @equinusocio with emphasis on "switching", at least I assume that's what the emphasis should be on. How you switch between themes is heavily dependent on the output platform, and I feel like I should point out why with an example.

Let's imagine you have a button with light and dark mode, you might have the option to choose between two ways of outputting the CSS to accompany the way you switch themes in your site:

This example is the one I see most developers think of first:

:root {
  --button-padding: 8px;
  --button-bg-color: #000;
}

:root[mode="dark"] {
  --button-bg-color: #FFF;
}
// or alternatively:
@media (prefers-color-scheme: dark) {
  :root {
    --button-bg-color: #FFF;
  }
}

That combines both modes into a single stylesheet, and following this pattern you would be including all theming options into 1 stylesheet. This is not ideal for performance reasons, your end users are downloading redundant kilobytes of CSS for rules that don't apply, because you can't both be on light and dark mode simultaneously. In the Web world, where initial load is super important for bounce rates (users leaving if sites load longer than 2 seconds etc.), the more ideal approach is to create different stylesheets:

button.css:

:root {
  --button-padding: 8px;
}

button-light.css:

:root {
  --button-bg-color: #000;
}

button-dark.css:

:root {
  --button-bg-color: #FFF;
}

Assume that the amount of CSS rules would be way more than 1, I'm just keeping the example simple, but the amount of KBs you save goes up fast with the amount of theme-specific CSS rules and theme variations you have in your output.

You'll have some utility that will allow theme switching to happen and for the stylesheets to be swapped out at runtime, here's a demo of a run-time themable button which applies this approach. Here's the source code for the stylesheet switcher on theme toggle.

What this means is that the initial amount of KBs is far lower, because you're only loading the CSS relevant for the current chosen combination of themes/modes. Then, upon changing a theming dimension, you load the CSS needed for it on-demand. This minor delay when switching is the tradeoff versus a big delay on initial load, in Web context that is usually very much worth it considering that Web users tend to download your "app" on the fly, often coming from a search engine, and initial load matters a lot for whether they leave your site prematurely.

Now imagine Android or iOS apps, these are downloaded from the app stores, and the downloading is a conscious choice by the user where waiting a couple of seconds doesn't deter them from then using your app. Every variation of the app based on the theming dimensions is downloaded at once, making the switching between themes very cheap. This changes the "initial load" versus "switching delay" tradeoff in favor of the former, it's the opposite when you compare it to Web context. Putting all the themes outputs in the same file (e.g. a single button.swift or button.xml file) probably makes more sense for these mobile platforms, at least when you come at it from this particular performance/UX angle.

Hence a platform-agnostic design token spec should not have an opinion on the theme switching itself. I hope I've managed to make a good argument on why that is, why approaches to theme-switching is heavily platform-dependent.

Edit

As @nesquarx points out after this post below: yes, the theming classification itself, 'what token
changes how for what theme' should definitely be something the design tokens spec addresses, I fully agree, just wanted to clarify that to prevent confusion

@nesquarx
Copy link

nesquarx commented Feb 19, 2024 via email

@equinusocio
Copy link

equinusocio commented Mar 21, 2024

the theming classification itself, 'what token
changes how for what theme', definitely should be the purview of tokens.

Any token that violates the core principles above is considered a "theming token", whose value may change across platform implementations (so it can't be considered part of the source of truth). The first step would be to define what is a constant token (respecting all the principles) and what is a "theming token" which is more flexible.

We should consider that constant/raw design tokens are mandatory in a design system, theming tokens aren't.

Btw this discussion seems dead...

@danosek
Copy link

danosek commented May 7, 2024

Just for inspiration - I'm using my own system (for now), and this is how I deal with themes.

Non-themeable tokens

"blue-10": {
    "name": "blue-10",
    "humanName": "Blue 10",
    "css": "--blue-10",
    "value": "oklch(0.95 0.02 250)",
    "modifier": "10",
    "def": "0.95 0.04 250",
    "colorSpaces": {
        "oklch": "oklch(0.95 0.02 250)",
        "hex": "#e5f0fc",
        "rgb": "rgb(230 240 250)",
        "hsl": "hsl(210 78% 94%)"
    }
},
"blue-100": {
    "name": "blue-100",
    "humanName": "Blue 100",
    "css": "--blue-100",
    "value": "oklch(0.3 0.07 250)",
    "modifier": "100",
    "def": "0.3 0.07 250",
    "colorSpaces": {
        "oklch": "oklch(0.3 0.07 250)",
        "hex": "#0d2f4f",
        "rgb": "rgb(13 47 79)",
        "hsl": "hsl(210 72% 18%)"
    }
},

Themeable token

"minimal": {
    "name": "surface-blue-minimal",
    "humanName": "Surface Blue Minimal",
    "css": "--surface-blue-minimal",
    "theme": {
        "light": {
            "value": "blue-10",
            "css": "--blue-10",
        },
        "dark": {
            "value": "blue-100",
            "css": "--blue-100",
        }
    },
},

For CSS I then generate something like this (we are using JS to manipulate themes).

:root {
    --blue-10: oklch(0.95 0.02 250);
    --blue-20: oklch(0.89 0.06 250);
    --blue-30: oklch(0.84 0.09 250);
    --blue-40: oklch(0.8 0.11 250);
    --blue-50: oklch(0.56 0.18 250);
    --blue-60: oklch(0.55 0.2 250);
    --blue-70: oklch(0.54 0.19 250);
    --blue-80: oklch(0.48 0.14 250);
    --blue-90: oklch(0.4 0.11 250);
    --blue-100: oklch(0.3 0.07 250);
}

:root,
[data-theme$="light"]:not(:root),
:is(:root[data-theme$="dark"] [data-theme="inverted"]) {
    --surface-blue-minimal: var(--blue-10);
}

:root[data-theme$="dark"],
[data-theme$="dark"]:not(:root),
:is(:root[data-theme$="light"] [data-theme="inverted"]) {
    --surface-blue-minimal: var(--blue-100);
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests