Proposal: Reference shortcut for nesting tables #744
Comments
|
I agree that TOML would benefit from something like this, and indeed it's not the first time this sort of thing has been proposed. I'm not thrilled about the use of [servers]
[*.alpha]
ip = "10.0.0.1"
dc = "eqdc10"
[*.beta]
ip = "10.0.0.2"
dc = "eqdc10" |
|
Strictly speaking, there is not even a need for a special marker – just using one or more dots to the left of the first explicitly written key part would do the trick. Then your example could be written as follows: [servers]
[.alpha] # servers.alpha
ip = "10.0.0.1"
dc = "eqdc10"
[..firewall] # servers.alpha.firewall
inbound = "0.0.0.0/24"
outbound = "0.0.0.0/24"
[.beta] # servers.beta
ip = "10.0.0.2"
dc = "eqdc10"Personally, I like the idea and would not mind seeing it in a future version of TOML. (It won't make it into v1.0 which is feature-complete.) |
|
@brunoborges To be honest, I'm slightly skeptical of proposals to reduce table headers' verbosity. But this one is more attractive than others I've seen, because there's no attempt to make the headers' reference level any less than absolute. This syntax is relative in that it does require previously defined headers to describe what super-tables are being referenced. But as long as the table hierarchy is preserved going back to the root table, this syntax could work. @ChristianSi @marzer Another character could be called for to express depth. I feel like single dots may be hard to read, even if they are easy to type. May I suggest the character |
|
I'd hope we would focus on two things here but separately:
For now, if possible, it may be better to use # just as for the purpose of discussing 1. Then later we can go to 2, which could be as simple as voting on multiple proposals. That said, here's my first restriction for this feature:
This means the following should be not allowed: fruit.apple="red"
#.banana="yellow" |
|
Agreed. Let's get a proper definition of This will require that we relinquish, partly, a rule that's been around a long time: that tables, subtables, and super-tables may be defined in any order. Order will make a difference if shortcuts are used. We'll then need explicit rules for the following. |
Yes, for table definitions. Are you excluding arrays of tables, then? |
|
@eksortso If we call the existing table definitions "explicit" and these new ones "implicit", then the wording of the ordering rule need only change to reflect the difference. Something like: begin spec snippet In addition to explicit subtable definitions, TOML supports implicit definitions of subtables using the placeholder [alpha]
this-table = "alpha"
[#.beta]
this-table = "alpha.beta"Placeholders will match against the most-recently defined super-table with the same depth (one must exist): [alpha]
[#.beta] # ok, starts table 'alpha.beta'
[#.#.gamma] # ok, starts table 'alpha.beta.gamma'
[#.#.#.#.yotta] # ERROR! too deepThis means that unlike explicit table definitions, which may appear in any order, implicit table definitions are order-dependent. end spec snippet @brunoborges Something that's not clear from your initial proposal: can placeholders appear to the right of an explicit key? e.g.: [alpha]
[alpha.beta]
[alpha.#.gamma] # should this be legal?? |
|
@marzer that's a great point. I think such thing as But then again, IMO the order must still be respected. Only explicit keys (i.e. without #) may appear out of order. |
|
Would just like to comment that the verbose syntax of nested keys is one of the common complaints about TOML - see https://hitchdev.com/strictyaml/why-not/toml/ and the criticism section of Wikipedia's TOML page. |
|
Related: #413 |
TOML is not for arbitrarily-nested data representation, but rather for configuration files for tooling. It serves as a well-defined replacement for INI files. In other words, arbitrarily-nested data structures are gonna "look better" in JSON/YAML since they are data representation formats and TOML is not that. Of all the proposals I've seen till date, this is probably the most compelling one, for a shorthand in table headers. That said, I'm not super keen on deciding on this just yet, so let's revisit this after 1.0 is out. :) |
|
Hey, isn't a 1.0 the perfect opportunity to introduce syntax change like this? When it gets out, everyone is going to update their parsers since it's such a big release, not sure people will be so keen for point releases. |
|
@alvincodes The short answer is No, because parsers have already updated, or they ought to have been. Adding new features (like this one) is usually done well in advance of major revision updates, typically during alpha or beta stages. It's a little more flexible before a 1.0 drops, but we have two pre-releases out already. Feature lock is essentially assumed, and new features would require extremely persuasive arguments to push through at this stage. When a "feature freeze" occurs in a project depends on how the project is run, but the idea is that enough time is given for users to try out the new features in alpha, beta, etc. stages. In the case of the TOML standard, the parser writers can try on the new syntaxes, shake out any issues that are found, and provide feedback. As it stands, there are a bunch of changes that went in since v0.5.0 that parsers will need to implement. Would we want to throw this onto that pile, even though we've not spoken about it for three months? Edit: adjusted text referring to changed since v0.5.0; better late than never. |
|
I agree this feature is not extremely magical for being pushed through 1.0. But I'd love to see it in the next version of the spec. |
|
I was just trying out TOML for the first time. Where the indentation mattered. But there's no where any mention that this isn't the case. I found some mention of white-space should be ignored around a table definition, but I couldn't find that under https://toml.io/en/v1.0.0-rc.3#table. So naturally they should have some meaning if the behavior around whitespace is undefined. But even without reading the specs (or if I missed it some where), naturally I would assume (coming from a Python background I guess) that the indentation mattered, and both {
"servers" : {
"alpha" : {
"ip" : "10.0.0.1",
"dc" : "eqdc10"
},
"beta" : {
"ip" : "10.0.0.2",
"dc" : "eqdc20"
}
} |
|
Well, @Torxed, you're not the first person with those same thoughts. TOML is influenced by old INI configuration formats, but I don't think anyone ever considered that there are people who have never come across an INI file before. Nesting can be done in any of three different ways: a table header with a dotted name (illustrated below), dotted keys inside a table section, and inline tables. The table headers are always absolute table references, and inline table names and dotted keys are always relative to the current table section. Indentation of table headers and keys are not significant (they're treated as whitespace) and are just available as visual reminders of nesting levels. At its simplest, your example would look like this (and the parent table header [servers.alpha]
ip = "10.0.0.1"
dc = "eqdc10"
[servers.beta]
ip = "10.0.0.2"
dc = "eqdc10"So now I'm considering writing a PR to either remove the indentation from the markdown document's examples, or to explain the indentations away. As it stands now, the indentation just confuses those with no experience in any dialect of INI. |
|
@eksortso Looks like you used the wrong syntax highlighter in your example. It shouldn't be all that red. |
|
@ChristianSi It was something that I'd copied and pasted, so I just typed in the code again, and it highlighted correctly. |
I'd say both. :) PRs welcome and this doesn't really need another round through the Release Candidate cycle. |
|
Hi all, I'd like to ask for a quick vote on which symbol/character to use. Please use the corresponding emoji and react to this comment with your preference.
|
|
Forgot the |
None of the above? Asterisks imo. |
|
@Torxed @marzer added both options to the poll. |
|
I'd suggested a single greater-than / right angle bracket I'd accept the two-character combo So I voted |
|
I like the fact that something.here = true
something.else = falseAnd omitting an entry prior to the dot, is simply a reference to a continuation. |
|
People seem very fond of the 'just dots' syntax. I must caution against this; adding this syntax to the language would be adding something visually indistinct and error-prone. Example: While still being unambiguous, absolute paths, I do not believe that they're visually distinct enough to be able to easily determine the depth at-a-glance. Even just writing this example, I added The alternative suggestion of @pradyunsg mentioned that the proposal as-written is more compelling than the many alternatives for solving this problem, and I'm inclined to agree. Thus, the vote should only be about deciding which character plays the role of the the proposal's 'placeholder' character, and anything else should be relegated to separate proposals. @eksortso so yeah, my suggestion of |
|
Thanks @marzer. I've changed my vote to support Thanks, also, for stating the case against single-char placeholders so convincingly. I hope that it compels others to reconsider. |
|
@sgarciac Personally I think dotted key-value-pairs are ugly and repetitive, while [headers] are clean and declarative. Being able to replace dotted KVP's with headers in more places would be great. Besides, suggestions of header shortening and/or 'references' is something that has come up quite a bit in TOML's history:
These are the few that I found from a quick skim of the issue list, though there's likely others. It being a repeat suggestion indicates there's an opportunity in this space. |
|
Dotted keys in KV pairs are explicitly discouraged in the documentation. Besides, the idea of a table shortcut is to keep the file DRY while still clear and structurally cohesive. The addition of table shortcuts would not take away the ability of repeating the full key in table headers. I'd still like to debate the value of using the |
|
My gripe against this feature is that the syntax is far from being obvious, which is the main goal of the language (it's even in its name). But, in particular, If I have never heard of TOML and I read an example of a TOML file (version 1.0), I can pretty much guess with good accuracy what kind of data structure is representing, and I can even modify it without having to read the spec. This is a huge advantage of TOML over, say, a complex YAML file. I think any feature that goes against this really need to be worth it. |
This is indeed a good point. Consider, though, that there's a number of features already in TOML with nuances that aren't necessarily immediately obvious:
If I were totally new to TOML and hadn't encountered analogous concepts in other contexts they'd be confusing to me. The "obviousness" here comes with a quick glance at the spec; given a few clear examples these things all click pretty quickly, the "ah-ha!" moment is never very hard-won. That's also what I like about this particular proposal over some of the others; it still maintains the core structure and simplicity of TOML, while also providing a simple sanity-saver as an almost-advanced feature for those who want it. A clear, simple example (like mine above) explains it pretty much in its entirety. Incidentally @brunoborges, that it's an almost advanced feature is why I think the additional idea of adding some numbered abbreviation notation to the thing ( |
|
@brunoborges I hate being that guy, but...
Where? Which documentation? I don't see that in the specification at all. There are notes about using inline tables sparingly, but there is no such advice for dotted keys.
Let's not confuse code with configuration. Clarity and cohesiveness are good principles for writing configuration files, but DRY is a coding principle. We're not necessarily introducing shortcuts to cut down on maintenance, nor to reduce the number of tests that would need to be written.
Personally I find the number to be confusing. It is possible to end with more than just a single specified key, and the number doesn't help to show the absolute depth of the keys, only how deep the unspecified keys go. An infix variant, like say Then again, if we were modeling a real-world configuration, would we need anything more succinct than, say,
@marzer A core guiding principle, nicely stated. |
|
@eksortso my apologies mate. I got confused with this segment in the toml.io website:
Reading through your examples, indeed the use of I was just stressing the idea as a mechanism to solve |
@brunoborges No worries, it's cool. Actually, we could probably use your insight in #781. We already see uses of TOML in the wild that go beyond config files, and deep nesting could have a definite purpose in the context of TOML and its future. I'm not sure how that will all shake out, but maybe you see some emerging trends. |
|
Hey friends, coming back to this with a new proposal: the diamond character as the placeholder. [servers]
[<>.alpha] # servers.alpha
ip = "10.0.0.1"
dc = "eqdc10"
[<>.beta] # servers.beta
ip = "10.0.0.2"
dc = "eqdc10"Actually, two characters: Diamonds are used for generic types in certain languages, where the type is inferred, so you don't need to repeat that. List<String> list = new Array list<>();The benefit here over Thoughts? |
|
I'd suggest for people discussing alternative placeholders to provide some argument for why it should be a good idea, instead of vague question. My argument to prefer |
|
@Kixunil fair point. I updated my comment with more reasoning. |
|
@brunoborges The diamond @Kixunil That's my argument for using the asterisk I'm hard-pressed to recall a precedent for something like this. All I can think of is, in |
Please no. I spend most of my day writing C++ template code; I could do with typing
I disagree. |
|
Also consider that a standard keyboard only has a finite set of bracket types:
Since two are used in TOML already, we only have two lives left. We should think very carefully about adding bracket-based syntax! Using a type of bracket on a construct that does not expect anything in between the opening and closing brackets, and thus treats them as a single semantic token, would be a waste. |
|
I don't think that Looking like generics doesn't make sense at all, since generics are about types, toml has values. Completely different things. Further generic syntax for languages differs. E.g. as someone who writes mostly Rust these days, Finally, note that |
|
@marzer you provide a good feedback regarding the limitation of characters that may be used for other features in the future, but are we really considering that once @Kixunil I'd appreciate if we left out comments such as your note out of the technical discussion. The suggestion of the diamond here was to provide another option to be considered and discussed, with technical merits, not to be a reason for subjective debates. Please continue with the |
That's not what I'm suggesting at all. Of course we could re-use brackets! TOML does this already for
OK, I've been with you up until this point, but come on man, that's BS. You asked for our thoughts, I gave mine. That makes them relevant by definition. Besides that, my point about C++ is extremely relevant from a language user-experience perspective. Have you written any C++? If not, ask your friendly neighbourhood C++ dev what it's like swimming through a sea of |
|
@brunoborges definitely let's focus on technical discussion. The cultural properties of various symbols can be used to decide in case there's no clear cut. There's a reason I put my note at the end, not beginning. :) |
|
Just had a thought about this, but it may be impractical. What do you all think? Ditto marks are used by people in lists as a shortcut for something previously mentioned. It's normally two quote marks put together. Here's a simple example. See how Doctor is dittoed. If we didn't allow zero-length strings for table names, we could have used But v1.0.0 is out already, and since banning empty-string table names would break the API, so to speak, then we could not use ditto marks in this way before a new major release, a v2 or higher, came out. |
|
On the idea of [*NN.subkey] notation I did want to point out that a glob style syntax might handle those cases (and some of the other cases) fairly well, while being intuitively similar to the Current [Importers.Images]
CreateMips = true
Compress = true
[*.*.Normals]
ColourSpace = 'TreatAsLinear'
AllowedChannels = 'rgb'
[*.*.HDR]
CreateMips = false
Compress = false
ColourSpace = 'TreatAsLinear'
[Animalia.Chordata.Mammalia.Carnivora.Feliformia.Felidae.Felinae.Felis]
[*.*.*.*.*.*.*.*.Felis_catus]
Name = "Domestic Cat"
[*.*.*.*.*.*.*.*.Felis_silvestris]
Name = "European Wildcat"
[*.*.*.*.*.*.*.*.Felis_chaus]
Name = "Jungle Cat"
# The following table names would be identical in context:
[Animalia.Chordata.Mammalia.Carnivora.Feliformia.Felidae.Felinae.Felis]
[*8.Felis_silvestris] # EIGHT?
[*6.Felinae.Felis.Felis_silvestris] # SIX?
[*2.Mammalia.Carnivora.Feliformia.Felidae.Felinae.Felis.Felis_silvestris] # TWO?Examples with [Importers.Images]
CreateMips = true
Compress = true
[**.Normals]
ColourSpace = 'TreatAsLinear'
AllowedChannels = 'rgb'
[**.HDR]
CreateMips = false
Compress = false
ColourSpace = 'TreatAsLinear'
[Animalia.Chordata.Mammalia.Carnivora.Feliformia.Felidae.Felinae.Felis]
[**.Felis_catus]
Name = "Domestic Cat"
[**.Felis_silvestris]
Name = "European Wildcat"
[**.Felis_chaus]
Name = "Jungle Cat"
# The following table names would be identical in context:
[Animalia.Chordata.Mammalia.Carnivora.Feliformia.Felidae.Felinae.Felis]
[**.Felis_silvestris] # EIGHT?
# [*6.Felinae.Felis.Felis_silvestris] # -- Not possible with globbing --
# [*2.Mammalia.Carnivora.Feliformia.Felidae.Felinae.Felis.Felis_silvestris] # -- Not possible with globbing -- |
|
Closing this as per discussion in #781 (closing note there: #781 (comment)) -- I don't think this approach works well for various kinds of nested structures and going down a JSON-like approach for nesting would be better. |
Consider the following exampe:
Idea: add a shortcut character to reference the outer table.
Example:
The character
#is illustrative and may have to be reconsidered giving it's used already for comments.The proposal also considers multiple levels:
The multi-level indicator, e.g.
#.#.#..., must match the current structure. For example,#.#.#.#.fooright below a single-level table[bar]must be considered invalid. IDEs and plugins for text editors should be able to provide tooltip to indicate the expanded name of the referenced tables, expanding the value.The benefit of this feature is to simplify typing while keeping explicitly defined the level of the table, and also in reducing the chances of typos when nesting tables. Can also facilitate when copying/pasting snippets.
The text was updated successfully, but these errors were encountered: