Proposal for encoding type information exclusively in member names of objects (using postfix tags)

[tarcieri]

The main bit of interesting feedback I've received so far is the idea of moving types to the keys of objects only. Another recurring idea has been moving them to a postfix tag on the key, rather than a prefix tag. See #28.

I think these ideas have merit, and would also be useful in resolving #23 (Homogeneous types for non-scalar elements/members) and #22 (Data type for Sets?). It also has the aesthetically pleasing property of getting the type information

Below is a concrete proposal to move all type information into object keys:

Objects as the source of types

Presently the root symbol of TJSON's grammar allows both objects and arrays as top-level nonterminals (2.2).

If we wish to use object keys as the exclusive source of type information, we can only allow an object as the top-level nonterminal. This allows objects to encode the type information for arrays (and potentially other non-scalar types encoded as arrays like sets #22). This approach effectively treats them as a self-describing record-like product type.

This has implications for redaction (#21): we would be limited to redacting a member-at-a-time (i.e. redacting a name/value pair, rather than being able to independently redact member names), and would also limit us to using strings as the names of members. I think this is probably ok.

This permits the sort of syntax proposed in #28. An example using only scalars:

{
    "a-number:i": "42",
    "a-string:s": "Hello, world!"
}

Type signatures for non-object non-scalar types

For non-scalars, we need tags that fully describe the type. Objects under this scheme are self-describing product types, but since arrays (and potentially sets #22) receive no type information (at least under this proposal), it needs to be specified in objects themselves.

The following names are proposed for identifying non-scalars:

• a: array
• o: object
• set: set (provisional, see Sets #22)

Alternatively, we could give non-scalars upper case names, which frees up the namespace a bit:

• A: array
• O: object
• S: set

Since these are non-scalar types, they're collections over some other type. We can add additional syntax to specify what type. Objects are a special case, as they are the only type which encodes type information, and are therefore self-describing.

For arrays (and potentially sets) we can use the following syntax, using a as a proposed type annotation for arrays and o for objects, as the signature for an array of objects:

• a<o> or A<O>: typical "generic" syntax used in several languages (my personal preference)
• a(o) or A(O): a lispier alternative
• ao or AO: a single scalar on the right, with nested non-scalars on the left. This syntax only works if every type is identified by a single letter (e.g. ab64 could work, sets does not). Potentially we could disambiguate scalars from non-scalars by case, e.g. S is set, s is string.

Here's an example with an array of integers using the above proposed syntax:

{
    "an-int-array:A<i>": ["1", "2", "3"]
}

Per #22, we could use the following for a set:

{
    "a-set-of-ints:S<i>": ["1", "2", "3"]
}

This syntax could also be used to describe nested arrays:

{
   "a-nested-int-array:A<A<i>>": [["1", "2", "3"], ["4", "5", "6"], ["7", "8", "9"]]
}

Or an array of sets (#22):

{
   "a-nested-int-array:A<S<i>>": [["1", "2", "3"], ["4", "5", "6"], ["7", "8", "9"]]
}

Since objects describe their own types, we don't need to provide additional type information for them:

{
    "array-of-objects:A<O>": [
        {"foo:i": "1"},
        {"bar:i": "2"},
        {"baz:i": "3"},
    ]
}

[wbolster]

what exactly is the point of stating in the key of a key/value pair that the value is (supposed to be) an object? that information is already available from the json structure after parsing it.

[OJFord]

@wbolster I believe the proposal is only that it would be e.g. when an array of objects :A<O>, the alternative would I suppose be :A or :A<>, but I think explicit is better here - one might equally interpret that as an array of floats otherwise.

I think this looks great, and was more what I expected reading the headline. I'm not sure I understand the reasons behind doing it in the value to begin with? (i.e. is there something lost by doing this?)

[tarcieri]

@wbolster to include such postfix tags they have to be syntactically unambiguous (i.e. for domain separation purposes, versus the rest of the contents of strings). They could potentially be left empty in the case of an object (e.g. keyname:), but is that really better than keyname:O?

[tarcieri]

I'm not sure I understand the reasons behind doing it in the value to begin with? (i.e. is there something lost by doing this?)

@OJFord yes, you lose the ability to have toplevel arrays, since they can't carry their own type information

[tarcieri]

Another reason to use an O syntax for objects: it keeps the door open for using objects to represent other data types in the same way A allows the array syntax to be used for sets with S.

[tarcieri]

I've begun work on converting the spec to use this syntax in #35

[tarcieri]

I've done a first pass on redoing the specification, but I'll need to add additional information about handling of non-scalar types.