JSON serialization and schema generation

[jlstevens]

This PR outlines another attempt at serializing/deserializing to JSON (for reference see
#153 for earlier attempts at tackling this problem). The design used in this PR aims to try to keep things simpler but also adds the ability to generate JSON Schemas which should help with testing as well as specification of allowable parameter values (e.g this could be used to specify suitable widgets in panel).

This PR is based off the ideas suggested in #382.

TODO:

• The main open problem is about finding the appropriate API and JSON (and schema) representation for entire parameterized objects (i.e including the class identity as well as all the parameters).

[coveralls]

Coverage decreased (-0.005%) to 76.447% when pulling 8c1adb9 on json_serialization into a8e34b2 on master.

[philippjfr]

So after a brief look I guess what I expected to see here is the serialize/deserialize and maybe schema implementations to live on the actual Parameters rather than the serialization class so that the approach can be easily extended. I'm happy to review further but that seemed like a pretty core part of the design we were talking about before so I'd like to hear about your reasoning here before diving deeper.

[jbednar]

Philipp, Jean-Luc, and I just met to discuss this approach, with some notes:

• swapping serializers vs. adding Parameters -- which is more common?

  • adding Parameters seems more important

• serialize to string immediately (as now), or with an intermediate Python representation (as for bokeh properties)?

  • Python IR:
    • Define a base set of supported Python types (int, float, bool, string, datetime64)
    • Build up a Python-based representation
    • Individual Parameters only know how to work with the IR
    • Specific serializers know how to map from IR<->textfile (JSON, XML, scriptrepr, etc.)

• deserialization?

  1. result is a bag of Parameters, right now?
  2. result is an instance of a generic Parameterized (no Parameters initially, just has whatever was deserialized, but otherwise can be treated like any other Parameterized, e.g. for pn.pane.Param purposes.
  3. deserializing to a specific Parameterized, setting its parameters
  4. creating an arbitrary collection of nested Parameterized objects de novo

Actual IR->JSON should presumably be done using a JSONEncoder subclass (https://docs.python.org/3/library/json.html#json.JSONEncoder).

[philippjfr]

Python IR Support

Just wanted to start making a list of types that I think the IR and therefore every serializer should (eventually) support.

• boolean
• int
• float
• str
• complex
• bytes
• list
• dict
• np.datetime64
• datetime.datetime
• datetime.date
• np.array
• pd.DataFrame

Feel free to edit and expand.

[jlstevens]

I agree with the above list. It is also keeping #382 as a reminder of what should be possible with JSON serialization (which I suspect will be the baseline for all serialization formats).

[philippjfr]

Can you also make sure you handle ObjectSelector/Selector with enums in your initial PR?

[philippjfr]

Also encode the label attribute as the 'title' property in the schema.

[philippjfr]

Oh also ListSelector should be {'type': 'array', 'items': {'enum': [...]}}

[jlstevens]

All good suggestions, thanks! I won't implement the regexp attribute for strings though as there is no easy/meaningful way to try to convert Python style regular expressions to equivalent javascript.

[jlstevens]

Still more work to be done, but the suggested refactor and all of Philipp's suggestions are now implemented.

[jlstevens]

@jbednar @philippjfr Coverage is very slightly down (-0.005%) but otherwise tests are passing. Other than the ir_serialize versus serialize renaming (and the same for deserialize), I think this might be a good place to merge. The subsequent PR I am planning would address serialization of entire parameterized objects but I think this first step of serializing parameters (with schemas) is already valuable.

[jlstevens]

@philippjfr @jbednar I think this PR is now in a useful state to merge. I'll go ahead and do that shortly unless you have any other suggestions or objections..

[philippjfr]

Looks good. Merge away.