[Features] Generate a json schema from LightningCLI

[julien-blanchon]

Description & Motivation

Idea

As the LightningArgumentParser did a very good job solving the possible parameters (see the --help output) with name, description, default value and type, it could be good add a command to export a custom json schema for the --config.

A json/yaml schema could be useful to:

• Validate that a config file is correct before symbolically before running
• Help the user writing config file with on IDE description, suggestion and validation

This is particularly useful in a Lightning context because the LightningArgumentParser did solve the possible model, optimizer, lr_scheduler ... and eventually custom classes such as transform. Using the project wide available classes. Having to write classes references without validation could be a pain and intelligent suggestion is always welcome.

Implementation idea

The easiest way to export a json schema is to build a Pydantic Model of the config and to use the pydantic .schema_json method.
So one idea could be to transform the cli.parser._actions object to a Pydantic Model

Pitch

No response

Alternatives

No response

Additional context

Here is an example of an IDE integration of json/yaml schema in VSCode: https://marketplace.visualstudio.com/items?itemName=redhat.vscode-yaml

We just need to specify # yaml-language-server: $schema=<urlToTheSchema> to enable schema completion and validation in the IDE.

Validation could also be made as a pre-commit hook

cc @Borda @carmocca @mauvilsa

[carmocca]

This proposal would fit better in the https://github.com/omni-us/jsonargparse repository, as there's nothing LightningCLI specific. jsonargparse is what powers the LightningCLI under the hood

[mauvilsa]

This is an interesting idea. I will think about it.

Regarding the implementation. The actions will not be changed to pydantic models. jsonargparse is an extension of argparse and features depend on this. For example changing this would break the compatibility with argcomplete and shtab.

[mauvilsa]

A feature request has been created in jsonargparse, see omni-us/jsonargparse#618. @julien-blanchon can you please close this one, since it fits better there.

[julien-blanchon]

This is awesome, thanks for pointing this out @mauvilsa