jcfg

jcfg is an easy-to-use json-based configuration tool for building python programs.

Suppose that you want to develop an program that has three config options: a file path (type of string), an integer (type of int), and a float number. And also want to be able to load configs from a local file. Then, you can write code like this, with jcfg:

# in file: my_program.py
from jcfg import JsonCfg

def main():
    cfg = JsonCfg({
        'input_path': '',
        'input_int': 0,
        'input_float': 0.0
    })
    cfg.parse_args()

    # do any work you want below
    # you can get the input_path, input_int, or input_float like this:

    res = cfg.input_int + cfg.input_float
    # `cfg.input_int` and `cfg.input_float` will get the actual value depending on the cli input. jcfg have done the type check work for you. If the input option cannot be parsed as an integer, it will raise error

    ....

if __name__ == '__main__':
    main()
    

Type python3 my_program.py -h in your terminal, you will get:

usage: test_readme.py [-h] [-c CONFIG_PATH] [--input_float INPUT_FLOAT]
                      [--input_int INPUT_INT] [--input_path INPUT_PATH]

optional arguments:
  -h, --help            show this help message and exit
  -c CONFIG_PATH        file path to update config
  --input_float INPUT_FLOAT
                        type: <class 'float'>, default: 0.0
  --input_int INPUT_INT
                        type: <class 'int'>, default: 0
  --input_path INPUT_PATH
                        type: <class 'str'>, default:

As you can see above, these config options can be loaded from either cli input or a json file with cli option -c your_config.json.

Usage

To define config options

The config options should be defined based on the dict object provided when constructing JsonCfg. Each key in the dict will be the name of an config option, and the corresponding value will be either the default value for this config or a sub-config under this key name, based on:

1. if the value is a dict without the key _default, this value will define a sub-config under this key,
2. otherwise, the value indicates the default value (and also some other meta info) for this key. The type of this config will be implicitly determined as the type of the default value.

For better understanding, here is an example:

cfg = jcfg.JsonCfg({
    'option_int': -1,  # this define an config option with name `option_int` with default value of -1, and also with type of int (type(-1) is int).
    'option_float': 0.0,  # this define config `option_float`, with default value 0.0, and type of float.
    'option_str': 'default_value',  # default value: 'default_value', with type str
    'option_list': [],  # this defins an config with default value of empty list, of type list
    'sub_config': {  # this will define and sub-config
        'sub_int': 0,
        'sub_float': 0.0,
        'sub_sub_config': {  # sub-config can be recursively defined
            'leaf_key': 0, 
        }
    },
    'not_sub_config': {  # however, this won't define a sub-config, but an config with value type: string
        '_default': 'this_is_default_value', 
        '_desc': 'this_si_description',  # this will provide detailed description for this config when typing --help
    },
    'option_by_tuple': (2, 'this is a description'),  # this option is defined with a 2-size tuple, the 1st is default value, 2rd is description
    'option_by_tuple_3': (0.0, 'this is a description', lambda x: x>=0.0),  # this option is defined with a 3-size tuple, the 3rd is to validate the value.
})

The config option value currently only supports following types:

• int
• float
• str
• list

Note that, if you want a config option with type of dict, you'd better define it as an 2-size list, then parse this option as dict in your code yourself.

To load configs from cli

All config options could be overrided from cli with the corresponding config key name. For sub-config, the config key name is defined by all the config key name between root config and leaf config. Here is an example to override the sub-config value:

python3 test.py --sub_config.sub_int 2

To load configs from file

All configs can also be loaded from a json file with a more tolerant json loader: jstyleson, which supports js-style comments and also allows trailing comma, or from yaml file. Here is an examples of loading from json:

\\ config.json
/* js-style comments supported!
*/
{
    "option_str": "new_string",  // in-line comments supported!
    "sub_config":{
        "sub_int": 1,   // trailing comma allowed
    },
    "sub_config.sub_float": 3
}

These config can be updated by python3 test.py -c config.json.

Note that, sub-config value can be defined as a dict or key-value pair with full config path, like above.

If both config file and cli option are provided, the config will be first overrided from config file, then overrided from cli options.

Access to configs.

The configs can be easily accessed like this:

cfg.option_int   # or 
cfg['option_int']

# for sub-config
cfg.sub_config.sub_int  # or
cfg['sub_config']['sub_int']  # or
cfg['sub_config.sub_int']

Other features

• Config key startswith _ denotes private config options, which will never be overrided from cli or file.
• '_default' is the reserved key, you should never define a config with key name '_default'.
• '_desc' is another reserved key for defining cli description of config key
• Some other reserved key startswith _ may be added someday, to provide more advanced meta control.

Test

Run cd test; python3 test_main.py -v.