Engineering/scientific notation is incorrectly deserialized as string

[bjtho08]

I want to use confuse for configuring applications that use TensorFlow/Keras for deep learning. Part of that is reading various numerical values from the config file. For most types, this is painless, but I encountered the following issue.

Consider a config.yaml that looks like this:

appName: MyMLApp

...

arch:
  lr: 1e-4

In python, the following will happen:

import confuse

config = confuse.Configuration('MyMLApp', __name__)
model_params = config['arch'].get()
assert isinstance(model_params['lr'], float)

----
Traceback (most recent call last):
  File "<pyshell#66>", line 5, in <module>
    assert  isinstance(model_params['lr'], float)
AssertionError

The expected behavior is of course that this assertion does not fail.

OS and info:
Ubuntu 16.04
Python 3.8.2
confuse 1.1.0

[sampsyo]

Hello! Sounds interesting—but this is a YAML specification or PyYAML issue, rather than a Confuse issue.

To start investigating, let's use PyYAML directly:

>>> import yaml
>>> type(yaml.safe_load("123"))
<class 'int'>
>>> type(yaml.safe_load("1e4"))
<class 'str'>

So, yep, there's the problem. To dig deeper, maybe let's look at the YAML spec for floating point literals:
https://yaml.org/spec/1.2/spec.html#id2804092

According to that regex, it seems like the + or - is required. So maybe this works?

>>> type(yaml.safe_load("1e+4"))
<class 'str'>

Sadly, no. A little googling leads to yaml/pyyaml#173 indicating that this is a bug. PyYAML needs a dot in the base part:

>>> type(yaml.safe_load("1.0e+4"))
<class 'float'>

That works! So as a workaround, I suggest using that literal format. According to the aforelinked issue thread, however, ruamel.yaml fixes this bug. So Confuse could "fix" it to by doing #52, for which I would be eternally grateful for a pull request! 😇

[bjtho08]

Thanks for the reply! I never considered that it needed to be that explicit, but in any case this is much less of a hassle than to write out the numbers as a decimal, so I will accept it as closed for now 👍 If I get the chance later, I will look into the fix that ruamel.yaml uses and see if I can implement something similar in confuse :)