Issues with floats

[djdevin]

Having issues replicating a field that is a float:

Traceback (most recent call last):
  File "/usr/local/bin/target-postgres", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.6/site-packages/target_postgres/__init__.py", line 138, in main
    state = persist_lines(config, input)
  File "/usr/local/lib/python3.6/site-packages/target_postgres/__init__.py", line 68, in persist_lines
    validators[stream].validate(o['record'])
  File "/usr/local/lib/python3.6/site-packages/jsonschema/validators.py", line 130, in validate
    raise error
jsonschema.exceptions.ValidationError: 0.57 is not a multiple of 0.01

Failed validating 'multipleOf' in schema['properties']['credits']:
    {'inclusion': 'available',
     'multipleOf': 0.01,
     'type': ['null', 'number']}

On instance['credits']:
    0.57

It looks like this is the issue here: python-jsonschema/jsonschema#247

Not sure if it's an issue that can be fixed here or somewhere else. Mathematically speaking I think it's accurate judging by that issue, but I'm not really sure what to do.

Here's a sample of the data from the tap:

{
  "type":"RECORD",
  "stream":"course_credit_awarded",
  "record":{
    "credits":0.57,
    "status":1,
    "created":"1970-01-01T00:00:00+00:00",
    "vid":1797,
    "expiration":null,
    "type":"case_manager",
    "date":"2012-07-11T19:48:55+00:00",
    "nid":200,
    "uid":1201,
    "ccaid":1895,
    "changed":"1970-01-01T00:00:00+00:00",
    "author_uid":null
  },
  "version":1538760868289,
  "time_extracted":"2018-10-05T17:34:28.297167Z"
}

[koszti]

@djdevin give it a try with the PR above

[djdevin]

Thanks, with the PR I get

ImportError: cannot import name 'FormatChecker'

[djdevin]

May also be related (error also without patch)

jsonschema.exceptions.ValidationError: 39.948314 is not a multiple of 1e-06

[djdevin]

I had to change the import to

from jsonschema import validators, Draft4Validator, FormatChecker
from decimal import Decimal

I have no experience with python (3 in this case) but it seems to work and it's importing data now.

[koszti]

thanks @djdevin the imports were not correct as you detailed. I added this into the original PR #11 . Hopefully somebody will get it merged soon.