-
Notifications
You must be signed in to change notification settings - Fork 21
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support for anyOf in property type declarations #86
Comments
@visch - I took the liberty of expanding the code snippet and adding notations for the types of It looks like all of these implementations are for the purpose of making the type "committer_email":{
"type":[
"null",
"string"
]
}, but not "authored_date":{
"anyOf":[ // <<<< nullable datetime
{
"type":"string",
"format":"date-time"
},
{
"type":"null"
}
]
}, Adding this context because nullability is much simpler to implement than something like allowing a list or a string, or a datetime or an int. Those types of |
@visch - One other question here: do you know if we currently fail gracefully (such as falling back to |
I believe it fails today with a KeyError due to https://github.com/MeltanoLabs/target-postgres/blob/main/target_postgres/connector.py#L89 yes it does, in that slack thread there's a stack trace:
In the process I made some bugs it seems! Need to add this to the test suite :O. We could try hard to test that we can push in types we haven't accounted for but I've had some issues with sqlalchemy -> pyodbc has there's sometimes typing issues specifically with JSON Arrays which makes that want tricky. Maybe there's a way to make it work in a general case. |
@visch I ran into this today. I got a KeyError for tap-gitlab's project schema using anyOf. It looks like its caused by
If I dig deeper and I compare to pipelinewise (which worked for my use case) I see that it uses a flatten schema method to cleanup the schema before parsing it in the same way that we're parsing here where |
@visch I'm not positive so you'd need to test to validate it but this might be fixed by meltano/sdk#1774. |
That didn't work unfortunately :/ |
A less-involved fix for anyOf processing that treats anyOf as if it was a `type` array. Closes #86 --------- Co-authored-by: Derek Visch <visch@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Need a test for this (Check the Standard Target tests in the SDK to see if we have one)
https://meltano.slack.com/archives/C01TCRBBJD7/p1673947888707009?thread_ts=1673874691.323399&cid=C01TCRBBJD7
Doesn't seem to be working.
Same as above but in a testable format
The text was updated successfully, but these errors were encountered: