initial run at creating output tests #619
Conversation
Co-authored-by: Julian Berman <[email protected]>
Co-authored-by: Julian Berman <[email protected]>
* origin/main: remove schemas that are never referenced move anchors into defs move tests to draft-next added type:object to contentSchema schemas add tests for $dynamicAnchor in multiple branches of propertyDependencies add tests for unevaluatedProperties seeing inside propertyDependencies Add RJP test 'multi-digit integer prefix' Remove unneeded remotes
Output tests aren't actually overlapping with the normal test schema, they don't contain the 'valid' property (which is required in normal tests).
It doesn't contain tests, so it's indeed not valid under the output schema.
|
I've added I also implemented this new suite in my lib (PR linked ☝️) and I'm happy to report that I fail the 2019 & 2020 cases! 🤦 |
|
I now pass most of the 2019/2020 cases. There's one test in each that I can't pass because my approach for |
And fix skipping regex format validation and the case where dynamicRef blows up. (The latter hopefully being temporary, but just so the PR passes)
|
I think we should likely merge these! I've only given the content of the tests themselves a few skims but they seemed nice as a starting point, may as well get them in and iterate I think, but would possibly give slightly more visibility if they were merged in even if we needed to make a tweak or two once others tried running them. So yeah thumbs up from me for merging when you're comfortable, or if you're planning on hearing from others obviously fine with me too. |
|
I'm fine with merging as a starting point. Forgive me for not re-reading the readme changes, but does it say somewhere that people shouldn't rely on these test yet because they are still being developed and may contain errors? |
|
Probably a very good idea especially considering the last conversation. |
Begins to address #247
This is a go at creating some output tests. It includes 2019-09/2020-12 and draft-next tests for:
typecreates a proper node, and may include a message if it failsreadOnlygenerates an annotation of its valueHat's off to @karenetheridge who proposed using a schema to validate the output. This creates a really nice way to target the bit of the output that a test is focused on without requiring an explicit error message or being reliant on specific output unit sequencing.
I still have structural tests planned, but I figured this would be enough to start.
I've also included a README for the folder.