Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add tests of non-deterministic behaviour #9

Open
glyn opened this issue Jul 5, 2022 · 1 comment
Open

Add tests of non-deterministic behaviour #9

glyn opened this issue Jul 5, 2022 · 1 comment

Comments

@glyn
Copy link
Contributor

glyn commented Jul 5, 2022

Tests are needed to cover the lack of deterministic ordering of objects. The preferred approach is for such tests to include a set of possible results, any one of which is valid.

glyn added a commit to glyn/jsonpath-compliance-test-suite that referenced this issue Feb 29, 2024
@glyn
Copy link
Contributor Author

glyn commented Mar 1, 2024

(The following is a summary of a discussion in PR #60.)

@f3ath wrote (with terminology adjusted to align with RFC 9535):

Have we considered adding a flag (e.g. "non-deterministic": true) to allow arbitrary order of the result array instead of trying to list it in every possible order?

As @gregsdennis pointed out in PR 60, the RFC does not allow arbitrary non-determinism in the ordering of the nodelist resulting from a query. (The Working Group discussed increasing the non-determinism in that way, but decided against it.)

Table 6, table 12, and table 16 in the spec include examples where there is some non-determinism without there being arbitrary non-determinism. The limited non-determinism is typically produced by a combination of objects (which are unordered) and arrays (which are ordered) in the query argument.

I might not understand the idea fully, but if we want to allow non-deterministic order somewhere deep inside of the complex result object, we could maybe introduce a list of json pointers pointing to the particular elements of the result which are allowed to be non-deterministic.

I considered that possibility, but it's hard to specify simply and each implementation that wants to run the CTS would need to implement it correctly. The approach I chose in PR #60 is for non-deterministic tests to list all the allowable result arrays, so implementations simply need to check that the result array is in the list. The burden then falls on the writers of the CTS to correctly list all the allowable result arrays, which isn't too difficult for small examples.

(Another potential approach is to generalise an implementation of the RFC so that it produces a list of all the allowable nodelists and then to use that to prime the CTS. This could be done in Haskell, for example, by using the "list monad", but it wouldn't be trivial.)

/cc @cabo

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant