-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add tests of non-deterministic behaviour #9
Comments
(The following is a summary of a discussion in PR #60.) @f3ath wrote (with terminology adjusted to align with RFC 9535):
As @gregsdennis pointed out in PR 60, the RFC does not allow arbitrary non-determinism in the ordering of the nodelist resulting from a query. (The Working Group discussed increasing the non-determinism in that way, but decided against it.) Table 6, table 12, and table 16 in the spec include examples where there is some non-determinism without there being arbitrary non-determinism. The limited non-determinism is typically produced by a combination of objects (which are unordered) and arrays (which are ordered) in the query argument.
I considered that possibility, but it's hard to specify simply and each implementation that wants to run the CTS would need to implement it correctly. The approach I chose in PR #60 is for non-deterministic tests to list all the allowable result arrays, so implementations simply need to check that the result array is in the list. The burden then falls on the writers of the CTS to correctly list all the allowable result arrays, which isn't too difficult for small examples. (Another potential approach is to generalise an implementation of the RFC so that it produces a list of all the allowable nodelists and then to use that to prime the CTS. This could be done in Haskell, for example, by using the "list monad", but it wouldn't be trivial.) /cc @cabo |
Tests are needed to cover the lack of deterministic ordering of objects. The preferred approach is for such tests to include a set of possible results, any one of which is valid.
The text was updated successfully, but these errors were encountered: