Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unexpected results when using expressions on keys with non-uniform fields #754

Closed
unendingblue opened this issue Nov 3, 2024 · 2 comments

Comments

@unendingblue
Copy link

Describe the bug
Search queries that utilize filter expressions return incorrect results if some keys have fields that others do not.

From my testing, it appears that if an expression references a field name that not all keys have, the keys that do not have it are completely omitted from the results, even if the expression contains a null-check condition.

Although I believe this is related to https://github.com/tidwall/expr, I am filing the issue here as it is more relevant to Tile38.

To Reproduce
SET USER 1 FIELD teamId 1 FIELD optionalId 999 POINT 0 0
SET USER 2 FIELD teamId 1 POINT 0 0
SCAN USER

Observe that both keys are returned.

SCAN USER WHEREIN teamId 1 1

Observe that both keys are returned.

SCAN USER WHEREIN teamId 1 1 WHERE '!optionalId || optionalId == 999'
SCAN USER WHEREIN teamId 1 1 WHERE '!!!optionalId || optionalId == 999'
SCAN USER WHEREIN teamId 1 1 WHERE 'optionalId == 0 || optionalId == 999'
SCAN USER WHEREIN teamId 1 1 WHERE '1 == 1 || optionalId == 999'

Observe that in all cases, only USER 1 is returned.
Also notice that even though 1 == 1 is always true, since USER 2 does not have the optionalId field, it is omitted from the results.

Expected behavior

  1. Expressions should evaluate !<field> (or <field> == 0) to true, if the field is not present. This behavior already exists in non-expression queries, for example: SCAN USER WHEREIN teamId 1 1 WHEREIN optionalId 2 0 999
  2. Keys should not be omitted from the results if they do not contain the fields referenced in the filter expression.
@tidwall tidwall closed this as completed in aa1caa6 Nov 5, 2024
@tidwall
Copy link
Owner

tidwall commented Nov 5, 2024

I just pushed a fix. It seems that Tile38 was treating non-existent fields as undefined. This is how the expr library and javascript works, but Tile38 should be treating them as zeros instead.

@unendingblue
Copy link
Author

Thank you! Can confirm it now works as expected in 1.33.4.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants