[Enhancement]: Optimize expression parsing efficiency

[xiaocai2333]

Is there an existing issue for this?

• I have searched the existing issues

What would you like to be added?

Optimize expression parsing efficiency.

1. Optimize the time of parsing Chinese characters.
2. Pass in the expression parameters to be queried through the template.

• such as expr=pk > {age} and city in {city}, search_params={age:3, city:['beijing","shanghai"]}

Why is this needed?

No response

Anything else?

No response

[xiaofan-luan]

Excited about the template feature!

[zhengbuqian]

we now support nullable fields, how do we handle null in template?

[xiaocai2333]

we now support nullable fields, how do we handle null in template?

Currently, our expression does not support null values. We can discuss it further when null support is added in the future.
/cc @smellthemoon

[xiaofan-luan]

null support is already added?

[xiaofan-luan]

did you mean isNull and isNotNull?
can we do a == null?

[zhengbuqian]

did you mean isNull and isNotNull?

yes. we should be able to filter based on null. chatted with @smellthemoon offline, we don't currently support this.

Maybe we should not allow null in template, we can simply use a NULL keyword:

• to use null only: a == NULL or a != NULL
• to use null with non null values: a == 100 || a == NULL
• to use null with template: a in [{accepted_values}] || a == NULL.
  • do not allow null in template values, it has to be provided directly in the expr

[smellthemoon]

did you mean isNull and isNotNull?

yes. we should be able to filter based on null. chatted with @smellthemoon offline, we don't currently support this.

Maybe we should not allow null in template, we can simply use a NULL keyword:

• to use null only: a == NULL or a != NULL
• to use null with non null values: a == 100 || a == NULL
• to use null with template: a in [{accepted_values}] || a == NULL

In fact, in the future, we will still not support expressions like == null and != null, which will directly report errors at the parse layer. Exposing the NULL keyword will bring a lot of complexity. It needs to be identified in all expressions, and the respective logical relationships need to be considered in complex expressions. Therefore, in the future, we will only support the form of isNull(field) and isNotNull(field).

[zhengbuqian]

did you mean isNull and isNotNull?

yes. we should be able to filter based on null. chatted with @smellthemoon offline, we don't currently support this.
Maybe we should not allow null in template, we can simply use a NULL keyword:

• to use null only: a == NULL or a != NULL
• to use null with non null values: a == 100 || a == NULL
• to use null with template: a in [{accepted_values}] || a == NULL

In fact, in the future, we will still not support expressions like == null and != null, which will directly report errors at the parse layer. Exposing the NULL keyword will bring a lot of complexity. It needs to be identified in all expressions, and the respective logical relationships need to be considered in complex expressions. Therefore, in the future, we will only support the form of isNull(field) and isNotNull(field).

that is fine. as long as we have a way of checking null, we can remove the need to support null in expression template.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.