Skip to content
This repository has been archived by the owner on Dec 1, 2022. It is now read-only.

Support lookup indexScan using IN expression as filter #1278

Closed
wants to merge 6 commits into from

Conversation

Aiee
Copy link
Contributor

@Aiee Aiee commented Jul 21, 2021

  1. Fix constant fold for container expressions
  2. Convert IN expression to OR expression in LOOKUP if the property in theIN expression has a valid index:
    • Stand alone IN expr: A in [B, C, D] => (A == B) or (A == C) or (A == D)
    • With AND expr : A in [a, b] AND B => (A==a AND B) OR (A==b AND B)
    • With OR expr: A in [a, b] OR B => A==a OR A==b OR B
  3. If the container has only one element, the IN expression will be transformed to a relEQ expression. e.g. A in [B] => A==B

@Aiee Aiee added the depend on common PR: this PR depends on PRs in the common repo label Jul 21, 2021
@Aiee Aiee added the wip Solution: work in progress label Jul 21, 2021
@Aiee Aiee force-pushed the rewrite-in-expr branch from 6bf7e98 to 91dd0f6 Compare July 22, 2021 13:31
@Aiee Aiee added ready-for-testing PR: ready for the CI test and removed wip Solution: work in progress labels Jul 22, 2021
| id | name | dependencies | operator info |
| 3 | Project | 2 | |
| 2 | Filter | 4 | {"condition": "(((player.age==40) OR (player.age==25)) AND ((player.name==\"ABC\") OR (player.name==\"Kobe Bryant\")))"} |
| 4 | TagIndexFullScan | 0 | |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the actual operator info of TagIndexFullScan? Could you paste it here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TagIndexFullScan scans the whole tag, so the "columnHints" is empty.

|  4 | TagIndexFullScan | 0            | ver: 0, rows: 56, execTime: 0us, totalTime: 9707us | outputVar: [                                                                                                   |
|    |                  |              |                                                    |   {                                                                                                            |
|    |                  |              |                                                    |     "colNames": [                                                                                              |
|    |                  |              |                                                    |       "VertexID",                                                                                              |
|    |                  |              |                                                    |       "player.age",                                                                                            |
|    |                  |              |                                                    |       "player.name"                                                                                            |
|    |                  |              |                                                    |     ],                                                                                                         |
|    |                  |              |                                                    |     "type": "DATASET",                                                                                         |
|    |                  |              |                                                    |     "name": "__TagIndexFullScan_1"                                                                             |
|    |                  |              |                                                    |   }                                                                                                            |
|    |                  |              |                                                    | ]                                                                                                              |
|    |                  |              |                                                    | inputVar:                                                                                                      |
|    |                  |              |                                                    | space: 1                                                                                                       |
|    |                  |              |                                                    | dedup: false                                                                                                   |
|    |                  |              |                                                    | limit: 9223372036854775807                                                                                     |
|    |                  |              |                                                    | filter:                                                                                                        |
|    |                  |              |                                                    | orderBy: []                                                                                                    |
|    |                  |              |                                                    | schemaId: 2                                                                                                    |
|    |                  |              |                                                    | isEdge: false                                                                                                  |
|    |                  |              |                                                    | returnCols: [                                                                                                  |
|    |                  |              |                                                    |   "_vid",                                                                                                      |
|    |                  |              |                                                    |   "age",                                                                                                       |
|    |                  |              |                                                    |   "name"                                                                                                       |
|    |                  |              |                                                    | ]                                                                                                              |
|    |                  |              |                                                    | indexCtx: [                                                                                                    |
|    |                  |              |                                                    |   {                                                                                                            |
|    |                  |              |                                                    |     "columnHints": [],                                                                                         |
|    |                  |              |                                                    |     "filter": "",                                                                                              |
|    |                  |              |                                                    |     "index_id": 8                                                                                              |
|    |                  |              |                                                    |   }                                                                                                            |
|    |                  |              |                                                    | ]                                                                                                              |

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So why does the condition of filter not be pushed (or be partitially pushed) into IndexScan?

@Aiee Aiee added the wip Solution: work in progress label Jul 23, 2021
@Aiee Aiee force-pushed the rewrite-in-expr branch from 24acf65 to 1542f72 Compare July 30, 2021 13:23
@Aiee
Copy link
Contributor Author

Aiee commented Sep 15, 2021

This PR will be moved to vesoft-nebula repo.

@Aiee
Copy link
Contributor Author

Aiee commented Sep 18, 2021

This PR has been transferred to vesoft-inc/nebula#2906

@Aiee Aiee closed this Sep 18, 2021
@Aiee Aiee deleted the rewrite-in-expr branch September 18, 2021 17:23
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
depend on common PR: this PR depends on PRs in the common repo ready-for-testing PR: ready for the CI test wip Solution: work in progress
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants