Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: use UNNEST to bypass parameter limit #226

Merged
merged 5 commits into from
Sep 4, 2024

Conversation

taka-oyama
Copy link
Collaborator

@taka-oyama taka-oyama commented Sep 3, 2024

Tried running WHERE IN UNNEST with 1000 parameters and here is the result.

Screenshot 2024-09-03 at 16 59 13

Index scan is used and everything seems to be working fine.

Summary by CodeRabbit

  • New Features

    • Introduced a new method to the query builder for excluding values in complex queries.
    • Enhanced the existing method to automatically handle large sets of values for improved performance.
  • Documentation

    • Updated documentation to clarify handling of query parameters exceeding limits and introduced a configuration option for users.
  • Tests

    • Added new tests to ensure correct functionality of the query builder under different configurations related to parameter limits.

@taka-oyama taka-oyama added the enhancement New feature or request label Sep 3, 2024
@taka-oyama taka-oyama requested review from t-matsuno-777 and a team September 3, 2024 09:06
@taka-oyama taka-oyama self-assigned this Sep 3, 2024
Copy link

coderabbitai bot commented Sep 3, 2024

Walkthrough

The changes introduce the whereNotInUnnest(...) method to the Query\Builder class and modify the existing whereIn method to automatically wrap input values in an UNNEST function when the number of values exceeds 950. Documentation updates clarify parameter handling for the Spanner database and introduce a configuration option, parameter_unnest_threshold. Additionally, new tests validate the behavior of the query builder concerning parameter limits and unnest overflow configurations.

Changes

Files Change Summary
CHANGELOG.md Added whereNotInUnnest(...) method and updated whereIn to wrap values in UNNEST for over 950 values.
README.md Clarified handling of query parameters exceeding 950 and introduced parameter_unnest_threshold.
src/Query/Builder.php Added PARAMETER_LIMIT constant, modified whereIn logic for unnest handling, and updated parameter type for whereInUnnest.
tests/Query/BuilderTest.php Added tests for whereIn with unnest overflow flag scenarios.

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Outside diff range, codebase verification and nitpick comments (1)
src/Query/Builder.php (1)

129-139: Excellent enhancement to handle large number of parameters!

The changes to the whereIn method provide a robust solution to handle queries with a high number of parameters by leveraging the UNNEST function. This will significantly improve the performance and usability of the query builder when working with large datasets.

Consider extracting the default threshold value (900) into a constant for better readability and maintainability. For example:

private const DEFAULT_UNNEST_THRESHOLD = 900;

// ...

$unnestThreshold = $this->connection->getConfig('parameter_unnest_threshold') ?? self::DEFAULT_UNNEST_THRESHOLD;
Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 81aebc9 and 5de5d62.

Files selected for processing (4)
  • CHANGELOG.md (1 hunks)
  • README.md (2 hunks)
  • src/Query/Builder.php (4 hunks)
  • tests/Query/BuilderTest.php (2 hunks)
Additional comments not posted (8)
src/Query/Builder.php (3)

36-36: LGTM!

The addition of the PARAMETER_LIMIT constant is approved. It provides a clear threshold for the maximum number of parameters allowed in a query.


167-173: LGTM!

The change to the whereInUnnest method signature to accept either a string or an Expression for the $column argument is approved. This enhances the method's flexibility and usability by allowing it to handle different types of column specifications.


22-22: LGTM!

The addition of the use statement for the Expression class is approved. It is required to correctly reference the Expression class in the updated whereInUnnest method signature.

CHANGELOG.md (2)

4-4: LGTM!

The change is appropriately documented in the changelog and is consistent with the PR summary.


5-5: LGTM!

The change is appropriately documented in the changelog and is consistent with the PR summary.

README.md (1)

102-104: LGTM!

The documentation clearly explains the limitation of binding more than 950 parameters in a single query and the driver's behavior to handle it by using Query\Builder::whereInUnnest(...) when the parameter count exceeds the parameter_unnest_threshold config value. The default value and the option to turn off the feature are also documented.

tests/Query/BuilderTest.php (2)

1089-1094: LGTM!

The test correctly verifies the behavior of whereIn when the unnest overflow flag is turned on. Passing 1000 values to whereIn should return an empty result set as expected.


1097-1105: LGTM!

The test correctly verifies the behavior of whereIn when the unnest overflow flag is turned off. Passing 1000 values to whereIn should throw a QueryException with the expected message.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 5de5d62 and bb813dc.

Files selected for processing (2)
  • phpstan.neon (1 hunks)
  • tests/Query/BuilderTest.php (2 hunks)
Files skipped from review as they are similar to previous changes (1)
  • tests/Query/BuilderTest.php
Additional comments not posted (1)
phpstan.neon (1)

51-53: Ensure that the code is handling the mixed type correctly.

The change ignores the error message related to the constructor of the Colopl\Spanner\Query\Nested class, which expects an array or an Arrayable object as the first parameter, but a mixed type was given.

Ignoring the error message might be necessary if the code is intentionally passing a mixed type to the constructor. However, it's important to ensure that the code is handling the mixed type correctly to avoid runtime errors.

Please verify that the code is correctly handling the mixed type passed to the constructor. If not, consider updating the code to handle the mixed type or enforce the expected type.

Copy link
Contributor

@t-matsuno-777 t-matsuno-777 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

@taka-oyama taka-oyama merged commit ac6789b into master Sep 4, 2024
1 check passed
@taka-oyama taka-oyama deleted the feature/where-unnest-auto branch September 4, 2024 03:10
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between bb813dc and e4d8f42.

Files selected for processing (1)
  • src/Query/Builder.php (4 hunks)
Additional comments not posted (5)
src/Query/Builder.php (5)

22-22: LGTM!

The import statement is necessary for using the Expression type in the whereInUnnest method signature.


36-37: LGTM!

The constants are used to define the parameter limit and the default threshold for using the UNNEST function. The constant names are clear and follow the naming convention.


130-140: LGTM!

The changes introduce a workaround to bypass the parameter limit by utilizing the UNNEST function. The threshold is configurable using the parameter_unnest_threshold config option, which is a good practice. The method falls back to the parent method if the threshold is not exceeded, which is a good fallback mechanism.


168-174: LGTM!

The change broadens the method's flexibility in handling different types of column specifications. The change is consistent with the import statement at line 22.


Line range hint 189-193:

@coderabbitai coderabbitai bot mentioned this pull request Sep 18, 2024
1 task
@coderabbitai coderabbitai bot mentioned this pull request Dec 18, 2024
15 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants