Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closes #6460 Maybe optimize lookups of tables in DB to stop using SHOW TABLES LIKE #6968

Merged
merged 2 commits into from
Sep 20, 2024

Conversation

Khadreal
Copy link
Contributor

@Khadreal Khadreal commented Sep 11, 2024

Description

Changed lookup table query.

Fixes #6460

Type of change

  • Enhancement (non-breaking change which improves an existing functionality).

Detailed scenario

With a large site(mostly multisites), the query is run on every page loads and this slows down the page.
More information here https://wp-media.slack.com/archives/C43T1AYMQ/p1709561232039849

Technical description

Changed the query from show tables to select with a limit of 1

Documentation

Mandatory Checklist

Code validation

  • I validated all the Acceptance Criteria. If possible, provide screenshots or videos.
  • I triggered all changed lines of code at least once without new errors/warnings/notices.

Code style

  • I wrote a self-explanatory code about what it does.
  • I protected entry points against unexpected inputs.
  • I did not introduce unnecessary complexity.

@Khadreal Khadreal added type: enhancement Improvements that slightly enhance existing functionality and are fast to implement module: database labels Sep 11, 2024
@Khadreal Khadreal self-assigned this Sep 11, 2024
Copy link

Coverage summary from Codacy

See diff coverage on Codacy

Coverage variation Diff coverage
Report missing for 84982f81 50.00% (target: 50.00%)
Coverage variation details
Coverable lines Covered lines Coverage
Common ancestor commit (84982f8) Report Missing Report Missing Report Missing
Head commit (e951e59) 37565 9464 25.19%

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details
Coverable lines Covered lines Diff coverage
Pull request (#6968) 4 2 50.00%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

See your quality gate settings    Change summary preferences

Codacy stopped sending the deprecated coverage status on June 5th, 2024. Learn more

Footnotes

  1. Codacy didn't receive coverage data for the commit, or there was an error processing the received data. Check your integration for errors and validate that your coverage setup is correct.

@Khadreal Khadreal requested a review from a team September 12, 2024 11:16
@Khadreal Khadreal marked this pull request as ready for review September 12, 2024 11:19
@MathieuLamiot
Copy link
Contributor

@wp-media/qa-team @Khadreal to discuss how to validate / test this.
A quick feedback here: the main risk I see is that we are not using "LIKE" anymore but a strictly equal table name. We would need to check on different setups (multi-site, a few hosting, etc.) that we always have the exact table name.
@Khadreal for instance, in most queries, we use $db->prefix, which does not seem to be the case in your PR. We'd need to ensure we are not missing a prefix mechanism here.

$prefixed_table_name = $db->prefix . $this->table_name;
$query = "SELECT id FROM `$prefixed_table_name` WHERE `last_accessed` <= date_sub(now(), interval $delay $unit)";

@Mai-Saad
Copy link
Contributor

Mai-Saad commented Sep 19, 2024

Test plan:

@Khadreal
Copy link
Contributor Author

@wp-media/qa-team @Khadreal to discuss how to validate / test this. A quick feedback here: the main risk I see is that we are not using "LIKE" anymore but a strictly equal table name. We would need to check on different setups (multi-site, a few hosting, etc.) that we always have the exact table name. @Khadreal for instance, in most queries, we use $db->prefix, which does not seem to be the case in your PR. We'd need to ensure we are not missing a prefix mechanism here.

$prefixed_table_name = $db->prefix . $this->table_name;
$query = "SELECT id FROM `$prefixed_table_name` WHERE `last_accessed` <= date_sub(now(), interval $delay $unit)";

@MathieuLamiot
Tested this on multisite and its work as it should. $db->{$this->table_name} will return the table name with prefix. This $this->get_db() returns the db name with prefix

@Miraeld
Copy link
Contributor

Miraeld commented Sep 19, 2024

Hello hello,

I've conducted some tests and some logs of execution time between the old version and new one.

  • For a database with 100 tables, it decreased from an average of 15ms to 5ms, representing a 66% improvement in performance.
  • When I scaled up to a database with 500 tables, the improvement was even more significant. The execution time decreased from an average of 75ms to 15ms, which is an 80% improvement.
  • Finally, in a large-scale scenario with a database containing 100,000 tables, the execution time went down from an average of 2 seconds to 20-30 ms.

So I confirm that it works as expected.

@Miraeld Miraeld self-requested a review September 19, 2024 17:26
@Mai-Saad Mai-Saad added this pull request to the merge queue Sep 20, 2024
Merged via the queue into develop with commit 975503d Sep 20, 2024
13 of 15 checks passed
@Mai-Saad Mai-Saad deleted the enhancement/6460-optimize-lookups-table branch September 20, 2024 06:03
github-merge-queue bot pushed a commit that referenced this pull request Oct 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
module: database type: enhancement Improvements that slightly enhance existing functionality and are fast to implement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Maybe optimize lookups of tables in DB to stop using SHOW TABLES LIKE
5 participants