Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

store/helper: fill data in the information.tidb_hot_table for partitioned table #14331

Merged
merged 3 commits into from
Jan 10, 2020
Merged

store/helper: fill data in the information.tidb_hot_table for partitioned table #14331

merged 3 commits into from
Jan 10, 2020

Conversation

tiancaiamao
Copy link
Contributor

What problem does this PR solve?

INFORMATION.TIDB_HOT_TABLE should take the partitioned table into consideration.

What is changed and how it works?

A partitioned table is not handled properly in tidb_hot_table.
After this change, we can get the correct information:

mysql> select * from tidb_hot_regions;
+----------+----------+---------+------------------+------------+-----------+-------+----------------+--------------+------------+
| TABLE_ID | INDEX_ID | DB_NAME | TABLE_NAME       | INDEX_NAME | REGION_ID | TYPE  | MAX_HOT_DEGREE | REGION_COUNT | FLOW_BYTES |
+----------+----------+---------+------------------+------------+-----------+-------+----------------+--------------+------------+
|       23 |     NULL | mysql   | stats_histograms | NULL       |        48 | read  |              3 |            0 |     211478 |
|       25 |     NULL | mysql   | stats_buckets    | NULL       |        52 | write |              3 |            0 |      93013 |
|      151 |     NULL | test    | sbtest3(p1)      | NULL       |      1156 | write |              3 |            0 |      93013 |
|      150 |     NULL | test    | sbtest3(p0)      | NULL       |      1152 | write |              3 |            0 |      93013 |
+----------+----------+---------+------------------+------------+-----------+-------+----------------+--------------+------------+
4 rows in set (0.08 sec)

Note, 151 and 150 are both belong to sbtest3, and the table name column is displayed as sbtest3(p1) and sbtest3(p0) to indicate they are partitions.

Check List

Tests

  • Manual test (add detailed scripts or steps below)

Mock the data is hard, the old code is not really unit tested.
I use a real tikv cluster and check the result.
B.T.W, a bug is fixed, pd returns something like this:

{
  "as_peer": {
    "1": {
      "total_flow_bytes": 190434.14973961416,
      "regions_count": 3,
      "statistics": [
        {
          "store_id": 1,
          "region_id": 1156,
          "hot_degree": 3,
          "AntiCount": 1,
          "kind": 0,
          "flow_bytes": 50956.69507956832,
          "flow_keys": 0,
          "RollingBytesRate": null,
          "last_update_time": "2020-01-03T16:10:54.147084771+08:00",
          "Version": 127
        },
        {
          "store_id": 1,
          "region_id": 1152,
          "hot_degree": 3,
          "AntiCount": 1,
          "kind": 0,
          "flow_bytes": 93013.72966004584,
          "flow_keys": 0,
          "RollingBytesRate": null,
          "last_update_time": "2020-01-03T16:10:51.146018185+08:00",
          "Version": 126
        },
        {
          "store_id": 1,
          "region_id": 52,
          "hot_degree": 3,
          "AntiCount": 1,
          "kind": 0,
          "flow_bytes": 46463.725,
          "flow_keys": 0,
          "RollingBytesRate": null,
          "last_update_time": "2020-01-03T16:10:48.507062772+08:00",
          "Version": 13
        }
      ]
    },
    "4": {
      "total_flow_bytes": 190434.14973961416,
      "regions_count": 3,
      "statistics": [
        {
          "store_id": 4,
          "region_id": 1156,
          "hot_degree": 3,
          "AntiCount": 1,
          "kind": 0,
          "flow_bytes": 50956.69507956832,
          "flow_keys": 0,
          "RollingBytesRate": null,
          "last_update_time": "2020-01-03T16:10:54.147078479+08:00",
          "Version": 127
        },
        {
          "store_id": 4,
          "region_id": 1152,
          "hot_degree": 3,
          "AntiCount": 1,
          "kind": 0,
          "flow_bytes": 93013.72966004584,
          "flow_keys": 0,
          "RollingBytesRate": null,
          "last_update_time": "2020-01-03T16:10:51.146019774+08:00",
          "Version": 126
        },
        {
          "store_id": 4,
          "region_id": 52,
          "hot_degree": 3,
          "AntiCount": 1,
          "kind": 0,
          "flow_bytes": 46463.725,
          "flow_keys": 0,
          "RollingBytesRate": null,
          "last_update_time": "2020-01-03T16:10:48.507064672+08:00",
          "Version": 13
        }
      ]
    },
    "5": {
      "total_flow_bytes": 190434.14973961416,
      "regions_count": 3,
      "statistics": [
        {
          "store_id": 5,
          "region_id": 1156,
          "hot_degree": 3,
          "AntiCount": 1,
          "kind": 0,
          "flow_bytes": 50956.69507956832,
          "flow_keys": 0,
          "RollingBytesRate": null,
          "last_update_time": "2020-01-03T16:10:54.147083702+08:00",
          "Version": 127
        },
        {
          "store_id": 5,
          "region_id": 1152,
          "hot_degree": 3,
          "AntiCount": 1,
          "kind": 0,
          "flow_bytes": 93013.72966004584,
          "flow_keys": 0,
          "RollingBytesRate": null,
          "last_update_time": "2020-01-03T16:10:51.146020767+08:00",
          "Version": 126
        },
        {
          "store_id": 5,
          "region_id": 52,
          "hot_degree": 3,
          "AntiCount": 1,
          "kind": 0,
          "flow_bytes": 46463.725,
          "flow_keys": 0,
          "RollingBytesRate": null,
          "last_update_time": "2020-01-03T16:10:48.507065709+08:00",
          "Version": 13
        }
      ]
    }
  },
  "as_leader": {
    "1": {
      "total_flow_bytes": 0,
      "regions_count": 0,
      "statistics": []
    },
    "4": {
      "total_flow_bytes": 143970.42473961416,
      "regions_count": 2,
      "statistics": [
        {
          "store_id": 4,
          "region_id": 1156,
          "hot_degree": 3,
          "AntiCount": 1,
          "kind": 0,
          "flow_bytes": 50956.69507956832,
          "flow_keys": 0,
          "RollingBytesRate": null,
          "last_update_time": "2020-01-03T16:10:54.147078479+08:00",
          "Version": 127
        },
        {
          "store_id": 4,
          "region_id": 1152,
          "hot_degree": 3,
          "AntiCount": 1,
          "kind": 0,
          "flow_bytes": 93013.72966004584,
          "flow_keys": 0,
          "RollingBytesRate": null,
          "last_update_time": "2020-01-03T16:10:51.146019774+08:00",
          "Version": 126
        }
      ]
    },
    "5": {
      "total_flow_bytes": 46463.725,
      "regions_count": 1,
      "statistics": [
        {
          "store_id": 5,
          "region_id": 52,
          "hot_degree": 3,
          "AntiCount": 1,
          "kind": 0,
          "flow_bytes": 46463.725,
          "flow_keys": 0,
          "RollingBytesRate": null,
          "last_update_time": "2020-01-03T16:10:48.507065709+08:00",
          "Version": 13
        }
      ]
    }
  }
}

From the PD result we can see flow_bytes is a float, not uint. @disksing @nolouch

"flow_bytes": 50956.69507956832

Related changes

  • Need to cherry-pick to the release branch

Release note

  • Write release note for bug-fix or new feature.

@tiancaiamao
Copy link
Contributor Author

PTAL @winoros @disksing

@tiancaiamao tiancaiamao requested a review from winoros January 6, 2020 09:07
@tiancaiamao
Copy link
Contributor Author

PTAL @winoros @disksing

zimulala
zimulala reviewed Jan 9, 2020
View reviewed changes
Copy link
Contributor

@zimulala zimulala left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zimulala zimulala added the status/LGT1 Indicates that a PR has LGTM 1. label Jan 9, 2020
winoros
winoros reviewed Jan 9, 2020
View reviewed changes
store/helper/helper.go Outdated Show resolved Hide resolved
@tiancaiamao
Copy link
Contributor Author

PTAL @winoros

zz-jason
zz-jason approved these changes Jan 10, 2020
View reviewed changes
Copy link
Member

@zz-jason zz-jason left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zz-jason zz-jason added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Jan 10, 2020
@zz-jason
Copy link
Member

could you add some tests for this bugfix?

@tiancaiamao
Copy link
Contributor Author

could you add some tests for this bugfix?

Not easy to do it in the unit test. If we mock data, it can't prevent PD from changing the interface in the future.

@tiancaiamao
Copy link
Contributor Author

/run-all-tests

@tiancaiamao tiancaiamao merged commit ad71a5a into pingcap:master Jan 10, 2020
@tiancaiamao tiancaiamao deleted the hot-table-partition branch January 10, 2020 15:48
@tiancaiamao tiancaiamao mentioned this pull request Jan 17, 2020
Closed
@jackysp
Copy link
Member

jackysp commented Apr 22, 2020

/run-cherry-picker

sre-bot pushed a commit to sre-bot/tidb that referenced this pull request Apr 22, 2020
@tiancaiamao @sre-bot
store/helper: fill data in the information.tidb_hot_table for partiti…
7ae0f0f 
…oned table (pingcap#14331)
@sre-bot sre-bot mentioned this pull request Apr 22, 2020
Merged
@sre-bot
Copy link
Contributor

sre-bot commented Apr 22, 2020

cherry pick to release-3.0 in PR #16726

sre-bot pushed a commit to sre-bot/tidb that referenced this pull request Apr 22, 2020
@tiancaiamao @sre-bot
store/helper: fill data in the information.tidb_hot_table for partiti…
8b09670 
…oned table (pingcap#14331)
@sre-bot sre-bot mentioned this pull request Apr 22, 2020
Merged
@sre-bot
Copy link
Contributor

sre-bot commented Apr 22, 2020

cherry pick to release-3.1 in PR #16727

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@winoros winoros winoros left review comments

@zimulala zimulala zimulala left review comments

@zz-jason zz-jason zz-jason approved these changes

Assignees
No one assigned
Labels
component/infoschema status/LGT2 Indicates that a PR has LGTM 2. type/bugfix This PR fixes a bug.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@tiancaiamao @zz-jason @jackysp @sre-bot @zimulala @winoros