Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

planner,executor,infoschema: add system tables tiflash_tables and tiflash_segments #18092

Merged
merged 33 commits into from
Jul 7, 2020

Conversation

lidezhu
Copy link
Contributor

@lidezhu lidezhu commented Jun 17, 2020

What problem does this PR solve?

we can read tiflash storage stat from tidb for the ease of debug.

What is changed and how it works?

What's Changed: add two new system tables tiflash_tables and tiflash_segments.

How it Works: it uses etcd client to get http port of tiflash nodes, and then get data from corresponding tiflash system tables through the http port.

mysql> select tidb_table,table_id,segment_count,tiflash_instance from tiflash_tables where tidb_database = 'test';
+------------+----------+---------------+------------------+
| tidb_table | table_id | segment_count | tiflash_instance |
+------------+----------+---------------+------------------+
| t          |       45 |             1 | 127.0.0.1:9500   |
| t2         |       47 |             1 | 127.0.0.1:9500   |
+------------+----------+---------------+------------------+
2 rows in set (0.01 sec)

mysql> select count(*) from tiflash_tables;
+----------+
| count(*) |
+----------+
|       22 |
+----------+
1 row in set (0.02 sec)

mysql> select tidb_table,table_id from tiflash_tables where tiflash_instance='127.0.0.1:9500' limit 5;
+------------------+----------+
| tidb_table       | table_id |
+------------------+----------+
| t                |       45 |
| t2               |       47 |
| tables_priv      |       11 |
| columns_priv     |       13 |
| global_variables |       15 |
+------------------+----------+
5 rows in set (0.02 sec)

mysql> select `tidb_table`,`segment_id`,`range`,`rows` from tiflash_segments where tidb_database='test';
+------------+------------+--------------------------------------------+------+
| tidb_table | segment_id | range                                      | rows |
+------------+------------+--------------------------------------------+------+
| t          |          1 | [-9223372036854775808,9223372036854775807) |    0 |
| t2         |          1 | [-9223372036854775808,9223372036854775807) |    0 |
+------------+------------+--------------------------------------------+------+
2 rows in set (0.01 sec)

mysql> select count(*) from tiflash_segments;
+----------+
| count(*) |
+----------+
|       22 |
+----------+
1 row in set (0.01 sec)

mysql> select `tidb_table`,`segment_id`,`range`,`rows` from tiflash_segments where tiflash_instance='127.0.0.1:9500' limit 5;
+------------------+------------+--------------------------------------------+------+
| tidb_table       | segment_id | range                                      | rows |
+------------------+------------+--------------------------------------------+------+
| t                |          1 | [-9223372036854775808,9223372036854775807) |   64 |
| t2               |          1 | [-9223372036854775808,9223372036854775807) |    0 |
| tables_priv      |          1 | [-9223372036854775808,9223372036854775807) |    0 |
| columns_priv     |          1 | [-9223372036854775808,9223372036854775807) |    0 |
| global_variables |          1 | [-9223372036854775808,9223372036854775807) |    0 |
+------------------+------------+--------------------------------------------+------+
5 rows in set (0.01 sec)

Related changes

  • Need to cherry-pick to the release branch

Check List

Tests

  • Unit test
  • Manual test: create a tidb cluster with three tiflash nodes, create 100000 tables in tidb,and then select from the two new added system tables to see whether the results meet expectation.

Release note

  • Add system tables tiflash_tables and tiflash_segments

@lidezhu lidezhu requested review from a team as code owners June 17, 2020 11:05
@lidezhu lidezhu requested review from qw4990 and SunRunAway and removed request for a team June 17, 2020 11:05
@lidezhu lidezhu changed the title read tiflash system table from tidb add tiflash storage info to tidb system tables Jun 17, 2020
@github-actions github-actions bot added sig/sql-infra SIG: SQL Infra sig/execution SIG execution labels Jun 17, 2020
@lidezhu lidezhu changed the title add tiflash storage info to tidb system tables planner,executor,infoschema: add system tables tiflash_tables and tiflash_segments Jun 17, 2020
@lidezhu
Copy link
Contributor Author

lidezhu commented Jun 17, 2020

/run-unit-tests

1 similar comment
@lidezhu
Copy link
Contributor Author

lidezhu commented Jun 18, 2020

/run-unit-tests

@lidezhu
Copy link
Contributor Author

lidezhu commented Jun 18, 2020

/run-all-tests

1 similar comment
@lidezhu
Copy link
Contributor Author

lidezhu commented Jun 18, 2020

/run-all-tests

@codecov
Copy link

codecov bot commented Jun 18, 2020

Codecov Report

Merging #18092 into master will not change coverage.
The diff coverage is n/a.

@@             Coverage Diff             @@
##             master     #18092   +/-   ##
===========================================
  Coverage   79.5852%   79.5852%           
===========================================
  Files           535        535           
  Lines        145679     145679           
===========================================
  Hits         115939     115939           
  Misses        20466      20466           
  Partials       9274       9274           

@lidezhu lidezhu force-pushed the add-dt-system-table branch from 9270649 to a53bd37 Compare June 29, 2020 06:49
@lidezhu
Copy link
Contributor Author

lidezhu commented Jul 3, 2020

/run-unit-test

2 similar comments
@lidezhu
Copy link
Contributor Author

lidezhu commented Jul 3, 2020

/run-unit-test

@lidezhu
Copy link
Contributor Author

lidezhu commented Jul 3, 2020

/run-unit-test

@lidezhu
Copy link
Contributor Author

lidezhu commented Jul 3, 2020

@crazycs520 PTAL

Copy link
Contributor

@crazycs520 crazycs520 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@crazycs520
Copy link
Contributor

/run-all-tests

@ti-srebot ti-srebot added the status/LGT1 Indicates that a PR has LGTM 1. label Jul 3, 2020
Copy link
Contributor

@ti-srebot ti-srebot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot
Copy link
Contributor

@crazycs520,Thanks for your review.

@lidezhu
Copy link
Contributor Author

lidezhu commented Jul 3, 2020

/run-all-tests

1 similar comment
@lidezhu
Copy link
Contributor Author

lidezhu commented Jul 3, 2020

/run-all-tests

@lidezhu
Copy link
Contributor Author

lidezhu commented Jul 3, 2020

/merge

@ti-srebot
Copy link
Contributor

Sorry @lidezhu, you don't have permission to trigger auto merge event on this branch. You are not a committer or co-leader or leader for the related sigs:ddl(slack),execution(slack).

@crazycs520 crazycs520 merged commit 5f983b8 into pingcap:master Jul 7, 2020
@lidezhu lidezhu deleted the add-dt-system-table branch July 8, 2020 03:46
ti-srebot pushed a commit to ti-srebot/tidb that referenced this pull request Jul 14, 2020
@ti-srebot
Copy link
Contributor

cherry pick to release-4.0 in PR #18536

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sig/execution SIG execution sig/sql-infra SIG: SQL Infra status/LGT1 Indicates that a PR has LGTM 1.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants