[Flash-664] Enable DDL for engine DeltaTree #526
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
JaySon-Huang
added
the
needs-cherry-pick-release-3.1
This comment has been minimized.
This comment has been minimized.
JaySon-Huang
changed the title
This comment has been minimized.
This comment has been minimized.
3 similar comments
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
1 similar comment
|
/run-integration-tests |
JaySon-Huang
added a commit
that referenced
this pull request
Mar 17, 2020
## Add fullstack test for engine DeltaTree.
* Refine `tests/docker/run.sh` and split `tests/docker/docker-compose.yaml` into `tests/docker/{gtest/mock-test/cluster/tiflash-dt/tiflash-tmt}.yaml`
`fullstack/ddl`,`fullstack-test/fault-inject` will be enabled in #526
## Others
* Add column `tidb_table_id` in `system.tables`
* Add some debugging info
Signed-off-by: JaySon-Huang <[email protected]>
JaySon-Huang
added a commit
to JaySon-Huang/tiflash
that referenced
this pull request
Mar 17, 2020
* Refine `tests/docker/run.sh` and split `tests/docker/docker-compose.yaml` into `tests/docker/{gtest/mock-test/cluster/tiflash-dt/tiflash-tmt}.yaml`
`fullstack/ddl`,`fullstack-test/fault-inject` will be enabled in pingcap#526
* Add column `tidb_table_id` in `system.tables`
* Add some debugging info
Conflicts:
tests/generate-fullstack-test.py
Signed-off-by: JaySon-Huang <[email protected]>
flowbehappy
reviewed
Mar 17, 2020
Split docker-compose file Signed-off-by: JaySon-Huang <[email protected]>
Signed-off-by: JaySon-Huang <[email protected]>
Signed-off-by: JaySon-Huang <[email protected]>
Signed-off-by: JaySon-Huang <[email protected]>
Signed-off-by: JaySon-Huang <[email protected]>
Signed-off-by: JaySon-Huang <[email protected]>
Signed-off-by: JaySon-Huang <[email protected]>
Signed-off-by: JaySon-Huang <[email protected]>
Signed-off-by: JaySon-Huang <[email protected]>
Signed-off-by: JaySon-Huang <[email protected]>
|
/run-integration-tests |
Signed-off-by: JaySon-Huang <[email protected]>
JaySon-Huang
commented
Mar 19, 2020
JaySon-Huang
commented
Mar 19, 2020
Signed-off-by: JaySon-Huang <[email protected]>
flowbehappy
approved these changes
Mar 20, 2020
…d default value test Signed-off-by: JaySon-Huang <[email protected]>
JaySon-Huang
commented
Mar 20, 2020
| mysql> alter table test.t add year_2 YEAR NULL DEFAULT '0000' | ||
| mysql> alter table test.t add year_a YEAR NOT NULL | ||
|
|
||
| #TODO: fix zero value for year |
There was a problem hiding this comment.
Find this bug, jira issue FLASH-1023
There was a problem hiding this comment.
cc @hanfei1991
There was a problem hiding this comment.
cc @windtalker
|
/run-integration-tests |
2 similar comments
|
/run-integration-tests |
|
/run-integration-tests |
|
/merge |
1 similar comment
|
/merge |
|
/run-integration-tests |
Signed-off-by: JaySon-Huang <[email protected]>
|
/run-integration-tests |
|
cherry pick to release-3.1 failed |
JaySon-Huang
added a commit
to JaySon-Huang/tiflash
that referenced
this pull request
Mar 20, 2020
- [x] Enable unittest in gtest_dbms - [x] Enable mock test in tests/delta-merge-test - [x] Enable fullstack-test/ddl - [x] Enable fullstack-test/inject (Imported in pingcap#443) - [x] Refine exception while read / write to DeltaTree (FLASH-994) * Use `Exception::addMessage` to add more diagnostics for locate which table is wrong (commit: 716ae4a) - [x] shutdown should cancel all background tasks (FLASH-995) (commit: 7470c2f) - [x] Run schrodinger/sddl test ## Others * Add atomic-rename table test in `tests/fullstack-test/fault-inject/rename-table.test`, but did not enable this. We will fix it later. * "dt" engine ONLY support disable_bg_flush = true. If background flush is enabled, read will not triggle schema sync. Which means that we may not get the right result with out-dated schema. * Found that 'zero' value of type year is not match with tikv (FLASH-1023) Signed-off-by: JaySon-Huang <[email protected]>
7 tasks
JaySon-Huang
added a commit
that referenced
this pull request
Mar 21, 2020
- [x] Enable unittest in gtest_dbms - [x] Enable mock test in tests/delta-merge-test - [x] Enable fullstack-test/ddl - [x] Enable fullstack-test/inject (Imported in #443) - [x] Refine exception while read / write to DeltaTree (FLASH-994) * Use `Exception::addMessage` to add more diagnostics for locate which table is wrong (commit: 716ae4a) - [x] shutdown should cancel all background tasks (FLASH-995) (commit: 7470c2f) - [x] Run schrodinger/sddl test ## Others * Add atomic-rename table test in `tests/fullstack-test/fault-inject/rename-table.test`, but did not enable this. We will fix it later. * "dt" engine ONLY support disable_bg_flush = true. If background flush is enabled, read will not triggle schema sync. Which means that we may not get the right result with out-dated schema. * Found that 'zero' value of type year is not match with tikv (FLASH-1023) Signed-off-by: JaySon-Huang <[email protected]>
windtalker
added a commit
that referenced
this pull request
Mar 26, 2020
* fix daily test fail (#520) * fix daily test fail * fix * Add fullstack test for engine DeltaTree (#524) ## Add fullstack test for engine DeltaTree. * Refine `tests/docker/run.sh` and split `tests/docker/docker-compose.yaml` into `tests/docker/{gtest/mock-test/cluster/tiflash-dt/tiflash-tmt}.yaml` `fullstack/ddl`,`fullstack-test/fault-inject` will be enabled in #526 ## Others * Add column `tidb_table_id` in `system.tables` * Add some debugging info Signed-off-by: JaySon-Huang <[email protected]> * Bugfix: schrodinger bank2 fail (#521) * Flush committed data in Region after resolve locks * Stop append into last packs after split. * Remove last_cache in Delta to reduce code complexity. * Add system table: dt_tables and dt_segments, for debug. * [FLASH-1008] Support br restore & ingest sst (#529) * [flash-1018]fix bug of datetime default value (#534) * fix bug of datetime default value * address comment * Using SegmentSnapshotPtr instead of SegmentSnapshot (#532) * [Flash-664] Enable DDL for engine DeltaTree (#526) - [x] Enable unittest in gtest_dbms - [x] Enable mock test in tests/delta-merge-test - [x] Enable fullstack-test/ddl - [x] Enable fullstack-test/inject (Imported in #443) - [x] Refine exception while read / write to DeltaTree (FLASH-994) * Use `Exception::addMessage` to add more diagnostics for locate which table is wrong (commit: 716ae4a) - [x] shutdown should cancel all background tasks (FLASH-995) (commit: 7470c2f) - [x] Run schrodinger/sddl test ## Others * Add atomic-rename table test in `tests/fullstack-test/fault-inject/rename-table.test`, but did not enable this. We will fix it later. * "dt" engine ONLY support disable_bg_flush = true. If background flush is enabled, read will not triggle schema sync. Which means that we may not get the right result with out-dated schema. * Found that 'zero' value of type year is not match with tikv (FLASH-1023) Signed-off-by: JaySon-Huang <[email protected]> * [FLASH-1027] Fix: proxy override system signal listening (#541) * remove signal listening from proxy. * while terminating, stop all learner read. * [FLASH-1026] Synchronization between drop table and remove region (#538) * Set default storage engine to DT (#547) * Do region flush in Region::handleWriteRaftCmd (#542) * [FLASH-1028] Region merge should not remove data (#544) * Remove region after region merge should not remove data Signed-off-by: JaySon-Huang <[email protected]> * Fix region init index Signed-off-by: JaySon-Huang <[email protected]> * Add region_merge.test for DT Signed-off-by: JaySon-Huang <[email protected]> * Fix different behavior between DT and TMT Signed-off-by: JaySon-Huang <[email protected]> * Fix mock remove region Signed-off-by: JaySon-Huang <[email protected]> Co-authored-by: pingcap-github-bot <[email protected]> * clean useless code Co-authored-by: JaySon <[email protected]> Co-authored-by: Flowyi <[email protected]> Co-authored-by: Tong Zhigao <[email protected]> Co-authored-by: Han Fei <[email protected]> Co-authored-by: pingcap-github-bot <[email protected]>
hanfei1991
added a commit
that referenced
this pull request
Jun 22, 2020
* implement join * fix * update tipb * update tipb * save work: refine dag interpreter, introduce dag interpreter query block * fix bug * comment useless code * refine code * fix bug * refine code * fix bug * save work * tiny fix * save work * support remote read * update tipb * refine code * fix bug * update client-c * refine cop read * update client-c to support cop reading from tiflash * refine code * support batch cop * fix build error * fix daily test fail * some bug fix * log dag execution time without encode to chunk * fix bug * fix bug * log dag execution time without encode * make encode multi processors * fix * parallel encode * refine code of batch coprocessor * refine code * delete useless code * update kvproto and client-c * [flash 1002]refine coprocessor read (#530) * refind coprocessor * fix * try fix ci * support key ranges in batch coprocessor (#533) * save work * save work * save work * save work * dt support key ranges in dag request * fix bug * fix bug * fix bug * add some comments * fix bug * address comments * merge master branch (#556) * fix daily test fail (#520) * fix daily test fail * fix * Add fullstack test for engine DeltaTree (#524) ## Add fullstack test for engine DeltaTree. * Refine `tests/docker/run.sh` and split `tests/docker/docker-compose.yaml` into `tests/docker/{gtest/mock-test/cluster/tiflash-dt/tiflash-tmt}.yaml` `fullstack/ddl`,`fullstack-test/fault-inject` will be enabled in #526 ## Others * Add column `tidb_table_id` in `system.tables` * Add some debugging info Signed-off-by: JaySon-Huang <[email protected]> * Bugfix: schrodinger bank2 fail (#521) * Flush committed data in Region after resolve locks * Stop append into last packs after split. * Remove last_cache in Delta to reduce code complexity. * Add system table: dt_tables and dt_segments, for debug. * [FLASH-1008] Support br restore & ingest sst (#529) * [flash-1018]fix bug of datetime default value (#534) * fix bug of datetime default value * address comment * Using SegmentSnapshotPtr instead of SegmentSnapshot (#532) * [Flash-664] Enable DDL for engine DeltaTree (#526) - [x] Enable unittest in gtest_dbms - [x] Enable mock test in tests/delta-merge-test - [x] Enable fullstack-test/ddl - [x] Enable fullstack-test/inject (Imported in #443) - [x] Refine exception while read / write to DeltaTree (FLASH-994) * Use `Exception::addMessage` to add more diagnostics for locate which table is wrong (commit: 716ae4a) - [x] shutdown should cancel all background tasks (FLASH-995) (commit: 7470c2f) - [x] Run schrodinger/sddl test ## Others * Add atomic-rename table test in `tests/fullstack-test/fault-inject/rename-table.test`, but did not enable this. We will fix it later. * "dt" engine ONLY support disable_bg_flush = true. If background flush is enabled, read will not triggle schema sync. Which means that we may not get the right result with out-dated schema. * Found that 'zero' value of type year is not match with tikv (FLASH-1023) Signed-off-by: JaySon-Huang <[email protected]> * [FLASH-1027] Fix: proxy override system signal listening (#541) * remove signal listening from proxy. * while terminating, stop all learner read. * [FLASH-1026] Synchronization between drop table and remove region (#538) * Set default storage engine to DT (#547) * Do region flush in Region::handleWriteRaftCmd (#542) * [FLASH-1028] Region merge should not remove data (#544) * Remove region after region merge should not remove data Signed-off-by: JaySon-Huang <[email protected]> * Fix region init index Signed-off-by: JaySon-Huang <[email protected]> * Add region_merge.test for DT Signed-off-by: JaySon-Huang <[email protected]> * Fix different behavior between DT and TMT Signed-off-by: JaySon-Huang <[email protected]> * Fix mock remove region Signed-off-by: JaySon-Huang <[email protected]> Co-authored-by: pingcap-github-bot <[email protected]> * clean useless code Co-authored-by: JaySon <[email protected]> Co-authored-by: Flowyi <[email protected]> Co-authored-by: Tong Zhigao <[email protected]> Co-authored-by: Han Fei <[email protected]> Co-authored-by: pingcap-github-bot <[email protected]> * fix type mismatch bug in broadcast join * broadcast join support join keys with different data type (#580) * fix type mismatch bug in broadcast join * refine code * refine code * some improvement for broadcast join (#600) * some improvement for broadcast join * format code * refine code * address comment * fix bug * address comments * fix bug * fix bug * make TiFlash backward compatible to old tipb (#653) * 1. re-enable exec info in dag response, 2. support old style dag request * basic support for execute summary * refine support of executor time for join plan * format code * address comments * fix bug * refine code * update header * Fix execute details regression after merge master (#678) * refine code * fix bug * fix bug * format code * update kvproto * fmt code * update client-c * update tipb Co-authored-by: xufei <[email protected]> Co-authored-by: xufei <[email protected]> Co-authored-by: JaySon <[email protected]> Co-authored-by: Flowyi <[email protected]> Co-authored-by: Tong Zhigao <[email protected]> Co-authored-by: pingcap-github-bot <[email protected]>
hanfei1991
added a commit
that referenced
this pull request
Jul 2, 2020
* implement join * fix * update tipb * update tipb * save work: refine dag interpreter, introduce dag interpreter query block * fix bug * comment useless code * refine code * fix bug * refine code * fix bug * save work * tiny fix * save work * support remote read * update tipb * refine code * fix bug * update client-c * refine cop read * update client-c to support cop reading from tiflash * refine code * support batch cop * fix build error * fix daily test fail * some bug fix * log dag execution time without encode to chunk * fix bug * fix bug * log dag execution time without encode * make encode multi processors * fix * parallel encode * refine code of batch coprocessor * refine code * delete useless code * update kvproto and client-c * [flash 1002]refine coprocessor read (#530) * refind coprocessor * fix * try fix ci * support key ranges in batch coprocessor (#533) * save work * save work * save work * save work * dt support key ranges in dag request * fix bug * fix bug * fix bug * add some comments * fix bug * address comments * merge master branch (#556) * fix daily test fail (#520) * fix daily test fail * fix * Add fullstack test for engine DeltaTree (#524) ## Add fullstack test for engine DeltaTree. * Refine `tests/docker/run.sh` and split `tests/docker/docker-compose.yaml` into `tests/docker/{gtest/mock-test/cluster/tiflash-dt/tiflash-tmt}.yaml` `fullstack/ddl`,`fullstack-test/fault-inject` will be enabled in #526 ## Others * Add column `tidb_table_id` in `system.tables` * Add some debugging info Signed-off-by: JaySon-Huang <[email protected]> * Bugfix: schrodinger bank2 fail (#521) * Flush committed data in Region after resolve locks * Stop append into last packs after split. * Remove last_cache in Delta to reduce code complexity. * Add system table: dt_tables and dt_segments, for debug. * [FLASH-1008] Support br restore & ingest sst (#529) * [flash-1018]fix bug of datetime default value (#534) * fix bug of datetime default value * address comment * Using SegmentSnapshotPtr instead of SegmentSnapshot (#532) * [Flash-664] Enable DDL for engine DeltaTree (#526) - [x] Enable unittest in gtest_dbms - [x] Enable mock test in tests/delta-merge-test - [x] Enable fullstack-test/ddl - [x] Enable fullstack-test/inject (Imported in #443) - [x] Refine exception while read / write to DeltaTree (FLASH-994) * Use `Exception::addMessage` to add more diagnostics for locate which table is wrong (commit: 716ae4a) - [x] shutdown should cancel all background tasks (FLASH-995) (commit: 7470c2f) - [x] Run schrodinger/sddl test ## Others * Add atomic-rename table test in `tests/fullstack-test/fault-inject/rename-table.test`, but did not enable this. We will fix it later. * "dt" engine ONLY support disable_bg_flush = true. If background flush is enabled, read will not triggle schema sync. Which means that we may not get the right result with out-dated schema. * Found that 'zero' value of type year is not match with tikv (FLASH-1023) Signed-off-by: JaySon-Huang <[email protected]> * [FLASH-1027] Fix: proxy override system signal listening (#541) * remove signal listening from proxy. * while terminating, stop all learner read. * [FLASH-1026] Synchronization between drop table and remove region (#538) * Set default storage engine to DT (#547) * Do region flush in Region::handleWriteRaftCmd (#542) * [FLASH-1028] Region merge should not remove data (#544) * Remove region after region merge should not remove data Signed-off-by: JaySon-Huang <[email protected]> * Fix region init index Signed-off-by: JaySon-Huang <[email protected]> * Add region_merge.test for DT Signed-off-by: JaySon-Huang <[email protected]> * Fix different behavior between DT and TMT Signed-off-by: JaySon-Huang <[email protected]> * Fix mock remove region Signed-off-by: JaySon-Huang <[email protected]> Co-authored-by: pingcap-github-bot <[email protected]> * clean useless code Co-authored-by: JaySon <[email protected]> Co-authored-by: Flowyi <[email protected]> Co-authored-by: Tong Zhigao <[email protected]> Co-authored-by: Han Fei <[email protected]> Co-authored-by: pingcap-github-bot <[email protected]> * fix type mismatch bug in broadcast join * broadcast join support join keys with different data type (#580) * fix type mismatch bug in broadcast join * refine code * refine code * some improvement for broadcast join (#600) * some improvement for broadcast join * format code * refine code * address comment * fix bug * address comments * fix bug * fix bug * make TiFlash backward compatible to old tipb (#653) * 1. re-enable exec info in dag response, 2. support old style dag request * basic support for execute summary * refine support of executor time for join plan * format code * address comments * fix bug * refine code * update header * Fix execute details regression after merge master (#678) * refine code * fix bug * fix bug * format code * update kvproto * fmt code * update client-c * update tipb Co-authored-by: xufei <[email protected]> Co-authored-by: xufei <[email protected]> Co-authored-by: JaySon <[email protected]> Co-authored-by: Flowyi <[email protected]> Co-authored-by: Tong Zhigao <[email protected]> Co-authored-by: pingcap-github-bot <[email protected]> Co-authored-by: xufei <[email protected]> Co-authored-by: xufei <[email protected]> Co-authored-by: JaySon <[email protected]> Co-authored-by: Flowyi <[email protected]> Co-authored-by: Tong Zhigao <[email protected]> Co-authored-by: pingcap-github-bot <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Exception::addMessageto add more diagnostics for locate which table is wrong (commit: 716ae4a)Others
tests/fullstack-test/fault-inject/rename-table.test, but did not enable this. We will fix it later.If background flush is enabled, read will not triggle schema sync. Which means that we may not get the right result with out-dated schema.