Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lightning: fail to import data into tables with expression index enabled #44497

Closed
lichunzhu opened this issue Jun 8, 2023 · 0 comments · Fixed by #44516
Closed

lightning: fail to import data into tables with expression index enabled #44497

lichunzhu opened this issue Jun 8, 2023 · 0 comments · Fixed by #44516
Assignees
Labels
affects-6.1 This bug affects the 6.1.x(LTS) versions. affects-6.5 This bug affects the 6.5.x(LTS) versions. affects-7.1 This bug affects the 7.1.x(LTS) versions. component/lightning This issue is related to Lightning of TiDB. severity/major type/bug The issue is confirmed as a bug.

Comments

@lichunzhu
Copy link
Contributor

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

use lightning to import these data into a tidb cluster with tidb's experimental.allow-expression-index setting to true.

test.t-schema.sql

/*!40101 SET NAMES binary*/;
CREATE TABLE `t` (
  `a` bigint(30) NOT NULL,
  `b` varchar(40) NOT NULL DEFAULT uuid(),
  UNIQUE KEY `c` (`b`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin;

test.t.0000000.csv

/!40101 SET NAMES binary/;
"a"
1
2
3
4
5

2. What did you expect to see? (Required)

Lightning can import these data and generate uuid for column b successfully.

mysql> select * from t;
+---+--------------------------------------+
| a | b                                    |
+---+--------------------------------------+
| 1 | 224bb472-05a3-11ee-91e8-b28e57435a9e |
| 2 | 224bb95e-05a3-11ee-91e8-b28e57435a9e |
| 3 | 224bba1c-05a3-11ee-91e8-b28e57435a9e |
| 4 | 224bba9e-05a3-11ee-91e8-b28e57435a9e |
| 5 | 224bbb0c-05a3-11ee-91e8-b28e57435a9e |
+---+--------------------------------------+
5 rows in set (0.00 sec)

3. What did you see instead (Required)

Lightning panics.

$ tiup tidb-lightning:v6.5.2 -c /Users/chauncy/code/goPath/src/github.com/pingcap/tidb/bin/lightning_e2e.toml 
Starting component `tidb-lightning`: /Users/chauncy/.tiup/components/tidb-lightning/v6.5.2/tidb-lightning -c /Users/chauncy/code/goPath/src/github.com/pingcap/tidb/bin/lightning_e2e.toml
Verbose debug logs will be written to /var/folders/5m/9r5zk5b942b4ktht2_5rklfr0000gn/T/lightning.log.2023-06-08T10.21.34+0800

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x91 pc=0x10470de90]

goroutine 322 [running]:
github.com/pingcap/tidb/sessionctx/variable.(*SessionVars).IsPessimisticReadConsistency(...)
        /Users/pingcap/workspace/build-common/go/src/github.com/pingcap/br/sessionctx/variable/session.go:1990
github.com/pingcap/tidb/planner/core.(*PlanBuilder).Init(0x14000d45d40, {0x105ca4500, 0x14000a96480}, {0x0?, 0x0}, 0x14000c66400)
        /Users/pingcap/workspace/build-common/go/src/github.com/pingcap/br/planner/core/planbuilder.go:730 +0x180
github.com/pingcap/tidb/planner/core.rewriteAstExpr({0x105ca4500, 0x14000a96480}, {0x105c5c5f8, 0x14000e64120}, 0x0, {0x0, 0x0, 0x0})
        /Users/pingcap/workspace/build-common/go/src/github.com/pingcap/br/planner/core/expression_rewriter.go:72 +0xb0
github.com/pingcap/tidb/planner/core.evalAstExpr({0x105ca4500?, 0x14000a96480?}, {0x105c5c5f8?, 0x14000e64120?})
        /Users/pingcap/workspace/build-common/go/src/github.com/pingcap/br/planner/core/expression_rewriter.go:58 +0xc8
github.com/pingcap/tidb/table.getColDefaultExprValue({0x105ca4500, 0x14000a96480}, 0x48?, {0x14000e4a970?, 0x0?})
        /Users/pingcap/workspace/build-common/go/src/github.com/pingcap/br/table/column.go:527 +0x118
github.com/pingcap/tidb/table.GetColDefaultValue({0x105ca4500?, 0x14000a96480?}, 0x5?)
        /Users/pingcap/workspace/build-common/go/src/github.com/pingcap/br/table/column.go:503 +0xbc
github.com/pingcap/tidb/br/pkg/lightning/backend/kv.(*tableKVEncoder).getActualDatum(0x140010c41e0, 0x1, 0x1, 0x0)
        /Users/pingcap/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/lightning/backend/kv/sql2kv.go:509 +0x3e0
github.com/pingcap/tidb/br/pkg/lightning/backend/kv.(*tableKVEncoder).Encode(0x140010c41e0, {0x14000e9a390?}, {0x14000bcd200?, 0x1, 0x10}, 0x1, {0x14000e4e528, 0x3, 0x14001145588?}, {0x102a49728?, ...}, ...)
        /Users/pingcap/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/lightning/backend/kv/sql2kv.go:377 +0x1b0
github.com/pingcap/tidb/br/pkg/lightning/restore.(*chunkRestore).encodeLoop(0x14000f24e20, {0x105c3a928, 0x14000cb8d80}, 0xc?, 0x14000a96880, {0x10296d93c?}, {0x105c1f560, 0x140010c41e0}, 0x0?, 0x1400034d6c0)
        /Users/pingcap/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/lightning/restore/restore.go:2690 +0xa2c
github.com/pingcap/tidb/br/pkg/lightning/restore.(*chunkRestore).restore(0x14000f24e20, {0x105c3a928, 0x14000cb8d80}, 0x14000a96880, 0x0, 0x14000f24e40, 0x14000cb0020, 0x1400034d6c0)
        /Users/pingcap/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/lightning/restore/restore.go:2787 +0x4c0
github.com/pingcap/tidb/br/pkg/lightning/restore.(*TableRestore).restoreEngine.func3(0x1400072dc20?, 0x14000f24e20)
        /Users/pingcap/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/lightning/restore/table_restore.go:582 +0x148
created by github.com/pingcap/tidb/br/pkg/lightning/restore.(*TableRestore).restoreEngine
        /Users/pingcap/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/lightning/restore/table_restore.go:572 +0xb84

4. What is your TiDB version? (Required)

master/v6.5.2 all have this problem

@lichunzhu lichunzhu added type/bug The issue is confirmed as a bug. severity/major component/lightning This issue is related to Lightning of TiDB. affects-6.5 This bug affects the 6.5.x(LTS) versions. affects-7.1 This bug affects the 7.1.x(LTS) versions. labels Jun 8, 2023
@ti-chi-bot ti-chi-bot bot added may-affects-5.1 This bug maybe affects 5.1.x versions. may-affects-5.2 This bug maybe affects 5.2.x versions. may-affects-5.3 This bug maybe affects 5.3.x versions. may-affects-5.4 This bug maybe affects 5.4.x versions. may-affects-6.1 labels Jun 8, 2023
@lichunzhu lichunzhu removed may-affects-5.1 This bug maybe affects 5.1.x versions. may-affects-5.2 This bug maybe affects 5.2.x versions. may-affects-5.3 This bug maybe affects 5.3.x versions. may-affects-5.4 This bug maybe affects 5.4.x versions. may-affects-6.1 labels Jun 8, 2023
@ti-chi-bot ti-chi-bot added the affects-6.1 This bug affects the 6.1.x(LTS) versions. label Sep 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affects-6.1 This bug affects the 6.1.x(LTS) versions. affects-6.5 This bug affects the 6.5.x(LTS) versions. affects-7.1 This bug affects the 7.1.x(LTS) versions. component/lightning This issue is related to Lightning of TiDB. severity/major type/bug The issue is confirmed as a bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants