Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

*: Selection can control the conditions of the below scan plan. #2834

Merged
merged 30 commits into from
Mar 28, 2017
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
5db58a9
*: add indexScan controlled by correlated column
winoros Mar 15, 2017
1fe024d
Merge branch 'master' into yiding/apply_optimize
winoros Mar 15, 2017
5ce9107
tiny change
winoros Mar 15, 2017
364e172
fix merge bug
winoros Mar 15, 2017
36283cf
remove log
winoros Mar 15, 2017
7451287
remove log
winoros Mar 15, 2017
053c2d4
fix bug and tiny change
winoros Mar 16, 2017
e663291
address some comments
winoros Mar 16, 2017
a25a3eb
manage cast in plan build phase
winoros Mar 17, 2017
6aadeaa
address comments and manage cast in executor phase
winoros Mar 17, 2017
ba29a95
address some comments
winoros Mar 20, 2017
02cbcf1
remove useless args
winoros Mar 20, 2017
9a9ae32
change some logic
winoros Mar 20, 2017
ef178a3
fix bug
winoros Mar 20, 2017
8029c1e
refactor
winoros Mar 21, 2017
9771129
address comments
winoros Mar 21, 2017
7251827
tiny change
winoros Mar 21, 2017
e45f669
address some comments
winoros Mar 22, 2017
4e0012b
add test
winoros Mar 22, 2017
f0ec2e0
remove redundant function
winoros Mar 22, 2017
02db63c
move function
winoros Mar 22, 2017
c791ff7
move code
winoros Mar 22, 2017
8674c3f
typo error && add explain
winoros Mar 22, 2017
107a042
Merge branch 'master' into yiding/inlj_1
winoros Mar 23, 2017
3fd0435
remove outdate comment
winoros Mar 23, 2017
8f29d17
add tests and some tiny change
winoros Mar 24, 2017
088e9e6
tiny change
winoros Mar 24, 2017
6351036
add test
winoros Mar 24, 2017
97998fc
address comments
winoros Mar 27, 2017
55f92e6
Merge branch 'master' into yiding/inlj_1
shenli Mar 28, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions executor/builder.go
Original file line number Diff line number Diff line change
Expand Up @@ -471,10 +471,11 @@ func (b *executorBuilder) buildAggregation(v *plan.PhysicalAggregation) Executor

func (b *executorBuilder) buildSelection(v *plan.Selection) Executor {
exec := &SelectionExec{
Src: b.build(v.Children()[0]),
Condition: expression.ComposeCNFCondition(b.ctx, v.Conditions...),
schema: v.Schema(),
ctx: b.ctx,
Src: b.build(v.Children()[0]),
schema: v.Schema(),
ctx: b.ctx,
scanController: v.ScanController,
Conditions: v.Conditions,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you store the conditions, you don't need to store condition any more. Use a for loop to check condition.

}
return exec
}
Expand Down
108 changes: 104 additions & 4 deletions executor/executor.go
Original file line number Diff line number Diff line change
Expand Up @@ -461,21 +461,111 @@ func (e *TableDualExec) Close() error {
return nil
}

func substituteCorCol2Constant(expr expression.Expression) (expression.Expression, error) {
switch x := expr.(type) {
case *expression.ScalarFunction:
allConstant := true
newArgs := make([]expression.Expression, 0, len(x.GetArgs()))
for _, arg := range x.GetArgs() {
newArg, err := substituteCorCol2Constant(arg)
_, ok := newArg.(*expression.Constant)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Put this after checking error.

if err != nil {
return nil, errors.Trace(err)
}
newArgs = append(newArgs, newArg)
allConstant = allConstant && ok
}
if allConstant {
val, err := x.Eval(nil)
if err != nil {
return nil, errors.Trace(err)
}
return &expression.Constant{Value: val}, nil
}
var newSf expression.Expression
if x.FuncName.L == ast.Cast {
newSf = expression.NewCastFunc(x.RetType, newArgs[0], x.GetCtx())
} else {
newSf, _ = expression.NewFunction(x.GetCtx(), x.FuncName.L, x.GetType(), newArgs...)
}
return newSf, nil
case *expression.CorrelatedColumn:
return &expression.Constant{Value: *x.Data, RetType: x.GetType()}, nil
case *expression.Constant:
return x.Clone(), nil
default:
return x.Clone(), nil
}
}

// SelectionExec represents a filter executor.
type SelectionExec struct {
Src Executor
Condition expression.Expression
ctx context.Context
schema *expression.Schema

// scanController will tell whether this selection need to
// control the condition of below scan executor.
scanController bool
controllerInit bool
Conditions []expression.Expression
usableIndices []*model.IndexInfo
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can remove this from selection executor.

}

// Schema implements the Executor Schema interface.
func (e *SelectionExec) Schema() *expression.Schema {
return e.schema
}

// initController will init the conditions of the below scan executor.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add more comments for this function.

// It will first substitute the correlated column to constant, then build range and filter by new conditions.
func (e *SelectionExec) initController() error {
sc := e.ctx.GetSessionVars().StmtCtx
client := e.ctx.GetClient()
newConds := make([]expression.Expression, 0, len(e.Conditions))
for _, cond := range e.Conditions {
newCond, err := substituteCorCol2Constant(cond.Clone())
if err != nil {
return errors.Trace(err)
}
newConds = append(newConds, newCond)
}

switch x := e.Src.(type) {
case *XSelectTableExec:
accessCondition, restCondtion := plan.DetachTableScanConditions(newConds, x.tableInfo)
x.where, _, _ = plan.ExpressionsToPB(sc, restCondtion, client)
ranges, err := plan.BuildTableRange(accessCondition, sc)
if err != nil {
return errors.Trace(err)
}
x.ranges = ranges
case *XSelectIndexExec:
x.indexPlan.AccessCondition, newConds = plan.DetachIndexScanConditions(newConds, x.indexPlan)
idxConds, tblConds := plan.DetachIndexFilterConditions(newConds, x.indexPlan.Index.Columns, x.indexPlan.Table)
x.indexPlan.IndexConditionPBExpr, _, _ = plan.ExpressionsToPB(sc, idxConds, client)
x.indexPlan.TableConditionPBExpr, _, _ = plan.ExpressionsToPB(sc, tblConds, client)
err := plan.BuildIndexRange(sc, x.indexPlan)
if err != nil {
return errors.Trace(err)
}
x.where = x.indexPlan.TableConditionPBExpr
default:
return errors.New("Error type of PhysicalPlan")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

print %T to tout put the plan type

}
return nil
}

// Next implements the Executor Next interface.
func (e *SelectionExec) Next() (*Row, error) {
if e.scanController && !e.controllerInit {
err := e.initController()
if err != nil {
return nil, errors.Trace(err)
}
e.controllerInit = true
}
for {
srcRow, err := e.Src.Next()
if err != nil {
Expand All @@ -484,18 +574,28 @@ func (e *SelectionExec) Next() (*Row, error) {
if srcRow == nil {
return nil, nil
}
match, err := expression.EvalBool(e.Condition, srcRow.Data, e.ctx)
if err != nil {
return nil, errors.Trace(err)
allMatch := true
for _, cond := range e.Conditions {
match, err := expression.EvalBool(cond, srcRow.Data, e.ctx)
if err != nil {
return nil, errors.Trace(err)
}
if !match {
allMatch = false
break
}
}
if match {
if allMatch {
return srcRow, nil
}
}
}

// Close implements the Executor Close interface.
func (e *SelectionExec) Close() error {
if e.scanController {
e.controllerInit = false
}
return e.Src.Close()
}

Expand Down
3 changes: 2 additions & 1 deletion plan/expr_to_pb.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,8 @@ import (
"github.com/pingcap/tipb/go-tipb"
)

func expressionsToPB(sc *variable.StatementContext, exprs []expression.Expression, client kv.Client) (pbExpr *tipb.Expr, pushed []expression.Expression, remained []expression.Expression) {
// ExpressionsToPB converts expression to tipb.Expr.
func ExpressionsToPB(sc *variable.StatementContext, exprs []expression.Expression, client kv.Client) (pbExpr *tipb.Expr, pushed []expression.Expression, remained []expression.Expression) {
pc := pbConverter{client: client, sc: sc}
for _, expr := range exprs {
v := pc.exprToPB(expr)
Expand Down
12 changes: 12 additions & 0 deletions plan/logical_plans.go
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,18 @@ type Selection struct {

// onTable means if this selection's child is a table scan or index scan.
onTable bool

// If ScanController is true, then the child of this selection is a scan,
// which use pk or index. we will record the accessConditions, idxConditions,
// and tblConditions to control the below plan.
ScanController bool

// Since one selection may call convert2PhysicalScan many times. We extract the PkName and indices
// used for scanController only once and store them to judge whether this selection can convert to
// scanController mode.
usefulPkName model.CIStr
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we don't need to store the usefulPkName and usefulIndices.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we don't store the usefulPkName and usefulIndices. We will need to extract them one more time when build controller.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay, it seems not cost too much.

usefulIndices []*model.IndexInfo
extractedUsefulThing bool
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

set a flag that we set controller when the flag is true. The flag is true only the selection has correlated column and its child is DataSource. You can check this during decorrelation.

}

func (p *Selection) extractCorrelatedCols() []*expression.CorrelatedColumn {
Expand Down
Loading