-
Notifications
You must be signed in to change notification settings - Fork 5.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
aggfuncs: implement avg(distinct) functions #7015
Conversation
/run-all-tests |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please fix CI.
/run-all-tests tidb-test=pr/569 |
1 similar comment
/run-all-tests tidb-test=pr/569 |
executor/aggfuncs/func_avg.go
Outdated
@@ -121,6 +121,68 @@ func (e *avgPartial4Decimal) UpdatePartialResult(sctx sessionctx.Context, rowsIn | |||
return nil | |||
} | |||
|
|||
type partialResult4AvgDistinctDecimal struct { | |||
partialResult4AvgDecimal | |||
exists map[types.MyDecimal]bool |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
map[types.MyDecimal]struct{}?
executor/aggfuncs/func_avg.go
Outdated
|
||
type partialResult4AvgDistinctFloat64 struct { | ||
partialResult4AvgFloat64 | ||
exists map[float64]bool |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ditto
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
map[types.MyDecimal]bool
is much convenient to use and will not introduce much memory overhead.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here is an experiment:
map[int]bool
with 10M elements: 315Mmap[int]struct{}
with 10M elements: 286M
1. memory usage of map[int]bool
package main
import (
"fmt"
"runtime"
)
func main() {
numElements := 10000000
var m1, m2 runtime.MemStats
runtime.ReadMemStats(&m1)
a := make(map[int]bool)
for i := 0; i < numElements; i++ {
a[i] = true
}
runtime.ReadMemStats(&m2)
fmt.Println("Bytes allocated for A:", m2.Alloc-m1.Alloc)
}
The result is:
[jianzhang.zj:/tmp]
➜ go build a.go
[jianzhang.zj:/tmp]
➜ ./a
Bytes allocated for A: 315457888
2. memory usage of map[int]struct{}
package main
import (
"fmt"
"runtime"
)
func main() {
numElements := 10000000
var m1, m2 runtime.MemStats
runtime.ReadMemStats(&m1)
b := make(map[int]struct{})
for i := 0; i < numElements; i++ {
b[i] = struct{}{}
}
runtime.ReadMemStats(&m2)
fmt.Println("Bytes allocated for B:", m2.Alloc-m1.Alloc)
}
the result is:
[jianzhang.zj:/tmp]
➜ go build b.go
[jianzhang.zj:/tmp]
➜ ./b
Bytes allocated for B: 286228080
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Using structure{} is the official practice in go
/run-all-tests tidb-test=pr/569 |
What have you changed? (mandatory)
Implement
AVG
functions which handles thedistinct
property:map[types.MyDecimal]bool
to check whether a decimal values already existsmap[float64]bool
for float64 values.What are the type of the changes (mandatory)?
How has this PR been tested (mandatory)?
-unit test
-explain test
Refer to a related PR or issue link (optional)
to #6952