Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize overflow check when cast to decimal #3912

Merged
merged 62 commits into from
Mar 11, 2022
Merged
Show file tree
Hide file tree
Changes from 55 commits
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
7e73663
optimize int cast to decimal to skip check overflow
guo-shaoge Jan 19, 2022
bcd8dc2
more impl and unittest
guo-shaoge Jan 20, 2022
5f5b04c
fix wrong if branch
guo-shaoge Jan 20, 2022
3d182cd
fix fmt
guo-shaoge Jan 20, 2022
57fca6c
Merge branch 'master' of github.com:pingcap/tics into optimize_cast_o…
guo-shaoge Jan 27, 2022
07e18bb
fix comment(more concise way to impl createWrapperForDecimal)
guo-shaoge Jan 27, 2022
0483fb7
Merge branch 'optimize_cast_overflow' of github.com:guo-shaoge/tics i…
guo-shaoge Jan 27, 2022
f37618d
fix comment(use is_integral_v instead of is_same_v)
guo-shaoge Feb 8, 2022
0dd4619
fix clang-format(canSkip -> can_skip)
guo-shaoge Feb 8, 2022
fec28c5
fix comment(update CMakeLists.txt)
guo-shaoge Feb 8, 2022
91a574b
Merge branch 'master' into optimize_cast_overflow
guo-shaoge Feb 8, 2022
9956f0b
fix IntPrec
guo-shaoge Feb 8, 2022
ee531ba
Merge branch 'optimize_cast_overflow' of github.com:guo-shaoge/tics i…
guo-shaoge Feb 8, 2022
a809678
Merge branch 'master' into optimize_cast_overflow
guo-shaoge Feb 8, 2022
b9470c0
make cast decimal to decimal correct
guo-shaoge Feb 9, 2022
6e46838
Merge branch 'optimize_cast_overflow' of github.com:guo-shaoge/tics i…
guo-shaoge Feb 9, 2022
d9b1cf2
add date/datetime to dec check
guo-shaoge Feb 9, 2022
aeb3302
fix fmt
guo-shaoge Feb 10, 2022
1f9d5ef
remove duplicated code in bench_function_cast.cpp
guo-shaoge Feb 11, 2022
7ee8afd
predicate ScaleMulType instead of using Int256 directly
guo-shaoge Feb 13, 2022
4942c99
compute scale_mul per block
guo-shaoge Feb 13, 2022
418ac1c
Merge branch 'master' into optimize_cast_overflow
guo-shaoge Feb 13, 2022
2301e9e
Merge branch 'master' of github.com:pingcap/tics into optimize_cast_o…
guo-shaoge Feb 14, 2022
6aac099
use Int512 for cast(decimal as decimal) if necessary.
guo-shaoge Feb 14, 2022
ac24ca3
use ScaleMulType instead of ScaleMulNativeType
guo-shaoge Feb 15, 2022
0e8bab9
add comments for some code
guo-shaoge Feb 16, 2022
235cef1
delete IntPrec::real_prec
guo-shaoge Feb 16, 2022
6808d65
fix unit test case
guo-shaoge Feb 16, 2022
88427a7
add benchmark which expose the huge cost of computation of Int256
guo-shaoge Feb 16, 2022
bca800b
Merge branch 'master' into optimize_cast_overflow
guo-shaoge Feb 16, 2022
f505339
remove file
guo-shaoge Feb 17, 2022
ca933de
Merge branch 'optimize_cast_overflow' of github.com:guo-shaoge/tics i…
guo-shaoge Feb 17, 2022
84de905
optimize decimal to decimal, use scale_mul instead of for loop
guo-shaoge Feb 17, 2022
a02254b
compute max_value for each block instead of for each row
guo-shaoge Feb 17, 2022
5b8d74b
fix datetime to decimal error(fsp)
guo-shaoge Feb 17, 2022
45e0660
fix
guo-shaoge Feb 17, 2022
fe17f40
Merge branch 'master' of github.com:pingcap/tics into optimize_cast_o…
guo-shaoge Feb 18, 2022
2485329
remove benchmark_case, update gtest
guo-shaoge Feb 21, 2022
037158c
fix
guo-shaoge Feb 21, 2022
d9f2589
avoid performance regression
guo-shaoge Feb 24, 2022
61bd210
fix comment, make code cleaner
guo-shaoge Feb 25, 2022
976e397
fix fmt
guo-shaoge Feb 25, 2022
0b5eafe
use original impl for cast datetime to decimal
guo-shaoge Feb 28, 2022
d227e5d
Merge branch 'master' of github.com:pingcap/tics into optimize_cast_o…
guo-shaoge Feb 28, 2022
e2165da
Merge branch 'optimize_cast_overflow' of github.com:guo-shaoge/tics i…
guo-shaoge Feb 28, 2022
d036bc8
Merge branch 'master' of github.com:pingcap/tics into optimize_cast_o…
guo-shaoge Mar 3, 2022
1f62770
address part of comments
guo-shaoge Mar 4, 2022
7a13eee
address some comments
guo-shaoge Mar 7, 2022
2366721
fix
guo-shaoge Mar 7, 2022
e385e22
address last(maybe) comments
guo-shaoge Mar 7, 2022
0c6d27a
fix
guo-shaoge Mar 7, 2022
37d219b
Merge branch 'master' into optimize_cast_overflow
guo-shaoge Mar 8, 2022
aecbc55
add more comments
guo-shaoge Mar 8, 2022
e96d4f6
Merge branch 'optimize_cast_overflow' of github.com:guo-shaoge/tics i…
guo-shaoge Mar 8, 2022
be4cd1a
clean comment
guo-shaoge Mar 8, 2022
9149ec8
fix comments
guo-shaoge Mar 9, 2022
751660e
Merge branch 'master' into optimize_cast_overflow
guo-shaoge Mar 9, 2022
a95945e
Merge branch 'master' into optimize_cast_overflow
ti-chi-bot Mar 11, 2022
af1a9be
Merge branch 'master' into optimize_cast_overflow
ti-chi-bot Mar 11, 2022
e518833
Merge branch 'master' into optimize_cast_overflow
ti-chi-bot Mar 11, 2022
3decfb4
Merge branch 'master' into optimize_cast_overflow
ti-chi-bot Mar 11, 2022
0fd8651
Merge branch 'master' into optimize_cast_overflow
ti-chi-bot Mar 11, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions dbms/src/Columns/ColumnDecimal.h
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,8 @@ class ColumnDecimal final : public COWPtrHelper<ColumnVectorHelper, ColumnDecima
const T & getElement(size_t n) const { return data[n]; }
T & getElement(size_t n) { return data[n]; }

UInt32 getScale() const { return scale; }

protected:
Container data;
UInt32 scale;
Expand Down
34 changes: 29 additions & 5 deletions dbms/src/Common/Decimal.h
Original file line number Diff line number Diff line change
Expand Up @@ -20,14 +20,20 @@ using ScaleType = UInt32;
constexpr PrecType decimal_max_prec = 65;
constexpr ScaleType decimal_max_scale = 30;

// IntPrec indicates the max precision of different integer types.
// For now, the binary arithmetic functions use it to calculate result precision.
// And cast function use it to do some optimizations, such as skipping overflow check.
// But in TiDB the signed types will plus 1, for example IntPrec<int8_t>::prec is 4.
// This is a little confusing because we will add 1 when return result to client.
// Here we make sure TiFlash code is clean and will fix TiDB later.
template <typename T>
struct IntPrec
{
};
template <>
struct IntPrec<int8_t>
{
static constexpr PrecType prec = 4;
static constexpr PrecType prec = 3;
guo-shaoge marked this conversation as resolved.
Show resolved Hide resolved
};
template <>
struct IntPrec<uint8_t>
Expand All @@ -37,7 +43,7 @@ struct IntPrec<uint8_t>
template <>
struct IntPrec<int16_t>
{
static constexpr PrecType prec = 6;
static constexpr PrecType prec = 5;
};
template <>
struct IntPrec<uint16_t>
Expand All @@ -47,7 +53,7 @@ struct IntPrec<uint16_t>
template <>
struct IntPrec<int32_t>
{
static constexpr PrecType prec = 11;
static constexpr PrecType prec = 10;
};
template <>
struct IntPrec<uint32_t>
Expand All @@ -57,14 +63,26 @@ struct IntPrec<uint32_t>
template <>
struct IntPrec<int64_t>
{
static constexpr PrecType prec = 20;
static constexpr PrecType prec = 19;
fuzhe1989 marked this conversation as resolved.
Show resolved Hide resolved
};
template <>
struct IntPrec<uint64_t>
{
static constexpr PrecType prec = 20;
};

template <>
struct IntPrec<Int128>
{
static constexpr PrecType prec = 39;
};

template <>
struct IntPrec<Int256>
{
static constexpr PrecType prec = 78;
guo-shaoge marked this conversation as resolved.
Show resolved Hide resolved
};

// 1) If the declared type of both operands of a dyadic arithmetic operator is exact numeric, then the declared
// type of the result is an implementation-defined exact numeric type, with precision and scale determined as
// follows:
Expand Down Expand Up @@ -384,12 +402,18 @@ class DecimalMaxValue final : public ext::Singleton<DecimalMaxValue>

// In some case, getScaleMultiplier and its callee may not be auto inline by the compiler.
// This may hurt performance. __attribute__((flatten)) tells compliler to inline the callee of this function.
template <typename T>
template <typename T, std::enable_if_t<IsDecimal<T>> * = nullptr>
__attribute__((flatten)) inline typename T::NativeType getScaleMultiplier(ScaleType scale)
{
return static_cast<typename T::NativeType>(DecimalMaxValue::get(scale) + 1);
guo-shaoge marked this conversation as resolved.
Show resolved Hide resolved
}

template <typename T, std::enable_if_t<is_integer_v<T>> * = nullptr>
__attribute__((flatten)) inline T getScaleMultiplier(ScaleType scale)
{
return static_cast<T>(DecimalMaxValue::get(scale) + 1);
}

template <typename T>
inline void checkDecimalOverflow(Decimal<T> v, PrecType prec)
{
Expand Down
Loading