Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support fine grained shuffle for window function #5048

Merged
merged 52 commits into from
Jul 11, 2022
Merged
Changes from 1 commit
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
21ea8e1
support fine grained shuffle
guo-shaoge May 24, 2022
1524bef
add unit test for StreamingDAGResponseWriter
guo-shaoge Jun 6, 2022
d860c45
add perf test for window function
guo-shaoge Jun 6, 2022
119b462
Merge branch 'master' of github.com:pingcap/tiflash into fine_grained…
guo-shaoge Jun 8, 2022
1e86122
fix conflict
guo-shaoge Jun 8, 2022
6dc4eab
fix contrib change
guo-shaoge Jun 8, 2022
2260514
fix some comment
guo-shaoge Jun 9, 2022
c1585b1
fix
guo-shaoge Jun 10, 2022
8211475
fix executeOrder()
guo-shaoge Jun 10, 2022
bcdf69f
Merge branch 'master' of github.com:pingcap/tiflash into fine_grained…
guo-shaoge Jun 15, 2022
162b1e3
add interpreter unittest
guo-shaoge Jun 15, 2022
df8aef7
Merge branch 'master' of github.com:pingcap/tiflash into fine_grained…
guo-shaoge Jun 16, 2022
d03f0eb
refine microbenchmark
guo-shaoge Jun 16, 2022
e5a01cf
rm exchange_perftest.cpp
guo-shaoge Jun 16, 2022
278240c
update kvproto dep
guo-shaoge Jun 20, 2022
0d4bf2e
fix fmt
guo-shaoge Jun 20, 2022
a4ee8a8
fix lint
guo-shaoge Jun 20, 2022
73e314d
Merge branch 'master' into fine_grained_shuffle
SeaRise Jun 21, 2022
eb605ce
Merge branch 'master' of github.com:pingcap/tiflash into fine_grained…
guo-shaoge Jun 23, 2022
e3d3ff3
fix conflict
guo-shaoge Jun 23, 2022
e8b9747
Merge branch 'fine_grained_shuffle' of github.com:guo-shaoge/tics int…
guo-shaoge Jun 23, 2022
db50925
enable fine_grained_shuffle in fragment level
guo-shaoge Jun 28, 2022
8624a85
Merge branch 'master' into fine_grained_shuffle
guo-shaoge Jun 29, 2022
c20c74c
stream_count from uint32_t to uint64_t
guo-shaoge Jun 30, 2022
b5df200
fix testcase
guo-shaoge Jul 1, 2022
4df7c9a
fix minor type
guo-shaoge Jul 1, 2022
d78a055
Merge branch 'fine_grained_shuffle' of github.com:guo-shaoge/tics int…
guo-shaoge Jul 1, 2022
9210111
fix comment
guo-shaoge Jul 1, 2022
e52fc32
add extra_info for stream
guo-shaoge Jul 2, 2022
6fbeb02
make enable_fine_grained_shuffle as template argument
guo-shaoge Jul 3, 2022
709ac80
change uint64_t to UInt64
guo-shaoge Jul 3, 2022
2c7ab24
fix some comment, add fullstack test
guo-shaoge Jul 4, 2022
505f49a
update tipb
guo-shaoge Jul 4, 2022
b1de8cd
Merge branch 'master' into fine_grained_shuffle
guo-shaoge Jul 4, 2022
383b89e
fix fmt
guo-shaoge Jul 4, 2022
db8497f
update window.test
guo-shaoge Jul 4, 2022
eda9596
Merge branch 'fine_grained_shuffle' of github.com:guo-shaoge/tics int…
guo-shaoge Jul 4, 2022
fb13e0e
fix some comments
guo-shaoge Jul 5, 2022
c23487a
update kvproto
guo-shaoge Jul 5, 2022
6e9f7b9
update kvproto
guo-shaoge Jul 5, 2022
030b874
fix bunch of comments
guo-shaoge Jul 7, 2022
59e3cb8
update
guo-shaoge Jul 7, 2022
c64d8ef
move send_exec_summary_at_last to top; Add RUNTIME_CHECK in DAGQueryB…
guo-shaoge Jul 7, 2022
c9ac908
using unique_ptr for msg_channels; use min(stream_count, max_stream)
guo-shaoge Jul 7, 2022
5893b37
Merge branch 'master' into fine_grained_shuffle
guo-shaoge Jul 7, 2022
0f035ec
fix
guo-shaoge Jul 7, 2022
031d1ed
Merge branch 'fine_grained_shuffle' of github.com:guo-shaoge/tics int…
guo-shaoge Jul 7, 2022
4f05ba7
Merge branch 'master' of github.com:pingcap/tiflash into fine_grained…
guo-shaoge Jul 8, 2022
8c34265
update tiflash-proxy
guo-shaoge Jul 8, 2022
ec3fe0d
fix conflict(mockExecutor.cpp)
guo-shaoge Jul 8, 2022
7b35d25
Merge branch 'master' into fine_grained_shuffle
guo-shaoge Jul 8, 2022
920673c
Merge branch 'master' into fine_grained_shuffle
guo-shaoge Jul 10, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix contrib change
Signed-off-by: guo-shaoge <shaoge1994@163.com>
guo-shaoge committed Jun 8, 2022
commit 6dc4eab67bf4aa76ca38c8c841ab8ef5381bcfe2
2 changes: 1 addition & 1 deletion contrib/client-c
2 changes: 1 addition & 1 deletion contrib/tiflash-proxy
Submodule tiflash-proxy updated 56 files
+44 −36 Cargo.lock
+3 −6 Cargo.toml
+27 −0 Makefile
+20 −7 build.sh
+6 −1 cargo-build.sh
+92 −0 cmd/tikv-ctl/Cargo.toml
+69 −8 components/cdc/src/delegate.rs
+1 −5 components/cdc/tests/integrations/test_cdc.rs
+3 −0 components/error_code/src/raftstore.rs
+2 −2 components/pd_client/src/client.rs
+2 −0 components/pd_client/src/util.rs
+1 −1 components/raft_log_engine/Cargo.toml
+1 −0 components/raftstore/Cargo.toml
+9 −0 components/raftstore/src/errors.rs
+3 −2 components/raftstore/src/store/config.rs
+1 −1 components/raftstore/src/store/fsm/apply.rs
+693 −206 components/raftstore/src/store/fsm/peer.rs
+85 −61 components/raftstore/src/store/fsm/store.rs
+1 −2 components/raftstore/src/store/mod.rs
+32 −13 components/raftstore/src/store/msg.rs
+348 −8 components/raftstore/src/store/peer.rs
+5 −0 components/raftstore/src/store/peer_storage.rs
+0 −14 components/raftstore/src/store/util.rs
+2 −1 components/raftstore/src/store/worker/mod.rs
+89 −142 components/raftstore/src/store/worker/pd.rs
+1 −0 components/raftstore/src/store/worker/raftlog_gc.rs
+1 −1 components/resource_metering/Cargo.toml
+1 −1 components/server/src/lib.rs
+8 −8 components/server/src/server.rs
+58 −51 components/test_raftstore/src/cluster.rs
+30 −28 components/test_raftstore/src/pd.rs
+4 −0 components/tikv_kv/src/lib.rs
+1 −1 components/tikv_util/Cargo.toml
+131 −91 components/tikv_util/src/config.rs
+10 −8 components/tikv_util/src/quota_limiter.rs
+65 −52 components/tikv_util/src/sys/cgroup.rs
+3 −4 components/tikv_util/src/sys/mod.rs
+10 −2 components/tikv_util/src/timer.rs
+10 −0 debug.sh
+1 −1 etc/config-template.toml
+2 −2 metrics/alertmanager/tikv.accelerate.rules.yml
+108 −2 metrics/grafana/tikv_details.json
+1 −1 metrics/grafana/tikv_fast_tune.json
+11 −0 release.sh
+65 −49 src/config.rs
+10 −6 src/server/raftkv.rs
+13 −2 src/server/service/kv.rs
+2 −20 src/server/status_server/profile.rs
+8 −0 src/storage/txn/scheduler.rs
+1 −1 tests/Cargo.toml
+78 −0 tests/failpoints/cases/test_merge.rs
+2 −2 tests/failpoints/cases/test_storage.rs
+408 −36 tests/failpoints/cases/test_unsafe_recovery.rs
+1 −1 tests/integrations/raftstore/mod.rs
+0 −86 tests/integrations/raftstore/test_unsafe_recover.rs
+1,070 −0 tests/integrations/raftstore/test_unsafe_recovery.rs