test: add security bvt #20345
test: add security bvt #20345
Conversation
…built-in function. (matrixorigin#19672) try to reuse the FunctionParameterWrapper. fix a spool bug will write a panic log incorrelty. Approved by: @XuPeng-SH, @sukki37, @aunjgr
…atrixorigin#19675) 1. Cherry-pick 2.0-dev Approved by: @XuPeng-SH, @sukki37
…ed (matrixorigin#19682) do not unsubscribe table before it finished waited, otherwise, it will never stop waiting for the table's first logtail as it is subscribed. Approved by: @triump2020, @sukki37
fix drop database without lock table Approved by: @qingxinhome, @zhangxu19830126, @m-schen, @ouyuanning, @daviszhen, @sukki37, @aunjgr
fix a bug in shuffle which cause panic Approved by: @ouyuanning, @sukki37, @aunjgr
- lower log print threshold - check rowid of insert batch Approved by: @LeftHandCold, @XuPeng-SH, @sukki37
1. Cherry-pick Approved by: @XuPeng-SH, @sukki37
- remove debug log - fix batch split flow in filterInMemUnCommittedInserts Approved by: @XuPeng-SH, @m-schen, @aunjgr, @sukki37
cherry-pick execute sql 执行的时候， checkTableDefChange里面调用HasNewerVersion判断 Approved by: @badboynt1, @aressu1985, @ouyuanning, @XuPeng-SH, @daviszhen, @aunjgr, @sukki37
…ator scan (matrixorigin#19677) 1. reduce the comparison of Exact, Prefix, In, Between. 2. TPCC 100w*1000t performance 10%+ Approved by: @triump2020, @XuPeng-SH, @sukki37
…trixorigin#19704) reduce the interval time of refresh cn nodes Approved by: @daviszhen, @zhangxu19830126, @sukki37
Operator analyzer adds CounterSet member for S3 resource statistics, reducing memory allocation Approved by: @XuPeng-SH, @m-schen, @sukki37
…2.0 (matrixorigin#19749) abort all active txn on invalid service and make new txn can work Approved by: @XuPeng-SH, @sukki37
Fix test global checkpoint5 2.0 Approved by: @XuPeng-SH, @sukki37
Fix lockservice dead lock with txn . Use separate goroutines handle getActiveTxns Approved by: @iamlinjunhong, @sukki37
fix dedup and add debug log Approved by: @XuPeng-SH, @sukki37
…matrixorigin#19780) return retry error if snapshot ts is smaller than ts of catalog. before retry, update snapshot ts to catalog start ts. Approved by: @XuPeng-SH, @sukki37
Password management policy to 2.0-dev Approved by: @iamlinjunhong, @daviszhen, @heni02, @zhangxu19830126, @XuPeng-SH, @sukki37
set sink mysql timeout to 10min Approved by: @daviszhen, @sukki37
After judging that the WAL has no Pendding LSN, the merge is executed again, so an exception is thrown. The merge needs to be closed in this case. Approved by: @XuPeng-SH, @sukki37
After physical recovery, the mo panic is started. Fix the bug of missing files in physical backup. Backup support snapshot Approved by: @daviszhen, @zhangxu19830126, @XuPeng-SH, @sukki37
Improve cdc send sql performance Approved by: @daviszhen, @sukki37
fix txn status invalid if commit failed or commit a readonly txn Approved by: @iamlinjunhong, @sukki37
Songxx-7
requested review from
triump2020,
badboynt1,
aptend,
xzxiong,
ouyuanning,
qingxinhome,
iamlinjunhong,
daviszhen,
LeftHandCold,
fengttt,
ck89119,
aressu1985,
heni02 and
sukki37
as code owners
|
PR-Agent was enabled for this repository. To continue using it, please link your git user with your CodiumAI identity here. PR Reviewer Guide 🔍Here are some key observations to aid the review process:
|
|
PR-Agent was enabled for this repository. To continue using it, please link your git user with your CodiumAI identity here. PR Code Suggestions ✨Explore these optional code suggestions:
|
User description
What type of PR is this?
Which issue(s) this PR fixes:
issue #
https://github.com/matrixorigin/MO-Cloud/issues/4247
https://github.com/matrixorigin/MO-Cloud/issues/4248
What this PR does / why we need it:
test: add security bvt
PR Type
Tests, Enhancement
Description
backExecwith execution statistics tracking.Entry_DataObjectandEntry_TombstoneObjectto protocol buffer definitions.ResumeInvalidCNmethod and associated types in protocol buffers.TNStorebefore service addition.tableReader.Changes walkthrough 📝
5 files
api.pb.go
Add new entry types to protocol buffer definitions.pkg/pb/api/api.pb.go
Entry_DataObjectandEntry_TombstoneObject.Entry_EntryType_nameandEntry_EntryType_valuemaps toinclude new entry types.
FileDescriptorProtobyte array to reflect changes.back_exec.go
Enhance backExec with execution statistics tracking.pkg/frontend/back_exec.go
statsArraytobackExecfor tracking execution statistics.ExecandExecRestoremethods to utilizestatsArray.lock.pb.go
Add support for ResumeInvalidCN method and typespkg/pb/lock/lock.pb.go
ResumeInvalidCNwith associated constants andmappings.
ResumeInvalidCNRequestandResumeInvalidCNResponsetypeswith their methods.
RequestandResponsetypes to includeResumeInvalidCN.types.
txn.pb.go
Added `Flag` field to `CNTxnSnapshot` struct and updated methods.pkg/pb/txn/txn.pb.go
Flagto theCNTxnSnapshotstruct.GetFlagfor theFlagfield.MarshalToSizedBuffer,ProtoSize, andUnmarshalmethods tohandle the
Flagfield.cluster.go
Added state check for `TNStore` before service addition.pkg/clusterservice/cluster.go
TNStorestate before adding services.TNStoresinNormalStateare processed.3 files
db_test.go
Enhance and add tests for merging and checkpointingpkg/vm/engine/tae/db/test/db_test.go
TestDeleteAndMerge,TestMergeAndTransfer,TestS3TransferInMerge, andTestDedup5.ForceGlobalCheckpointinstead ofGlobalCheckpoint.defaultGlobalCheckpointTimeoutconstant.workspace_test.go
Added tests for transaction operations and rollback scenarios.pkg/vm/engine/test/workspace_test.go
operations.
reader_test.go
Renamed function and updated tests in `tableReader`.pkg/cdc/reader_test.go
restartFunctoresetWatermarkFuncintableReader.WatermarkUpdaterinitialization in tests.1 files
util.go
Updated SQL utility functions and removed unused import.pkg/sql/compile/util.go
types.dropTableBeforeDropDatabase.101 files
mysql_sql.go
...pkg/sql/parsers/dialect/mysql/mysql_sql.go
...
pipeline.pb.go
...pkg/pb/pipeline/pipeline.pb.go
...
query.pb.go
...pkg/pb/query/query.pb.go
...
vector_test.go
...pkg/container/vector/vector_test.go
...
bind_insert.go
...pkg/sql/plan/bind_insert.go
...
eval_expr_util.go
...pkg/sql/util/eval_expr_util.go
...
query_builder.go
...pkg/sql/plan/query_builder.go
...
s3writer.go
...pkg/sql/colexec/multi_update/s3writer.go
...
compile.go
...pkg/sql/compile/compile.go
...
cdc.go
...pkg/frontend/cdc.go
...
authenticate.go
...pkg/frontend/authenticate.go
...
sinker.go
...pkg/cdc/sinker.go
...
func_mo_explain_phy_test.go
...pkg/sql/plan/function/func_mo_explain_phy_test.go
...
pitr_test.go
...pkg/frontend/pitr_test.go
...
external.go
...pkg/sql/colexec/external/external.go
...
session.go
...pkg/frontend/session.go
...
deepcopy.go
...pkg/sql/plan/deepcopy.go
...
s3_fs_test.go
...pkg/fileservice/s3_fs_test.go
...
baseTemplate.go
...pkg/sql/plan/function/baseTemplate.go
...
sinker_test.go
...pkg/cdc/sinker_test.go
...
bind_update.go
...pkg/sql/plan/bind_update.go
...
password_management.go
...pkg/frontend/password_management.go
...
publication_subscription.go
...pkg/frontend/publication_subscription.go
...
join.go
...pkg/sql/colexec/dedupjoin/join.go
...
operator.go
...pkg/sql/compile/operator.go
...
txn.go
...pkg/vm/engine/disttae/txn.go
...
db.go
...pkg/vm/engine/disttae/db.go
...
snapshot.go
...pkg/frontend/snapshot.go
...
apply_indices.go
...pkg/sql/plan/apply_indices.go
...
opt_misc.go
...pkg/sql/plan/opt_misc.go
...
build_constraint_util.go
...pkg/sql/plan/build_constraint_util.go
...
local_disttae_datasource.go
...pkg/vm/engine/disttae/local_disttae_datasource.go
...
mysql_cmd_executor.go
...pkg/frontend/mysql_cmd_executor.go
...
update_test.go
...pkg/sql/colexec/multi_update/update_test.go
...
rows_iter.go
...pkg/vm/engine/disttae/logtailreplay/rows_iter.go
...
upgrade_test.go
...pkg/bootstrap/versions/v2_0_1/upgrade_test.go
...
cdc_test.go
...pkg/frontend/cdc_test.go
...
s3writer_util.go
...pkg/sql/colexec/multi_update/s3writer_util.go
...
mock_trace_test.go
...pkg/util/export/mock_trace_test.go
...
bind_delete.go
...pkg/sql/plan/bind_delete.go
...
vector.go
...pkg/container/vector/vector.go
...
ddl.go
...pkg/sql/compile/ddl.go
...
txn_test.go
...pkg/frontend/txn_test.go
...
fulltext.go
...pkg/sql/colexec/table_function/fulltext.go
...
password_management_test.go
...pkg/frontend/password_management_test.go
...
multi_update.go
...pkg/sql/colexec/multi_update/multi_update.go
...
build_insert.go
...pkg/sql/plan/build_insert.go
...
insert.go
...pkg/sql/colexec/multi_update/insert.go
...
pitr.go
...pkg/frontend/pitr.go
...
mock_batch_pipe_test.go
...pkg/util/export/mock_batch_pipe_test.go
...
strhashmap.go
...pkg/common/hashmap/strhashmap.go
...
remoterun.go
...pkg/sql/compile/remoterun.go
...
dml_context.go
...pkg/sql/plan/dml_context.go
...
join_test.go
...pkg/sql/colexec/dedupjoin/join_test.go
...
hashmap_util.go
...pkg/sql/colexec/hashmap_util/hashmap_util.go
...
build_ddl.go
...pkg/sql/plan/build_ddl.go
...
shuffle.go
...pkg/sql/plan/shuffle.go
...
stats.go
...pkg/sql/plan/stats.go
...
value_scan.go
...pkg/sql/colexec/value_scan/value_scan.go
...
make.go
...pkg/sql/plan/make.go
...
s3_fs.go
...pkg/fileservice/s3_fs.go
...
operator.go
...pkg/txn/client/operator.go
...
publication_subscription_test.go
...pkg/frontend/publication_subscription_test.go
...
waiter_queue_test.go
...pkg/lockservice/waiter_queue_test.go
...
tuple.go
...pkg/container/types/tuple.go
...
constant_fold.go
...pkg/sql/plan/rule/constant_fold.go
...
disk_cache.go
...pkg/fileservice/disk_cache.go
...
fifo.go
...pkg/fileservice/fifocache/fifo.go
...
grafana_dashboard_fs.go
...pkg/util/metric/v2/dashboard/grafana_dashboard_fs.go
...
reader.go
...pkg/cdc/reader.go
...
issue_test.go
...pkg/tests/issues/issue_test.go
...
fulltext_tokenize.go
...pkg/sql/colexec/table_function/fulltext_tokenize.go
...
aws_sdk_v2.go
...pkg/fileservice/aws_sdk_v2.go
...
injects.go
...pkg/objectio/injects.go
...
build.go
...pkg/sql/plan/build.go
...
insert_test.go
...pkg/sql/colexec/multi_update/insert_test.go
...
scope.go
...pkg/sql/compile/scope.go
...
txn_table.go
...pkg/vm/engine/disttae/txn_table.go
...
bind_load.go
...pkg/sql/plan/bind_load.go
...
mock_aggr_test.go
...pkg/util/export/mock_aggr_test.go
...
lock_table_allocator.go
...pkg/lockservice/lock_table_allocator.go
...
engine_mock.go
...pkg/frontend/test/engine_mock.go
...
service_remote.go
...pkg/lockservice/service_remote.go
...
session_test.go
...pkg/frontend/session_test.go
...
logtail_consumer.go
...pkg/vm/engine/disttae/logtail_consumer.go
...
cron_task.go
...pkg/util/metric/mometric/cron_task.go
...
base.go
...pkg/vm/engine/tae/tables/base.go
...
partition_state.go
...pkg/vm/engine/disttae/logtailreplay/partition_state.go
...
types.go
...pkg/frontend/types.go
...
lock_op.go
...pkg/sql/colexec/lockop/lock_op.go
...
functionTools.go
...pkg/container/vector/functionTools.go
...
pubsub.go
...pkg/bootstrap/versions/v2_0_0/pubsub.go
...
variables.go
...pkg/frontend/variables.go
...
join.go
...pkg/sql/colexec/right/join.go
...
delete_test.go
...pkg/sql/colexec/multi_update/delete_test.go
...
server_query.go
...pkg/cnservice/server_query.go
...
checkpoint.go
...pkg/vm/engine/tae/db/gc/v3/checkpoint.go
...
snapshot.go
...pkg/vm/engine/tae/logtail/snapshot.go
...
simple.go
...pkg/monlp/tokenizer/simple.go
...
types.go
...pkg/sql/colexec/dedupjoin/types.go
...
Additional 593 files not shown
...Additional 593 files not shown
...
1 files
buffer_pipe_test.go
...pkg/util/trace/impl/motrace/buffer_pipe_test.go
...