Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Release 1.12.3 #506

Closed
neverchanje opened this issue Mar 30, 2020 · 2 comments
Closed

Release 1.12.3 #506

neverchanje opened this issue Mar 30, 2020 · 2 comments
Labels
1.12.3 release-note Notes on the version release

Comments

@neverchanje
Copy link
Contributor

neverchanje commented Mar 30, 2020 •
edited
Loading

1.12.3-RC1

rDSN

PR (48 TOTAL) TITLE
XiaoMi/rdsn#430 refactor(backup): delay the removal of checkpoint files produced by cold backup
XiaoMi/rdsn#431 refactor: move log-block-writing-related codes from mutation_log to log_block
XiaoMi/rdsn#429 feat(dup): support multiple fail modes for duplication
XiaoMi/rdsn#427 refactor: move log_block class from mutation_log.h to separated file
XiaoMi/rdsn#428 fix(backup): use https to access fds, instead of http
XiaoMi/rdsn#425 fix(dup): multiple fixes on replica duplication
XiaoMi/rdsn#424 feat(dup): add dsn_replica_dup_test to CI testing and fix http outputs
XiaoMi/rdsn#421 feat: add a new app_env to limit scan time
XiaoMi/rdsn#423 feat(dup): rename change_dup_status to modify_dup
XiaoMi/rdsn#422 fix: fix memory leak in tls_transient_memory_t
XiaoMi/rdsn#416 feat: add query_disk_info api for shell command
XiaoMi/rdsn#411 feat(dup): handle non-idempotent writes during duplication
XiaoMi/rdsn#409 feat: add query_disk_info support for replica_server
XiaoMi/rdsn#420 refactor: delete unused log in replica::on_config_sync
XiaoMi/rdsn#399 feat(split): re-implement function split_replica_exec
XiaoMi/rdsn#415 feat(dup): support replica http api for duplication state query
XiaoMi/rdsn#414 feat: forbid large-size-value writes to Pegasus
XiaoMi/rdsn#390 feat(split): child notify parent catch up
XiaoMi/rdsn#401 feat(dup): support HTTP API to inspect duplication state
XiaoMi/rdsn#403 refactor(utils): make logger decoupled from dsn runtime
XiaoMi/rdsn#406 fix: weak constraint of ipv4 and incorrect code logic in hostname_from_ip
XiaoMi/rdsn#407 feat(dup): respond with hint when add_dup failed
XiaoMi/rdsn#394 feat(split): add partition_version
XiaoMi/rdsn#393 feat(dup): add metrics for duplication
XiaoMi/rdsn#398 refactor: make clock decoupled from rdsn runtime
XiaoMi/rdsn#319 feat(split): child replica apply private logs, in-memory mutations and catch up parent
XiaoMi/rdsn#395 fix(utils): fix bug in time_utils
XiaoMi/rdsn#388 refactor(util): make time_utils decoupled from utils.cpp
XiaoMi/rdsn#387 fix(thrift): add start_time_ns for structure mutation_update in thrift file
XiaoMi/rdsn#384 fix(lsan): memory leak in perf_counter_atomic.h
XiaoMi/rdsn#353 feat: add check for app envs
XiaoMi/rdsn#380 refactor(network): refine codes and comments
XiaoMi/rdsn#381 fix(asan): fix heap-use-after-free in replica_split_test
XiaoMi/rdsn#379 refactor: remove useless files (version.h, coredump.win.cpp, ReleaseNote)
XiaoMi/rdsn#377 fix(asan): heap-buffer-overflow in meta_test_base.h
XiaoMi/rdsn#375 fix(asan): heap-use-after-free in simple_lb_cure_test.cpp
XiaoMi/rdsn#374 feat(dup): add is_duplicating API on replication_app_base
XiaoMi/rdsn#371 refactor: remove aspects for dsn runtime
XiaoMi/rdsn#370 feat(dup): implement load_mutation
XiaoMi/rdsn#355 feat(dup): preserve data consistency during replica learn
XiaoMi/rdsn#365 refactor: move load balance related tests under another test suite
XiaoMi/rdsn#361 fix(asan): pointer used after free in autoref_ptr_test
XiaoMi/rdsn#363 fix(asan): global-buffer-overflow in string_conv_test
XiaoMi/rdsn#362 refactor: remove unused function in greedy_load_balancer
XiaoMi/rdsn#360 fix: sanitizer options don't take effect
XiaoMi/rdsn#359 build: add sanitizer support
XiaoMi/rdsn#356 feat(dup): implement is_duplicating
XiaoMi/rdsn#348 refactor: refine replica restart by adding func get_replay_start_decree
XiaoMi/rdsn#320 feat(dup): protect private log from missing when duplication is enabled

Pegasus

PR (25 TOTAL) TITLE
#503 refactor: simplify logging in pegasus_server_impl
#498 feat: add shell command for query_disk_info
#479 feat(hotspot): new algorithm of hotspot detection
#489 fix: fix bug in del function of redis_proxy
#491 fix: fix the invalid IP address in src/server/test/config.ini
#485 feat(shell): mlog_dump support parsing check_and_set
#474 feat(split): add partition_version
#478 fix(asan): heap-use-after-free in pegasus_write_service.cpp
#481 fix: use db when it has been opened
#480 fix: add libraries missing for pegasus_client_lib.so
#444 feat(collector): add statistics for partition hotspot
#472 docs(community): add issue template for feature-request
#466 feat(dup): implement duplication related shell commands
#470 feat(shell): add dependency argh to parse options and arguments
#469 fix(rdsn): update rdsn and adapt to time_utils refactoring
#467 refactor: refactor rocksdb options usage
#463 refactor: refactor the code about prometheus in pegasus_counter_reporter
#462 refactor(config): remove duplicated config items for cluster name
#460 refactor: update rdsn and remove DSN_CORE_VERSION
#456 feat(dup): implement server handling of duplicate rpc (part 1)
#450 feat(rocksdb): Select the option of Direct-IO in Rocksdb
#455 fix: fix the bug that threads don't stop when pegasus_io_service is released
#399 feat(dup): implement pegasus_mutation_duplicator
#453 fix: retry starting zookeeper if it failed on onebox environment
#446 build: add sanitizer support

New Perf-Counters

New HTTP APIs

New App Env

Configuration Changes

[replication]
- allow_non_idempotent_write = false
+ max_allowed_write_size = 1048576 # default = 1MB

[pegasus.server]
+ rocksdb_use_direct_reads = true
+ rocksdb_use_direct_io_for_flush_and_compaction = true
+ rocksdb_compaction_readahead_size = 2097152
+ rocksdb_writable_file_max_buffer_size = 1048576
- perf_counter_cluster_name = %{cluster.name}
- perf_counter_enable_falcon = false
- perf_counter_enable_prometheus = false
+ perf_counter_sink = <falcon | prometheus>

[pegasus.collector]
- cluster = %{cluster.name}
+ hotspot_detect_algorithm = <hotspot_algo_qps_variance | hotspot_algo_qps_skew> 

[replication]
+ cluster_name = %{cluster.name}

Blame list:

New limitation of requests

Like aws dynamodb limits, we imposed read/write limitation to protect the service of Pegasus in XiaoMi. All of the limits can be reconfigured:

  max_allowed_write_size = 1048576 # default = 1MB

Please remember to check if any requests to your service exceed the limits.
@neverchanje neverchanje added the release-note Notes on the version release label Mar 30, 2020
@neverchanje
Copy link
Contributor Author

neverchanje commented Apr 10, 2020 •
edited by qinzuoyan
Loading

1.12.3-RC2

rDSN

PR (1 RELEASED) TITLE
XiaoMi/rdsn#430 refactor(backup): delay the removal of checkpoint files produced by cold backup

Pegasus

PR (6 RELEASED) TITLE
#511 fix: node qps sum inconsistent with app qps sum
#512 fix(collector): throughput statistics isn’t right
#500 feat: limit long time rocksdb iteration operation
#505 feat(collector): add throughput statistic for table
#509 fix(asan): global-buffer-overflow in function escape_sds_argv of data_operations.cpp
#508 fix(shell): disk_replica printer bug

New perf-counter

On replica-server

On collector

New configuration

[replication]
+cold_backup_checkpoint_reserve_minutes = 10

[pegasus.server]
# cluster level restriction {3000, 30MB, 1000, 30s}
+rocksdb_multi_get_max_iteration_count = 3000
+rocksdb_multi_get_max_iteration_size = 31457280
+rocksdb_max_iteration_count = 1000
+rocksdb_iteration_threshold_time_ms = 30000

Blame list:

@neverchanje
Copy link
Contributor Author

neverchanje commented Apr 14, 2020 •
edited
Loading

1.12.3-RC3

rDSN

PR (1 TOTAL) TITLE
XiaoMi/rdsn#435 feat: tcmalloc memory release improvements

Pegasus

PR (1 TOTAL) TITLE
#515 fix: fix the bug in restore_test

New perf-counter

New remote command

command help: replica.release-tcmalloc-memory <true|false>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
1.12.3 release-note Notes on the version release
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@neverchanje