Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[RIP-9] The best practice of rocketmq #659

Closed
wants to merge 7 commits into from
Closed

[RIP-9] The best practice of rocketmq #659

wants to merge 7 commits into from

Conversation

fusiming
Copy link

@fusiming fusiming commented Jan 4, 2019

What is the purpose of the change

RocketMQ Developer Guide doc

Brief changelog

Add the file named best_practice.md

Verifying this change

no functional change

Follow this checklist to help us incorporate your contribution quickly and easily. Notice, it would be helpful if you could finish the following 5 checklist(the last one is not necessary)before request the community to review your PR.

  • Make sure there is a Github issue filed for the change (usually before you start working on it). Trivial changes like typos do not require a Github issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
  • Format the pull request title like [ISSUE #123] Fix UnknownException when host config not exist. Each commit in the pull request should have a meaningful subject line and body.
  • Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
  • Write necessary unit-test(over 80% coverage) to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add integration-test in test module.
  • Run mvn -B clean apache-rat:check findbugs:findbugs checkstyle:checkstyle to make sure basic checks pass. Run mvn clean install -DskipITs to make sure unit-test pass. Run mvn clean test-compile failsafe:integration-test to make sure integration-test pass.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

@coveralls
Copy link

coveralls commented Jan 4, 2019 •
edited
Loading

Coverage Status

Coverage increased (+0.08%) to 43.205% when pulling fcb6d57 on fusiming:master into d4d02b5 on apache:master.

@vongosling vongosling changed the title Add the file named best_practice.md [DOC]Add the best practice part Jan 4, 2019
@fusiming fusiming changed the title [DOC]Add the best practice part [RIP-9]Add the best practice part Jan 16, 2019
@fusiming fusiming changed the title [RIP-9]Add the best practice part [RIP-9] The best practice of rocketmq Jan 16, 2019
@rudy2steiner
Copy link

great

wlliqipeng
wlliqipeng reviewed Jan 23, 2019
View reviewed changes
docs/cn/best_practise.md
| retryTimesWhenSendFailed | 2 | 如果消息发送失败，最大重试次数，该参数只对同步发送模式起作用 |
| maxMessageSize | 4MB | 客户端限制的消息大小，超过报错，同时服务端也会限制，所以需要跟服务端配合使用。 |
| transactionCheckListener | | 事务消息回查监听器，如果发送事务消息，必须设置 |
| checkThreadPoolMinSize | 1 | Broker回查Producer事务状态时，线程池大小 |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

说明需斟酌

wlliqipeng
wlliqipeng reviewed Jan 23, 2019
View reviewed changes
docs/cn/best_practise.md
| consumeFromWhere | CONSUME_FROM_LAST_OFFSET | Consumer启动后，默认从上次消费的位置开始消费，这包含两种情况：一种是上次消费的位置未过期，则消费从上次中止的位置进行；一种是上次消费位置已经过期，则从当前队列第一条消息开始消费 |
| consumeTimestamp | 半个小时前 | 只有当consumeFromWhere值为CONSUME_FROM_TIMESTAMP时才起作用。 |
| allocateMessageQueueStrategy | AllocateMessageQueueAveragely | Rebalance算法实现策略 |
| subscription | {} | 订阅关系 |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

{}?

wlliqipeng
wlliqipeng reviewed Jan 23, 2019
View reviewed changes
docs/cn/best_practise.md
| subscription | {} | 订阅关系 |
| messageListener | | 消息监听器 |
| offsetStore | | 消费进度存储 |
| consumeThreadMin | 10 | 消费线程池数量 |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

消费线程池最大线程数

wlliqipeng
wlliqipeng reviewed Jan 23, 2019
View reviewed changes
docs/cn/best_practise.md
| brokerSuspendMaxTimeMillis | 20000 | 长轮询，Consumer拉消息请求在Broker挂起最长时间，单位毫秒 |
| consumerTimeoutMillisWhenSuspend | 30000 | 长轮询，Consumer拉消息请求在Broker挂起超过指定时间，客户端认为超时，单位毫秒 |
| consumerPullTimeoutMillis | 10000 | 非长轮询，拉消息超时时间，单位毫秒 |
| messageModel | BROADCASTING | 消息模型，支持以下两种 1、集群消费 2、广播消费 |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

说明同push consumer

wlliqipeng
wlliqipeng reviewed Jan 23, 2019
View reviewed changes
docs/cn/best_practise.md
| messageModel | BROADCASTING | 消息模型，支持以下两种 1、集群消费 2、广播消费 |
| messageQueueListener | | 监听队列变化 |
| offsetStore | | 消费进度存储 |
| registerTopics | [] | 注册的topic集合 |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[]?

wlliqipeng
wlliqipeng reviewed Jan 23, 2019
View reviewed changes
docs/cn/best_practise.md Outdated
```
​ 消息发送成功或者失败要打印消息日志，务必要打印SendResult和key字段。send消息方法只要不抛异常，就代表发送成功。发送成功会有多个状态，在sendResult里定义。以下对每个状态进行说明：

● SEND_OK（发送成功）
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

用项目编号-

wlliqipeng
wlliqipeng reviewed Jan 23, 2019
View reviewed changes
docs/cn/best_practise.md Outdated
### 7.2.3选择oneway形式发送

​ 通常消息的发送是这样一个过程：
​ 1) 客户端发送请求到服务器
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

文字格式，下同

wlliqipeng
wlliqipeng reviewed Jan 23, 2019
View reviewed changes
docs/cn/best_practise.md

1) 建议消息的大小最好不要超过512K。

异步发送（AsyncSending） 默认方法send（msg）会在得到返回响应之前一直被阻塞。因此，如果用户重视性能，那么建议使用send（msg、callback），这样消息发送将以异步方式进行。
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

与标题无关，建议delete

wlliqipeng
wlliqipeng reviewed Jan 23, 2019
View reviewed changes
docs/cn/best_practise.md Outdated
1 提高消费并行度

​ 绝大部分消息消费行为都属于 IO 密集型，即可能是操作数据库，或者调用 RPC，这类消费行为的消费速度在于后端数据库或者外系统的吞吐量，通过增加消费并行度，可以提高总的消费吞吐量，但是并行度增加到一定程度，反而会下降。所以，应用必须要设置合理的并行度。 如下有几种修改消费并行度的方法：
​ 1) 同一个 ConsumerGroup 下，通过增加 Consumer 实例数量来提高并行度（需要注意的是超过订阅队列数的 Consumer 实例无效）。可以通过加机器，或者在已有机器启动多个进程的方式。
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

项目编号-

wlliqipeng
wlliqipeng reviewed Jan 23, 2019
View reviewed changes
docs/cn/best_practise.md Outdated
```
4 优化每条消息消费过程
举例如下，某条消息的消费过程如下：
1) 根据消息从 DB 查询【数据 1】
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

格式

wlliqipeng
wlliqipeng reviewed Jan 23, 2019
View reviewed changes
docs/cn/best_practise.md
### 7.3.3 消费打印日志

如果消息量较少，建议在消费入口方法打印消息，消费耗时等，方便后续排查问题。
​```java
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

格式

wlliqipeng
wlliqipeng reviewed Jan 23, 2019
View reviewed changes
docs/cn/best_practise.md

7 关于消费位点

​ 当建立一个新的消费者组时，需要决定是否需要消费已经存在于 Broker 中的历史消息CONSUME_FROM_LAST_OFFSET 将会忽略历史消息，并消费之后生成的任何消息。CONSUME_FROM_FIRST_OFFSET 将会消费每个存在于 Broker 中的信息。你也可以使用 CONSUME_FROM_TIMESTAMP 来消费在指定时间戳后产生的消息。
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"...于Broker中的历史消息”后面是句号

wlliqipeng
wlliqipeng reviewed Jan 23, 2019
View reviewed changes
docs/cn/best_practise.md

​ 推荐使用最新发布的JDK 1.8版本。通过设置相同的Xms和Xmx值来防止JVM调整堆大小以获得更好的性能。简单的JVM配置如下所示：
​ -server -Xms8g -Xmx8g -Xmn4g
​ 如果您不关心RocketMQ代理的启动时间，还有一种更好的选择，就是通过“预触摸”Java堆以确保在JVM初始化期间每个页面都将被分配。那些不关心启动时间的人可以启用它：
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

“RocketMQ 代理”改为Broker

wlliqipeng
wlliqipeng reviewed Jan 23, 2019
View reviewed changes
docs/cn/best_practise.md
-XX:G1ReservePercent=25
-XX:InitiatingHeapOccupancyPercent=30
```
​ 这些GC选项看起来有点激进，但事实证明它在我们的生产环境中具有良好的性能。另外不要把-XX:MaxGCPauseMillis的值设置太小，否则JVM将使用一个小的年轻代来实现这个目标，这将导致非常频繁的minor GC，所以官方建议使用rolling GC日志文件：
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

“官方”delete

wlliqipeng
wlliqipeng reviewed Jan 23, 2019
View reviewed changes
docs/cn/best_practise.md

​ os.sh脚本在bin文件夹中列出了许多内核参数，可以进行微小的更改然后用于生产用途。下面的参数需要注意，更多细节请参考/proc/sys/vm/*的文档。
（<https://www.kernel.org/doc/Documentation/sysctl/vm.txt> ）
​ vm.extra_free_kbytes，告诉VM在后台回收（kswapd）启动的阈值与直接回收（通过分配进程）的阈值之间保留额外的可用内存.RocketMQ使用此参数来避免内存分配中的长延迟。（与具体内核版本相关）
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

格式

@fusiming fusiming closed this Jan 24, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wlliqipeng wlliqipeng wlliqipeng left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@fusiming @coveralls @rudy2steiner @wlliqipeng