perf(auth): use batch flush & async way to output log #1448
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
More details refer to [doc](xx), here are some points: 1. change log4j2 appender to `RandomAccessFile` to improve perf (default is BufferedOutputStream) 2. change log4j2 to not flush immediately (default is true) 3. use bigger (512K) buffer size (default is 4k) 4. use (LMAX Disruptor) async way to improve log perf (default is sync) 5. separate audit log & server log to keep sever log clean & readable 6. use simple layout format for audit log 7. use `StringBuilder` to reduce about `9%` cost for audit log 8. record mysql connection url info for user issue
Codecov Report
@@ Coverage Diff @@
## master #1448 +/- ##
============================================
+ Coverage 59.68% 62.24% +2.55%
- Complexity 5715 5861 +146
============================================
Files 381 381
Lines 32139 32147 +8
Branches 4504 4504
============================================
+ Hits 19183 20009 +826
+ Misses 10968 10085 -883
- Partials 1988 2053 +65 Continue to review full report at Codecov.
|
javeme
reviewed
May 17, 2021
hugegraph-core/src/main/java/com/baidu/hugegraph/auth/ResourceObject.java
Outdated
Show resolved
Hide resolved
hugegraph-mysql/src/main/java/com/baidu/hugegraph/backend/store/mysql/MysqlSessions.java
Outdated
Show resolved
Hide resolved
javeme
reviewed
May 17, 2021
| <!-- Separate audit log, buffer size is 512KB --> | ||
| <RollingRandomAccessFile name="audit" fileName="logs/audit-hugegraph-server.log" | ||
| filePattern="logs/$${date:yyyy-MM}/audit-hugegraph-server-%d{yyyy-MM-dd}-%i.log" | ||
| bufferSize="524288" immediateFlush="false"> |
There was a problem hiding this comment.
Seems it can't. There is no unit conversion inside
hugegraph-core/src/main/java/com/baidu/hugegraph/auth/ResourceObject.java
Show resolved
Hide resolved
hugegraph-core/src/main/java/com/baidu/hugegraph/auth/ResourceObject.java
Outdated
Show resolved
Hide resolved
javeme
previously approved these changes
May 17, 2021
zhoney
approved these changes
May 18, 2021
javeme
approved these changes
May 18, 2021
Linary
approved these changes
May 18, 2021
javeme
changed the title
|
some logs of v0.11 server with log-level=debug(before optimize): blocked threads: |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
More perf details refer to doc, here are some points, for
log4j2:RandomAccessFileto improve perf (default is BufferedOutputStream)Others:
StringBuilderto reduce about9%cost for audit log