Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(auth): use batch flush & async way to output log #1448

Merged
merged 4 commits into from
May 18, 2021
Merged

perf(auth): use batch flush & async way to output log #1448

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
d188eb7
refact: use batch flush & async way to output log
imbajin May 13, 2021
193c7f1
format align & remove mysql log(todo: replace other log4j2.xml later)
imbajin May 17, 2021
84bdae5
define capacity for char array
imbajin May 17, 2021
71f5275
unified example module
imbajin May 17, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 11 additions & 2 deletions hugegraph-core/src/main/java/com/baidu/hugegraph/auth/ResourceObject.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,8 +58,17 @@ public String toString() {
if (this.type.isAuth()) {
operated = ((AuthElement) this.operated).idString();
}
return String.format("Resource{graph=%s,type=%s,operated=%s}",
this.graph, this.type, operated);

String typeStr = this.type.toString();
String operatedStr = operated.toString();
int capacity = this.graph.length() + typeStr.length() +
operatedStr.length() + 36;

StringBuilder sb = new StringBuilder(capacity);
return sb.append("Resource{graph=").append(this.graph)
.append(",type=").append(typeStr)
.append(",operated=").append(operatedStr)
.append("}").toString();
imbajin marked this conversation as resolved.
Show resolved Hide resolved
}

public static ResourceObject<SchemaElement> of(String graph,
Expand Down
27 changes: 22 additions & 5 deletions hugegraph-dist/src/assembly/static/conf/log4j2.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,26 @@
<PatternLayout pattern="%-d{yyyy-MM-dd HH:mm:ss} %-5r [%t] [%-5p] %c %x - %m%n"/>
</Console>

<RollingFile name="file" fileName="logs/hugegraph-server.log"
filePattern="logs/$${date:yyyy-MM}/hugegraph-server-%d{yyyy-MM-dd}-%i.log">
<!-- Normal server log config -->
<RollingRandomAccessFile name="file" fileName="logs/hugegraph-server.log"
filePattern="logs/$${date:yyyy-MM}/hugegraph-server-%d{yyyy-MM-dd}-%i.log"
immediateFlush="false">
<ThresholdFilter level="TRACE" onMatch="ACCEPT" onMismatch="DENY"/>
<PatternLayout pattern="%-d{yyyy-MM-dd HH:mm:ss} %-5r [%t] [%-5p] %c %x - %m%n"/>
<SizeBasedTriggeringPolicy size="50MB"/>
</RollingFile>
</RollingRandomAccessFile>

<!-- Separate audit log, buffer size is 512KB -->
<RollingRandomAccessFile name="audit" fileName="logs/audit-hugegraph-server.log"
filePattern="logs/$${date:yyyy-MM}/audit-hugegraph-server-%d{yyyy-MM-dd}-%i.log"
bufferSize="524288" immediateFlush="false">
<ThresholdFilter level="TRACE" onMatch="ACCEPT" onMismatch="DENY"/>
<!-- Use simple format for audit log to speed up -->
<PatternLayout pattern="%-d{yyyy-MM-dd HH:mm:ss} - %m%n"/>
<SizeBasedTriggeringPolicy size="50MB"/>
</RollingRandomAccessFile>
</appenders>

<loggers>
<root level="INFO">
<appender-ref ref="file"/>
Expand All @@ -39,8 +52,12 @@
<logger name="org.apache.commons" level="INFO" additivity="false">
<appender-ref ref="file"/>
</logger>
<logger name="com.baidu.hugegraph" level="INFO" additivity="false">
<!-- Use mixed async way to output logs -->
<AsyncLogger name="com.baidu.hugegraph" level="INFO" additivity="false">
<appender-ref ref="file"/>
</logger>
</AsyncLogger>
<AsyncLogger name="com.baidu.hugegraph.auth" level="INFO" additivity="false">
<appender-ref ref="audit"/>
</AsyncLogger>
</loggers>
</configuration>
28 changes: 23 additions & 5 deletions hugegraph-dist/src/main/resources/log4j2.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,26 @@
<PatternLayout pattern="%-d{yyyy-MM-dd HH:mm:ss} %-5r [%t] [%-5p] %c %x - %m%n"/>
</Console>

<RollingFile name="file" fileName="logs/hugegraph-server.log"
filePattern="logs/$${date:yyyy-MM}/hugegraph-server-%d{yyyy-MM-dd}-%i.log">
<!-- Normal server log config -->
<RollingRandomAccessFile name="file" fileName="logs/hugegraph-server.log"
filePattern="logs/$${date:yyyy-MM}/hugegraph-server-%d{yyyy-MM-dd}-%i.log"
immediateFlush="false">
<ThresholdFilter level="TRACE" onMatch="ACCEPT" onMismatch="DENY"/>
<PatternLayout pattern="%-d{yyyy-MM-dd HH:mm:ss} %-5r [%t] [%-5p] %c %x - %m%n"/>
<SizeBasedTriggeringPolicy size="50MB"/>
</RollingFile>
</RollingRandomAccessFile>

<!-- Separate audit log, buffer size is 512KB -->
<RollingRandomAccessFile name="audit" fileName="logs/audit-hugegraph-server.log"
filePattern="logs/$${date:yyyy-MM}/audit-hugegraph-server-%d{yyyy-MM-dd}-%i.log"
bufferSize="524288" immediateFlush="false">
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we use unit like 512kb

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems it can't. There is no unit conversion inside

<ThresholdFilter level="TRACE" onMatch="ACCEPT" onMismatch="DENY"/>
<!-- Use simple format for audit log to speed up -->
<PatternLayout pattern="%-d{yyyy-MM-dd HH:mm:ss} - %m%n"/>
<SizeBasedTriggeringPolicy size="50MB"/>
</RollingRandomAccessFile>
</appenders>

<loggers>
<root level="INFO">
<appender-ref ref="console"/>
Expand Down Expand Up @@ -47,9 +60,14 @@
<appender-ref ref="console"/>
<appender-ref ref="file"/>
</logger>
<logger name="com.baidu.hugegraph" level="INFO" additivity="false">
<!-- Use mixed async way to output logs -->
<AsyncLogger name="com.baidu.hugegraph" level="INFO" additivity="false">
<appender-ref ref="console"/>
<appender-ref ref="file"/>
</logger>
</AsyncLogger>
<AsyncLogger name="com.baidu.hugegraph.auth" level="INFO" additivity="false">
<appender-ref ref="console"/>
<appender-ref ref="audit"/>
</AsyncLogger>
</loggers>
</configuration>
28 changes: 23 additions & 5 deletions hugegraph-example/src/main/resources/log4j2.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,26 @@
<PatternLayout pattern="%-d{yyyy-MM-dd HH:mm:ss} %-5r [%t] [%-5p] %c %x - %m%n"/>
</Console>

<RollingFile name="file" fileName="logs/hugegraph-example.log"
filePattern="logs/$${date:yyyy-MM}/hugegraph-example-%d{yyyy-MM-dd}-%i.log">
<!-- Normal server log config -->
<RollingRandomAccessFile name="file" fileName="logs/hugegraph-example.log"
filePattern="logs/$${date:yyyy-MM}/hugegraph-example-%d{yyyy-MM-dd}-%i.log"
immediateFlush="false">
<ThresholdFilter level="TRACE" onMatch="ACCEPT" onMismatch="DENY"/>
<PatternLayout pattern="%-d{yyyy-MM-dd HH:mm:ss} %-5r [%t] [%-5p] %c %x - %m%n"/>
<SizeBasedTriggeringPolicy size="50MB"/>
</RollingFile>
</RollingRandomAccessFile>

<!-- Separate audit log, buffer size is 512KB -->
<RollingRandomAccessFile name="audit" fileName="logs/audit-hugegraph-example.log"
filePattern="logs/$${date:yyyy-MM}/audit-hugegraph-example-%d{yyyy-MM-dd}-%i.log"
bufferSize="524288" immediateFlush="false">
<ThresholdFilter level="TRACE" onMatch="ACCEPT" onMismatch="DENY"/>
<!-- Use simple format for audit log to speed up -->
<PatternLayout pattern="%-d{yyyy-MM-dd HH:mm:ss} - %m%n"/>
<SizeBasedTriggeringPolicy size="50MB"/>
</RollingRandomAccessFile>
</appenders>

<loggers>
<root level="INFO">
<appender-ref ref="console"/>
Expand Down Expand Up @@ -47,9 +60,14 @@
<appender-ref ref="console"/>
<appender-ref ref="file"/>
</logger>
<logger name="com.baidu.hugegraph" level="INFO" additivity="false">
<!-- Use mixed async way to output logs -->
<AsyncLogger name="com.baidu.hugegraph" level="INFO" additivity="false">
<appender-ref ref="console"/>
<appender-ref ref="file"/>
</logger>
</AsyncLogger>
<AsyncLogger name="com.baidu.hugegraph.auth" level="INFO" additivity="false">
<appender-ref ref="console"/>
<appender-ref ref="audit"/>
</AsyncLogger>
</loggers>
</configuration>
28 changes: 23 additions & 5 deletions hugegraph-test/src/main/resources/log4j2.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,26 @@
<PatternLayout pattern="%-d{yyyy-MM-dd HH:mm:ss} %-5r [%t] [%-5p] %c %x - %m%n"/>
</Console>

<RollingFile name="file" fileName="logs/hugegraph-test.log"
filePattern="logs/$${date:yyyy-MM}/hugegraph-test-%d{yyyy-MM-dd}-%i.log">
<!-- Normal server log config -->
<RollingRandomAccessFile name="file" fileName="logs/hugegraph-test.log"
filePattern="logs/$${date:yyyy-MM}/hugegraph-test-%d{yyyy-MM-dd}-%i.log"
immediateFlush="false">
<ThresholdFilter level="TRACE" onMatch="ACCEPT" onMismatch="DENY"/>
<PatternLayout pattern="%-d{yyyy-MM-dd HH:mm:ss} %-5r [%t] [%-5p] %c %x - %m%n"/>
<SizeBasedTriggeringPolicy size="50MB"/>
</RollingFile>
</RollingRandomAccessFile>

<!-- Separate audit log, buffer size is 512KB -->
<RollingRandomAccessFile name="audit" fileName="logs/audit-hugegraph-test.log"
filePattern="logs/$${date:yyyy-MM}/audit-hugegraph-test-%d{yyyy-MM-dd}-%i.log"
bufferSize="524288" immediateFlush="false">
<ThresholdFilter level="TRACE" onMatch="ACCEPT" onMismatch="DENY"/>
<!-- Use simple format for audit log to speed up -->
<PatternLayout pattern="%-d{yyyy-MM-dd HH:mm:ss} - %m%n"/>
<SizeBasedTriggeringPolicy size="50MB"/>
</RollingRandomAccessFile>
</appenders>

<loggers>
<root level="INFO">
<appender-ref ref="console"/>
Expand Down Expand Up @@ -47,9 +60,14 @@
<appender-ref ref="console"/>
<appender-ref ref="file"/>
</logger>
<logger name="com.baidu.hugegraph" level="INFO" additivity="false">
<!-- Use mixed async way to output logs -->
<AsyncLogger name="com.baidu.hugegraph" level="INFO" additivity="false">
<appender-ref ref="console"/>
<appender-ref ref="file"/>
</logger>
</AsyncLogger>
<AsyncLogger name="com.baidu.hugegraph.auth" level="INFO" additivity="false">
<appender-ref ref="console"/>
<appender-ref ref="audit"/>
</AsyncLogger>
</loggers>
</configuration>