Add data size class adhering to the correct SI and IEC prefixes (drop…

…wizard#2686)

The `Size` and `SizeUnit` classes in Dropwizard are using a common but incorrect definition of data size units, in which 1 kilobyte equals 1024 bytes.

The `DataSize` and `DataSizeUnit` classes are using the correct definitions according to SI and IEC, in which 1 kilobyte equals 1000 bytes and 1 kibibytes equals 1024 bytes.

Instead of modifying the existing `Size` and `SizeUnit` classes, this change set introduces a new set of classes, `DataSize` and `DataSizeUnit`, with the possibility to convert the old units into the new, correct units.

Example:
```
assertEquals(Size.parse("128 kb").toDataSize(), DataSize.kibibytes(128L));
```

This is a breaking change and has to be properly documented in the release notes and upgrade notes of Dropwizard.

Closes dropwizard#2152
Closes dropwizard#2154

docs/source/manual/configuration.rst

@@ -661,7 +661,7 @@ File
           archivedFileCount: 5
           timeZone: UTC
           logFormat: "%-5p [%d{ISO8601,UTC}] %c: %m%n%rEx"
-          bufferSize: 8KB
+          bufferSize: 8KiB
           immediateFlush: true
           filterFactories:
             - type: URI
@@ -690,9 +690,10 @@ archivedLogFilenamePattern   (none)                                     Required
 archivedFileCount            5                                          The number of archived files to keep. Must be greater than or equal to ``0``. Zero is a
                                                                         special value signifying to keep infinite logs (use with caution)
 maxFileSize                  (unlimited)                                The maximum size of the currently active file before a rollover is triggered. The value can be
-                                                                        expressed in bytes, kilobytes, megabytes, gigabytes, and terabytes by appending B, K, MB, GB, or
-                                                                        TB to the numeric value.  Examples include 100MB, 1GB, 1TB.  Sizes can also be spelled out, such
-                                                                        as 100 megabytes, 1 gigabyte, 1 terabyte.
+                                                                        expressed in bytes, kibibytes, kilobytes, mebibytes, megabytes, gibibytes, gigabytes, tebibytes,
+                                                                        terabytes, pebibytes, and petabytes by appending B, KiB, KB, MiB, MB, GiB, GB, TiB, TB, PiB, or PB
+                                                                        to the numeric value.  Examples include 5KiB, 100MiB, 1GiB, 1TB.  Sizes can also be spelled out, such
+                                                                        as 5 kibibytes, 100 mebibytes, 1 gibibyte, 1 terabyte.
 totalSizeCap                 (unlimited)                                Controls the total size of all files.
                                                                         Oldest archives are deleted asynchronously when the total size cap is exceeded.
 timeZone                     UTC                                        The time zone to which event timestamps will be converted.
@@ -703,8 +704,8 @@ filterFactories              (none)                                     The list
                                                                         the threshold.
 neverBlock                   false                                      Prevent the wrapping asynchronous appender from blocking when its underlying queue is full.
                                                                         Set to true to disable blocking.
-bufferSize                   8KB                                        The buffer size of the underlying FileAppender (setting added in logback 1.1.10). Increasing this
-                                                                        from the default of 8KB to 256KB is reported to significantly reduce thread contention.
+bufferSize                   8KiB                                       The buffer size of the underlying FileAppender (setting added in logback 1.1.10). Increasing this
+                                                                        from the default of 8KiB to 256KiB is reported to significantly reduce thread contention.
 immediateFlush               true                                       If set to true, log events will be immediately flushed to disk. Immediate flushing is safer, but
                                                                         it degrades logging throughput.
 ============================ =========================================  ==================================================================================================
@@ -769,7 +770,7 @@ TCP
           port: 4560
           connectionTimeout: 500ms
           immediateFlush: true
-          sendBufferSize: 8KB
+          sendBufferSize: 8KiB
 
 
 ============================ =============  ==================================================================
@@ -780,7 +781,7 @@ port                         4560           The port on which the TCP server is
 connectionTimeout            500ms          The timeout to connect to the TCP server.
 immediateFlush               true           If set to true, log events will be immediately send to the server
                                             Immediate flushing is safer, but it degrades logging throughput.
-sendBufferSize               8KB            The buffer size of the underlying SocketAppender.
+sendBufferSize               8KiB           The buffer size of the underlying SocketAppender.
                                             Takes into effect if immediateFlush is disabled.
 ============================ =============  ==================================================================
 

dropwizard-benchmarks/src/main/java/io/dropwizard/benchmarks/util/DataSizeBenchmark.java

@@ -0,0 +1,39 @@
+package io.dropwizard.benchmarks.util;
+
+import io.dropwizard.util.DataSize;
+import org.openjdk.jmh.annotations.Benchmark;
+import org.openjdk.jmh.annotations.BenchmarkMode;
+import org.openjdk.jmh.annotations.Mode;
+import org.openjdk.jmh.annotations.OutputTimeUnit;
+import org.openjdk.jmh.annotations.Scope;
+import org.openjdk.jmh.annotations.State;
+import org.openjdk.jmh.runner.Runner;
+import org.openjdk.jmh.runner.options.OptionsBuilder;
+
+import java.util.concurrent.TimeUnit;
+
+@BenchmarkMode(Mode.AverageTime)
+@OutputTimeUnit(TimeUnit.NANOSECONDS)
+@State(Scope.Benchmark)
+public class DataSizeBenchmark {
+
+    /**
+     * Don't trust the IDE, it's advisedly non-final to avoid constant folding
+     */
+    private String size = "256KiB";
+
+    @Benchmark
+    public DataSize parseSize() {
+        return DataSize.parse(size);
+    }
+
+    public static void main(String[] args) throws Exception {
+        new Runner(new OptionsBuilder()
+                .include(DataSizeBenchmark.class.getSimpleName())
+                .forks(1)
+                .warmupIterations(5)
+                .measurementIterations(5)
+                .build())
+                .run();
+    }
+}

dropwizard-example/example.yml

@@ -76,7 +76,7 @@ logging:
       archivedLogFilenamePattern: /tmp/application-%d{yyyy-MM-dd}-%i.log.gz
       archivedFileCount: 7
       timeZone: UTC
-      maxFileSize: 10MB
+      maxFileSize: 10MiB
 
 # the key needs to match the configuration key of the renderer (ViewRenderer::getConfigurationKey)
 viewRendererConfiguration:

dropwizard-example/src/test/resources/test-example.yml

@@ -25,5 +25,5 @@ logging:
       archivedLogFilenamePattern: './logs/application-%d-%i.log.gz'
       archive: true
       archivedFileCount: 7
-      maxFileSize: '1mb'
+      maxFileSize: '1MiB'
 

dropwizard-jetty/src/main/java/io/dropwizard/jetty/GzipHandlerFactory.java

@@ -1,7 +1,7 @@
 package io.dropwizard.jetty;
 
 import com.fasterxml.jackson.annotation.JsonProperty;
-import io.dropwizard.util.Size;
+import io.dropwizard.util.DataSize;
 import org.eclipse.jetty.server.Handler;
 import org.eclipse.jetty.server.handler.gzip.GzipHandler;
 
@@ -36,12 +36,12 @@
  *     <tr>
  *         <td>{@code minimumEntitySize}</td>
  *         <td>256 bytes</td>
- *         <td>All response entities under this size are not compressed.</td>
+ *         <td>All response entities under this DataSize are not compressed.</td>
  *     </tr>
  *     <tr>
  *         <td>{@code bufferSize}</td>
  *         <td>8KiB</td>
- *         <td>The size of the buffer to use when compressing.</td>
+ *         <td>The DataSize of the buffer to use when compressing.</td>
  *     </tr>
  *     <tr>
  *         <td>{@code excludedUserAgentPatterns}</td>
@@ -96,10 +96,10 @@ public class GzipHandlerFactory {
     private boolean enabled = true;
 
     @NotNull
-    private Size minimumEntitySize = Size.bytes(256);
+    private DataSize minimumEntitySize = DataSize.bytes(256);
 
     @NotNull
-    private Size bufferSize = Size.kilobytes(8);
+    private DataSize bufferSize = DataSize.kibibytes(8);
 
     // By default compress responses for all user-agents
     private Set<String> excludedUserAgentPatterns = new HashSet<>();
@@ -138,22 +138,22 @@ public void setEnabled(boolean enabled) {
     }
 
     @JsonProperty
-    public Size getMinimumEntitySize() {
+    public DataSize getMinimumEntitySize() {
         return minimumEntitySize;
     }
 
     @JsonProperty
-    public void setMinimumEntitySize(Size size) {
+    public void setMinimumEntitySize(DataSize size) {
         this.minimumEntitySize = requireNonNull(size);
     }
 
     @JsonProperty
-    public Size getBufferSize() {
+    public DataSize getBufferSize() {
         return bufferSize;
     }
 
     @JsonProperty
-    public void setBufferSize(Size size) {
+    public void setBufferSize(DataSize size) {
         this.bufferSize = requireNonNull(size);
     }
 

dropwizard-jetty/src/main/java/io/dropwizard/jetty/HttpConnectorFactory.java

@@ -3,11 +3,11 @@
 import com.codahale.metrics.MetricRegistry;
 import com.fasterxml.jackson.annotation.JsonProperty;
 import com.fasterxml.jackson.annotation.JsonTypeName;
+import io.dropwizard.util.DataSize;
+import io.dropwizard.util.DataSizeUnit;
 import io.dropwizard.util.Duration;
-import io.dropwizard.util.Size;
-import io.dropwizard.util.SizeUnit;
+import io.dropwizard.validation.MinDataSize;
 import io.dropwizard.validation.MinDuration;
-import io.dropwizard.validation.MinSize;
 import io.dropwizard.validation.PortRange;
 import org.eclipse.jetty.http.HttpCompliance;
 import org.eclipse.jetty.io.ArrayByteBufferPool;
@@ -228,48 +228,48 @@ public static ConnectorFactory admin() {
     private boolean inheritChannel = false;
 
     @NotNull
-    @MinSize(128)
-    private Size headerCacheSize = Size.bytes(512);
+    @MinDataSize(128)
+    private DataSize headerCacheSize = DataSize.bytes(512);
 
     @NotNull
-    @MinSize(value = 8, unit = SizeUnit.KILOBYTES)
-    private Size outputBufferSize = Size.kilobytes(32);
+    @MinDataSize(value = 8, unit = DataSizeUnit.KIBIBYTES)
+    private DataSize outputBufferSize = DataSize.kibibytes(32);
 
     @NotNull
-    @MinSize(value = 1, unit = SizeUnit.KILOBYTES)
-    private Size maxRequestHeaderSize = Size.kilobytes(8);
+    @MinDataSize(value = 1, unit = DataSizeUnit.KIBIBYTES)
+    private DataSize maxRequestHeaderSize = DataSize.kibibytes(8);
 
     @NotNull
-    @MinSize(value = 1, unit = SizeUnit.KILOBYTES)
-    private Size maxResponseHeaderSize = Size.kilobytes(8);
+    @MinDataSize(value = 1, unit = DataSizeUnit.KIBIBYTES)
+    private DataSize maxResponseHeaderSize = DataSize.kibibytes(8);
 
     @NotNull
-    @MinSize(value = 1, unit = SizeUnit.KILOBYTES)
-    private Size inputBufferSize = Size.kilobytes(8);
+    @MinDataSize(value = 1, unit = DataSizeUnit.KIBIBYTES)
+    private DataSize inputBufferSize = DataSize.kibibytes(8);
 
     @NotNull
     @MinDuration(value = 1, unit = TimeUnit.MILLISECONDS)
     private Duration idleTimeout = Duration.seconds(30);
 
     @NotNull
-    @MinSize(0)
-    private Size minResponseDataPerSecond = Size.bytes(0);
+    @MinDataSize(0)
+    private DataSize minResponseDataPerSecond = DataSize.bytes(0);
 
     @NotNull
-    @MinSize(0)
-    private Size minRequestDataPerSecond = Size.bytes(0);
+    @MinDataSize(0)
+    private DataSize minRequestDataPerSecond = DataSize.bytes(0);
 
     @NotNull
-    @MinSize(value = 1, unit = SizeUnit.BYTES)
-    private Size minBufferPoolSize = Size.bytes(64);
+    @MinDataSize(value = 1, unit = DataSizeUnit.BYTES)
+    private DataSize minBufferPoolSize = DataSize.bytes(64);
 
     @NotNull
-    @MinSize(value = 1, unit = SizeUnit.BYTES)
-    private Size bufferPoolIncrement = Size.bytes(1024);
+    @MinDataSize(value = 1, unit = DataSizeUnit.BYTES)
+    private DataSize bufferPoolIncrement = DataSize.bytes(1024);
 
     @NotNull
-    @MinSize(value = 1, unit = SizeUnit.BYTES)
-    private Size maxBufferPoolSize = Size.kilobytes(64);
+    @MinDataSize(value = 1, unit = DataSizeUnit.BYTES)
+    private DataSize maxBufferPoolSize = DataSize.kibibytes(64);
 
     @Min(value = 1, payload = Unwrapping.Unwrap.class)
     private Optional<Integer> acceptorThreads = Optional.empty();
@@ -320,52 +320,52 @@ public void setInheritChannel(boolean inheritChannel) {
     }
 
     @JsonProperty
-    public Size getHeaderCacheSize() {
+    public DataSize getHeaderCacheSize() {
         return headerCacheSize;
     }
 
     @JsonProperty
-    public void setHeaderCacheSize(Size headerCacheSize) {
+    public void setHeaderCacheSize(DataSize headerCacheSize) {
         this.headerCacheSize = headerCacheSize;
     }
 
     @JsonProperty
-    public Size getOutputBufferSize() {
+    public DataSize getOutputBufferSize() {
         return outputBufferSize;
     }
 
     @JsonProperty
-    public void setOutputBufferSize(Size outputBufferSize) {
+    public void setOutputBufferSize(DataSize outputBufferSize) {
         this.outputBufferSize = outputBufferSize;
     }
 
     @JsonProperty
-    public Size getMaxRequestHeaderSize() {
+    public DataSize getMaxRequestHeaderSize() {
         return maxRequestHeaderSize;
     }
 
     @JsonProperty
-    public void setMaxRequestHeaderSize(Size maxRequestHeaderSize) {
+    public void setMaxRequestHeaderSize(DataSize maxRequestHeaderSize) {
         this.maxRequestHeaderSize = maxRequestHeaderSize;
     }
 
     @JsonProperty
-    public Size getMaxResponseHeaderSize() {
+    public DataSize getMaxResponseHeaderSize() {
         return maxResponseHeaderSize;
     }
 
     @JsonProperty
-    public void setMaxResponseHeaderSize(Size maxResponseHeaderSize) {
+    public void setMaxResponseHeaderSize(DataSize maxResponseHeaderSize) {
         this.maxResponseHeaderSize = maxResponseHeaderSize;
     }
 
     @JsonProperty
-    public Size getInputBufferSize() {
+    public DataSize getInputBufferSize() {
         return inputBufferSize;
     }
 
     @JsonProperty
-    public void setInputBufferSize(Size inputBufferSize) {
+    public void setInputBufferSize(DataSize inputBufferSize) {
         this.inputBufferSize = inputBufferSize;
     }
 
@@ -380,52 +380,52 @@ public void setIdleTimeout(Duration idleTimeout) {
     }
 
     @JsonProperty
-    public Size getMinBufferPoolSize() {
+    public DataSize getMinBufferPoolSize() {
         return minBufferPoolSize;
     }
 
     @JsonProperty
-    public void setMinBufferPoolSize(Size minBufferPoolSize) {
+    public void setMinBufferPoolSize(DataSize minBufferPoolSize) {
         this.minBufferPoolSize = minBufferPoolSize;
     }
 
     @JsonProperty
-    public Size getBufferPoolIncrement() {
+    public DataSize getBufferPoolIncrement() {
         return bufferPoolIncrement;
     }
 
     @JsonProperty
-    public void setBufferPoolIncrement(Size bufferPoolIncrement) {
+    public void setBufferPoolIncrement(DataSize bufferPoolIncrement) {
         this.bufferPoolIncrement = bufferPoolIncrement;
     }
 
     @JsonProperty
-    public Size getMaxBufferPoolSize() {
+    public DataSize getMaxBufferPoolSize() {
         return maxBufferPoolSize;
     }
 
     @JsonProperty
-    public void setMaxBufferPoolSize(Size maxBufferPoolSize) {
+    public void setMaxBufferPoolSize(DataSize maxBufferPoolSize) {
         this.maxBufferPoolSize = maxBufferPoolSize;
     }
 
     @JsonProperty
-    public Size getMinResponseDataPerSecond() {
+    public DataSize getMinResponseDataPerSecond() {
         return minResponseDataPerSecond;
     }
 
     @JsonProperty
-    public void setMinResponseDataPerSecond(Size minResponseDataPerSecond) {
+    public void setMinResponseDataPerSecond(DataSize minResponseDataPerSecond) {
         this.minResponseDataPerSecond = minResponseDataPerSecond;
     }
 
     @JsonProperty
-    public Size getMinRequestDataPerSecond() {
+    public DataSize getMinRequestDataPerSecond() {
         return minRequestDataPerSecond;
     }
 
     @JsonProperty
-    public void setMinRequestDataPerSecond(Size minRequestDataPerSecond) {
+    public void setMinRequestDataPerSecond(DataSize minRequestDataPerSecond) {
         this.minRequestDataPerSecond = minRequestDataPerSecond;
     }