Loggly

The Logback Loggly extension enables Logback to post messages to Loggly, a cloud logging service.

Check out Loggly's Java Logback logging documentation to learn more.

Usage

Add the following .jar to your application's classpath:

Jar	Maven or Ant+Ivy
logback-ext-loggly-version.jar	<dependency> <groupId>org.logback-extensions</groupId> <artifactId>logback-ext-loggly</artifactId> <version>version</version> <scope>runtime</scope> </dependency>

Pattern-formatted Messages

Simple logback.xml configuration:

<configuration>
    <appender name="logglyAppender" class="ch.qos.logback.ext.loggly.LogglyAppender">
        <endpointUrl>https://logs-01.loggly.com/bulk/TOKEN/tag/TAGS</endpointUrl>
    </appender>
    ...
</configuration>

By default the Loggly Appender uses a PatternLayout to control log event formatting. The default pattern normalizes timestamps to UTC time, which is usually best for clustered environments that process requests from around the world. The default pattern is:

%d{"yyyy-MM-dd'T'HH:mm:ss.SSS'Z'",UTC} %-5level [%thread] %logger: %m%n

If this is not suitable, you can configure the pattern directly on the appender:

<configuration>
    <appender name="logglyAppender" class="ch.qos.logback.ext.loggly.LogglyAppender">
        <endpointUrl>https://logs-01.loggly.com/bulk/TOKEN/tag/TAGS</endpointUrl>
        <pattern>%date %-5level [%thread] %logger{36} %m%n</pattern>
    </appender>
    ...
</configuration>

JSON Messages

Loggly enables some powerful features when you send it JSON. To enable JSON messages:

<configuration>
    <appender name="logglyAppender" class="ch.qos.logback.ext.loggly.LogglyAppender">
        <endpointUrl>https://logs-01.loggly.com/bulk/TOKEN/tag/TAGS</endpointUrl>
        <layout class="ch.qos.logback.contrib.json.classic.JsonLayout">
            <jsonFormatter class="ch.qos.logback.contrib.jackson.JacksonJsonFormatter">
                <!-- prettyPrint is probably ok in dev, but usually not ideal in production: -->
                <prettyPrint>false</prettyPrint>
            </jsonFormatter>
        </layout>
    </appender>
    ...
</configuration>

You will also need to add the following jars to your application's classpath:

Jar	Maven or Ant+Ivy
logback-json-classic-version.jar	<dependency> <groupId>ch.qos.logback.contrib</groupId> <artifactId>logback-json-classic</artifactId> <version>version</version> <scope>runtime</scope> </dependency>
logback-jackson-version.jar	<dependency> <groupId>ch.qos.logback.contrib</groupId> <artifactId>logback-jackson</artifactId> <version>version</version> <scope>runtime</scope> </dependency>

JsonFormatter

As you can see, we've chosen to use the Jackson JSON Processor (from logback-contrib) in the above example. You may provide your own JsonFormatter implementation if you do not want to use Jackson. A JsonFormatter must be specified to direct which processor should be used at runtime since the JDK does not provide one by default.

Timestamp Formatting

By default the JsonLayout will NOT format the log event's timestamp. It will just print the timestamp as a long (milliseconds since epoch). If you want a nicely formatted timestamp string, you can specify the timestamp format (as understood by SimpleDateFormat) and time zone ID (as understood by the JDK TimeZone class). If a timestamp format is specified, but a time zone ID is not specified, the system default time zone will be assumed.

Example config normalizing timestamps to UTC time:

<configuration>
    <appender name="logglyAppender" class="ch.qos.logback.ext.loggly.LogglyAppender">
        <endpointUrl>https://logs-01.loggly.com/bulk/TOKEN/tag/TAGS</endpointUrl>
        <layout class="ch.qos.logback.contrib.json.classic.JsonLayout">
            <jsonFormatter class="ch.qos.logback.contrib.jackson.JacksonJsonFormatter">
                <!-- prettyPrint is probably ok in dev, but usually not ideal in production: -->
                <prettyPrint>false</prettyPrint>
            </jsonFormatter>
            <timestampFormat>yyyy-MM-dd'T'HH:mm:ss.SSSZ</timestampFormat>
            <timestampFormatTimezoneId>UTC</timestampFormatTimezoneId>
        </layout>
    </appender>
    ...
</configuration>

Batched Mode

Note: Loggly's Syslog API is much more scalable than the HTTP API which should mostly be used in low-volume or non-production systems. The HTTP API can be very convenient to workaround firewalls

If the LogglyBatchAppender saturates and discards log messages, the following warning message is appended to both Loggly and Java's System.err: $date - OutputStream is full, discard previous logs.

Configuration settings

Property Name	Type	Description
inputKey	String	Loggly input key. Either inputKey or endpointUrl is required. Sample: 12345678-90ab-cdef-1234-567890abcdef
endpointUrl	String	Loggly HTTP API endpoint URL. Either inputKey or endpointUrl is required. By specifying endpointUrl, the user has the option of turning HTTPS on or off for debugging purposes. Otherwise, the plugin will default to sending logs via HTTPS. Sample: https://logs.loggly.com/bulk/12345678-90ab-cdef-1234-567890abcdef
pattern	String	Pattern used for Loggly log messages. Default value is: %d{"yyyy-MM-dd'T'HH:mm:ss.SSS'Z'",UTC} %-5level [%thread] %logger: %m%n
proxyHost	String	hostname of a proxy server. If blank, no proxy is used.
proxyPort	int	port of a proxy server. Must be a valid int but is ignored if proxyHost is blank or null.
jmxMonitoring	boolean	Enable registration of a monitoring MBean named ch.qos.logback:type=LogglyBatchAppender,name=LogglyBatchAppender@#hashcode#. Default: true.
maxNumberOfBuckets	int	Max number of buckets of in the byte buffer. Default value: 8.
maxBucketSizeInKilobytes	int	Max size of each bucket. Default value: 1024 Kilobytes (1Mo).
flushIntervalInSeconds	int	Interval of the buffer flush to Loggly API. Default value: 3.

Performance Notes

• The Default configuration consumes up to 8 buffers of 1024 Kilobytes (1Mo) each which seemed very reasonable even for small JVMs.
• If logs are discarded, try first to shorten the flushIntervalInSeconds parameter to 2 or even 1 so that the buffers are flushed before they are filled.

Configuration Sample

<configuration scan="true" scanPeriod="30 seconds" debug="true">
  <if condition='isDefined("logback.loggly.inputKey")'>
    <then>
      <appender name="loggly" class="ch.qos.logback.ext.loggly.LogglyBatchAppender">
        <inputKey>${logback.loggly.inputKey}</inputKey>
        <pattern>%d{yyyy/MM/dd HH:mm:ss,SSS} [${HOSTNAME}] [%thread] %-5level %logger{36} - %m %throwable{5}%n</pattern>
        <proxyHost>${logback.loggly.proxy.host:-}</proxyHost>
        <proxyPort>${logback.loggly.proxy.port:-8080}</proxyPort>
        <debug>${logback.loggly.debug:-false}</debug>
      </appender>
      <root level="WARN">
        <appender-ref ref="loggly"/>
      </root>
    </then>
  </if>
</configuration>