6.1.2 / 2018-04-05

Docker, Windows, Linux, Mac OSX

Download links

• macOS DMG installer
• Windows MSI installer

Bug Fixes

• Fix some edge cases where flare could contain secrets if the secrets where encapsulated in quotes.

6.1.1

Docker, Windows, Linux

Download links

• Windows MSI installer

Bug Fixes

• Fix a crash in the docker check when collecting sizes on an image
  with no repository tags.
• Fixes bug on Windows where, if configuration options are specified
  on the installation command line, invalid proxy options are set.
• Removed the read timeout for UDP connections causing the agent to
  stop forwarding logs after one minute of nonactivity.
• Updating the data type of the CPU of the task and the metadata name
  for Version to Revision.

Other Notes

• Add environment variable DD_ENABLE_GOHAI for setting option
  enable_gohai when running in a container.

6.1.0

Docker, Windows, Linux, Mac OSX

Download links

• macOS DMG installer
• Windows MSI installer

New Features

• Add Agent Version to flare form

• Add the DD_CHECK_RUNNERS environment variable binding

• Add the status command to the DCA.

• Docker check: ignore the new exec_die event type by default

• Extract the swarm_namespace tag for docker swarm containers, in addition
  to the already present swarm_service tag.

• Allow configuration of the enabled-state of process, logs, and apm to be
  specified on the installation command line for Windows.

• Add a jmx_use_cgroup_memory_limit option to set jmxfetch to use cgroup
  memory limits when calculating its heap size. It is enabled by default
  in the docker image.

• Add option to extract kubernetes pod annotations as tags, similar to labels

• Added an environment variable DD_LOGS_CONFIG_CONTAINER_COLLECT_ALL to enable logs tailing on all containers.

• Adding the 'bind_host' option to configure the interface to bind by dogstatsd and JMX.

• Support setting tags as a YAML array in the logs agent integration configuration

Bug Fixes

• Fix docker memory metrics parsing from cgroup files

• Fix docker.mem.in_use metric computation

• When using the import script, change the group owner of configuration files to the dd-agent user.

• Fix a false positive in the collector-queue healthcheck

• The old docker_daemon check is now properly converted in the "import" command by default

• Docker check: fix event filtering for exec events

• Improve docker monitoring when the system is under a very high load. The agent
  might still temporarily miss a healthcheck, but will be able to run already
  scheduled checks, and recover once the spike ends

• Fixes the container startup on Fargate, where we tried and remove the same
  file twice, failing hard (stopping) on the second attempt.

• Fix flare failing on zipping individual components

• Fixed an issue where the import script would put an empty histogram aggregates and percentiles in datadog.yaml if they didn't exist in datadog.conf.

• Fix the build for platforms not supporting Gohai.

• Fixes flaw where Windows Performance counters were not properly initialized
  on non EN-US versions of windows

• Menu in system tray reports wrong version (6.0.0) for all versions of Agent. This fixes the system tray menu to report the correct version.

• Fixing clear passwords in "config-check.log" when sending a flare.

• Allow network proxy settings set on the Windows installation command
  line to be set in the registry, where they'll be translated to the
  configuration

• Accept now short names for docker image in logs configuration file and added to the possibilty to filter containers by image name with Kubernetes.

• Fixes an issue that would prevent the agent from stopping when it was tailing logs
  of a container that had no logs.

• fixes an issue with wildcard tailing of logs files on windows

• Allow Linux package uninstallation to proceed without errors even on platforms
  that aren't supported by the Agent

• Fixes agent to run on Server "Core" versions

• Changes default precision of pdh-based counters from int to float. Fixes bug where fidelity of some counters is quite low, especially counters with values between 0 and 1.

• Adds back the removed system.mem.usable metric for Agents running on Windows.

• Avoid multiple initializations of the tagger subsystem

Other Notes

• Normalize support of nested config options defined with env vars.

• Make the check-rate command more visible when running "check` to get a list of metrics.

6.0.3

Docker, Windows, Linux, Mac OSX

Download links

• macOS DMG installer
• Windows MSI installer

Prelude

• This release also includes bugfixes to the process agent. See diff.

Bug Fixes

• Fixed the issue preventing from having docker tags when collecting logs from containers.
• Fix docker metrics collection on Moby Linux hosts (default Swarm AMI)

6.0.2

Docker, Windows, Linux, Mac OSX

Download links

• macOS DMG installer
• Windows MSI installer

Critical Issues

• Packaging issue in 6.0.1 resulted in the release of nightly builds for trace-agent and process-agent. 6.0.2 ships the stable intended versions.

6.0.1

Windows, Linux

Download links

There was a problem with the packaging of this release, please use 6.0.2.

Enhancements

• Add information about Log Agent checks to the GUI General Status page.

Bug Fixes

• Run the service mapper on all the agents running the apiserver check. Exit before running the rest of the check if the agent is not the leader.

• Fixing docker network metrics collection for the docker check and the process agent on some network configurations.

• Replaces the system.mem.free metric with gopsutil's 'available' and splits the windows and linux memory checks. Previously this reported with a value of 0 and system.mem.used was reporting the same as system.mem.total

• ".pdh" suffix was added to system.io metrics on windows for side-by-side
  testing when changed the collection mechanism, and inadvertently left.

• Fix bug where global tags for PDH based python checks are not read
  correctly from the configuration yaml.

• IE does not support String.prototype.endsWith, add implementation to the
  string prototype to enable the functionality.

• remove .pdh suffix from system.io.wkb_s, system.io_w_s, system.io.rkb_s,
  system.io.r_s, system.io.avg_q_sz

• Fix GUI for JMX checks, they are now manageable from the web UI.

• Fix the launch of JMXFetch on windows and make multiplatform treatment of
  the launch more robust.

6.0.0

Docker, Windows, Linux, Mac OSX

Download links

• macOS DMG installer
• Windows MSI installer

Prelude

The Datadog Agent 6.0 release is a complete rewrite of Agent 5 written in the Go language with the goal of improving performance, reducing resource utilization, and allowing for check concurrency. Python based checks and plugins will continue to be supported by 6.x without need for modification. While this is a major release, backwards compatibility was a strong focus, and in most cases this should be a simple upgrade process.

Highlights from this release include:

• Increased throughput for DogstatsD metrics.
• Live Process Monitoring support for Windows
• Support for the upcoming Datadog Log Management offering.
• Beta support for global distributions and percentiles for custom metrics.
• Native support for ingestion of metrics via the prometheus exposition format.
• Windows and OSX now have a configuration UI to simplify local check and agent configuration management.

Other Features and Improvements

• The Agent can find relevant listeners on its host by using the "auto" listener, for now only docker is supported. More information can be found here
• APM / Process / Log agents can be enabled/disabled via the DD_*_ENABLED environment variables, see the agent docker image readme for details
• short_image tag has been added the to Docker tagger collector
• Make the number of workers used by the forwarder configurable and set default to 1.
• The agent now has internal healthchecks on all subsystems. The result is exposed via the agent health command and used in the docker image as a healthcheck. The /probe.sh wrapper is provided for compatibility with agent5 and future-proofing.
• Add jmx_custom_jars option to make sure they are loaded by jmxfetch
• Docker_daemon is now a core agent check named docker.
• Kubernetes Integration has been replaced by a few Integrations. See here for more information.
• Support tagging on Nomad 0.6.0+ clusters
• Support for additional metrics and metadata from kubernetes API server, as well as the addition of a new Control Plane status check.
• Kubernetes events now include additional formatting for improved readability as Datadog events.
• APM now utilizes apm_non_local_traffic = true to listen on the network
• Docker image: APM is now disabled by default, you need to set DD_APM_ENABLED=true to run the trace agent.
• Agent now comes bundled with a Web GUI that listens by default on (configurable) port 5002. Only available on localhost, more information about this can be found here
• Added a windows systray icon. System tray icon includes menu options for starting/stopping/restarting services, creating a flare, and launching the browser-based configuration GUI.

Upgrade Notes

• Full upgrade instructions can be found here.
• Custom checks (located by default on Linux in /etc/datadog-agent/checks.d/) now have a lower precedence than the checks that are bundled with the Agent. This means that a custom check with the same name as a bundled check will now be ignored, and the bundled check will be loaded instead. If you want to override a bundled check with a custom check, please use a new name for your custom check, and use that new name for the related yaml configuration file.
• Tags in the DD_TAGS environment variable are now separated by spaces instead of commas in agent5
• Process and Trace Agents can be enabled in datadog.yaml by using

apm_config: 
  enabled:true
process_config: 
  enabled:true

• The Windows Agent no longer has an application GUI and has been moved to a Web based GUI accompanied by a systray icon that can be used to handle core functionality of the Agent.
• The Agent’s configuration file has been changed from datadog.conf to datadog.yaml An example of the available options can be seen here
• Datadog no longer offers an Alpine based container image. RHEL and Debian based images are available on Docker Hub and in the Docker Store. Images now use the S6 light init system to start and manage agent processes.
• APM now listens to localhost only by default, you need to set apm_config.apm_non_local_traffic = true to enable listening on the network

Deprecation Notes

• For breaking changes between Agent 5 and this Agent, please see our deprecations and changes page

Major items include:

• Agent configuration is no longer in datadog.conf, and has moved to datadog.yaml. The datadog-agent import command will migrate your configuration files to the new form.
• The execution of the main agent, trace-agent (APM), and process-agent is now orchestrated using systemd/upstart facilities on Linux. SysV-init scripts are no longer included.
• Modification of the installed checks directory (default on Linux was /opt/datadog-agent/checks.d/ has been removed and changed to an embedded wheels format. More information about this can be found here
• Some libraries that custom checks may rely on have been removed. Please see here for a list of the removed dependencies.
• Custom Emitters are no longer supported as of 6.0

Deprecated Agent Integrations

• The Jenkins Integration has been deprecated and removed in favor of a Jenkins Plugin instead
• The standalone Mesos Integration has been retired, in favor of the newer mesos_master and mesos_slave integrations..
• The standalone HDFS Integration has been retired and removed, please use hdfs_namenode and hdfs_datanode instead.

6.0.0-rc.4 / 2018-02-23

Linux, Docker, macOS and Windows release

Download links

• macOS DMG installer
• Windows MSI installer

Changes

Enhancements

• Change the kubernetes leader election system to use configmaps instead of endpoints. This
  allows a simpler migration from Agent5, as Agent6 will not require additional permissions.

• Adds in the proc.queue_length and proc.count metrics with the windows version of the Agent.

Bug Fixes

• Add the windows icon to the Infrastructure List for Agents installed on Windows machines.

• Fix Docker container --pid=host operations. Previous RCs can cause host system
  instabilities and should not be run in pid host mode.

• Windows: set correct default value for apm config to enabled, so that the trace agent is
  started by default

• Removes deprecated process_agent_enabled flag

• metrics.yaml is not a "configurable" file - it provides default metrics for
  checks and shouldn't be altered. Removed from the GUI configuration file
  list.

• Windows: gopsutil calls to the CPU module require COM threading model to be
  in multi-threaded mode, to guarantee it's safe to make those calls we load
  the python checks setting the right COM concurrency mode first. Once loaded
  we clear the concurrency mode and python checks that might use COM will set
  it as they need.

• Windows: make stop/restart of DatadogAgent service stop/restart dependent
  services accordingly

• Windows: Prevent system tray icon from being displayed more than once

• Windows: Make default start behavior of process-agent consistent with Linux Agent

• Windows: Fix the item launching the web-based GUI in the systray icon menu

• Windows: Process agent service now passes the configuration file argument to the
  executable when launching - otherwise service will always come up on
  reboots.

Other Notes

• Windows: Added developer documentation regarding the caveats of the COM
  concurrency model and its implications moving forward. The current state affects
  auto-discovery and dynamic scheduling of checks.

6.0.0-rc.3 / 2018-02-22

Linux, Docker, macOS and Windows release

Download links

• macOS DMG installer
• Windows MSI installer

Changes

Enhancements

• Adds windows systray icon. System tray icon includes menu options for
  starting/stopping/restarting services, creating a flare, and launching the
  browser-based GUI.

• allow auth token path to be set in the config file

• Implementation for disabling checks from the web UI

• Agent restart message on UI, clears after restart.

• Add SSL support & label joins for the prometheus check

Bug Fixes

• Fix the command-line flag parsing regression caused by a transitive dependency importing the
  glog library. agent flags should now behave as in beta9.

• GUI had broken after the introduction of integrations as wheels this PR ensures we collect the full list of available integrations so we can enable the corresponding configurations from the UI.

• Fix an issue preventing logs-agent to tail container logs when docker API version is prior to 1.25

• Fix line miss issue that could happen when tailing new files found when scanning

• On windows Automatic services would fail to start across reboots due to a known go issue on 1.9.2: golang/go#23479
  We now start windows services as delayed start automatic services (ie. they now start automatically after all other automatic services).

Other Notes

• The OSX build of the agent does not include the containers integrations as they are only supported on Linux for now. The Windows build already excluded them since beta1

• The auth_token file, used to store the api authentication token, is now only readable/writable by the user running the agent instead of inheriting datadog.yaml permissions.

6.0.0-rc.2 / 2018-02-20

Linux, Docker, macOS and Windows release

Download links

• macOS DMG installer
• Windows MSI installer

Changes

New Features

• Add namespace configuration for metric names for dogstatsd

• Rework autodiscovery label names to be consistent, still support the
  previous names

• Ships updated integrations from integrations-core 6.0.0-rc.2, including new kubelet check

• Add envvar bindings for docker/kubernetes custom tag extraction features

Upgrade Notes

• Normal installations: APM now listens to localhost only by default, you need to set
  apm_config.apm_non_local_traffic = true to enable listening on the network

• Docker image: APM is now disabled by default, you need to set DD_APM_ENABLED=true
  to run the trace agent. It listens on all interfaces by default when running, you can
  set DD_APM_NON_LOCAL_TRAFFIC=false to only listen on localhost

Bug Fixes

• Don't try to match containers by image name if they provide an AD template
  via docker labels or pod annotations. This avoid scheduling double checks.

• Fix handling of the %%host%% and %%port%% autodiscovery tags

• The aggregator now discards metric samples with NaN values. Also solves a serializing error
  on metric payloads.

• Fixes bug whereby device tag was (correctly) removed from tags list, but
  device field was only added to the metric on the first run.

• Fix an issue unscheduling checks discovered through auto-discovery

• Upstart would indefinitely respawn trace and process agents even when exiting
  with a zero status code. We now explicitly define exit code 0 as a valid exit
  code to prevent respawn when the agents are disabled.

• Fix cases where empty host tags in the Agent datadog.yaml configuration caused
  the host metadata payload parsing to fail in the backend.

• Fix resources metadata collector so that its payload is correctly parsed in the
  backend even when empty.

• Make sure we don't get stuck if the API server does not return events.

• make tagger more resilient to malformed docker events

• Removing vsphere and sqlserver from the blacklist. The former is
  available on all platforms, sqlserver is currently windows-only.

Other Notes

• The apm.yaml.default config file was removed on linux and the
  trace-agent.conf.example was removed on every other platform.

• Only enable the resources metadata collector on Linux by default, to match
  Agent 5's behavior.