Telegraf after upgrade from 1.9.5 to 1.10.4 - missing VMware dsname values.

[radekhuda]

Relevant telegraf.conf:

# Global tags can be specified here in key="value" format.
[global_tags]
  # dc = "us-east-1" # will tag all metrics with dc=us-east-1
  # rack = "1a"
  ## Environment variables can be used as tags, and throughout the config file
  # user = "$USER"


# Configuration for telegraf agent
[agent]
  ## Default data collection interval for all inputs
  interval = "10s"
  ## Rounds collection interval to 'interval'
  ## ie, if interval="10s" then always collect on :00, :10, :20, etc.
  round_interval = true

  ## Telegraf will send metrics to outputs in batches of at most
  ## metric_batch_size metrics.
  ## This controls the size of writes that Telegraf sends to output plugins.
  metric_batch_size = 10000

  ## For failed writes, telegraf will cache metric_buffer_limit metrics for each
  ## output, and will flush this buffer on a successful write. Oldest metrics
  ## are dropped first when this buffer fills.
  ## This buffer only fills when writes fail to output plugin(s).
  metric_buffer_limit = 1000000

  ## Collection jitter is used to jitter the collection by a random amount.
  ## Each plugin will sleep for a random time within jitter before collecting.
  ## This can be used to avoid many plugins querying things like sysfs at the
  ## same time, which can have a measurable effect on the system.
  collection_jitter = "0s"

  ## Default flushing interval for all outputs. Maximum flush_interval will be
  ## flush_interval + flush_jitter
  flush_interval = "5s"
  ## Jitter the flush interval by a random amount. This is primarily to avoid
  ## large write spikes for users running a large number of telegraf instances.
  ## ie, a jitter of 5s and interval 10s means flushes will happen every 10-15s
  flush_jitter = "0s"

  ## By default or when set to "0s", precision will be set to the same
  ## timestamp order as the collection interval, with the maximum being 1s.
  ##   ie, when interval = "10s", precision will be "1s"
  ##       when interval = "250ms", precision will be "1ms"
  ## Precision will NOT be used for service inputs. It is up to each individual
  ## service input to set the timestamp at the appropriate precision.
  ## Valid time units are "ns", "us" (or "µs"), "ms", "s".
  precision = ""

  ## Logging configuration:
  ## Run telegraf with debug log messages.
  debug = true
  ## Run telegraf in quiet mode (error log messages only).
  quiet = false
  ## Specify the log file name. The empty string means to log to stderr.
  logfile = "/var/log/telegraf/telegraf.log"

  ## Override default hostname, if empty use os.Hostname()
  hostname = ""
  ## If set to true, do no set the "host" tag in the telegraf agent.
  omit_hostname = false
###############################################################################
#                            OUTPUT PLUGINS                                   #
###############################################################################

# Configuration for sending metrics to InfluxDB
[[outputs.influxdb]]
  ## The full HTTP or UDP URL for your InfluxDB instance.
  ##
  ## Multiple URLs can be specified for a single cluster, only ONE of the
  ## urls will be written to each interval.
  # urls = ["unix:///var/run/influxdb.sock"]
  # urls = ["udp://127.0.0.1:8089"]
   urls = ["http://127.0.0.1:8086"]

  ## The target database for metrics; will be created as needed.
   database = "VC-ANT"

  ## If true, no CREATE DATABASE queries will be sent.  Set to true when using
  ## Telegraf with a user without permissions to create databases or when the
  ## database already exists.
  # skip_database_creation = false

  ## Name of existing retention policy to write to.  Empty string writes to
  ## the default retention policy.  Only takes effect when using HTTP.
  # retention_policy = ""

  ## Write consistency (clusters only), can be: "any", "one", "quorum", "all".
  ## Only takes effect when using HTTP.
  # write_consistency = "any"

  ## Timeout for HTTP messages.
   timeout = "5s"

  ## HTTP Basic Auth
   username = "*****"
   password = "*****"

# # Read metrics from VMware vCenter
 [[inputs.vsphere]]
#   ## List of vCenter URLs to be monitored. These three lines must be uncommented
#   ## and edited for the plugin to work.
   vcenters = [ "https://pv1vmwvc1001.csin.cz/sdk" ]
   username = "*****"
   password = "*******"
   interval = "20s"
#
#   ## VMs
#   ## Typical VM metrics (if omitted or empty, all metrics are collected)
#vm_include = [ "/*/vm/**"]
   vm_include = [ "/**"]
   vm_metric_include = [
	"cpu.usage.average",
	"cpu.ready.summation",
	"cpu.costop.summation",
	"mem.usage.average",
	"mem.swapped.average",
	"net.bytesTx.average",
	"net.bytesRx.average",
	"virtualDisk.totalWriteLatency.average",
	"virtualDisk.totalReadLatency.average",
	"disk.numberReadAveraged.average",
	"disk.numberWriteAveraged.average",
	"disk.read.average",
	"disk.write.average",
	"sys.osUptime.latest",
   ]

#   #vm_metric_exclude = [] ## Nothing is excluded by default
#   # vm_instances = true ## true by default
#
#   ## Hosts
#   ## Typical host metrics (if omitted or empty, all metrics are collected)
   host_include = [ "/**"]
   host_metric_include = [
	"cpu.usage.average",
        "cpu.ready.summation",
        "cpu.costop.summation",
	"mem.usage.average",
	"net.bytesRx.average",
	"net.bytesTx.average",
	"net.droppedRx.summation",
	"net.droppedTx.summation",
	"disk.kernelReadLatency.average",
	"disk.kernelWriteLatency.average",
	"datastore.sizeNormalizedDatastoreLatency.average",
        "disk.read.average",
        "disk.write.average",
	"disk.numberReadAveraged.average",
        "disk.numberWriteAveraged.average",
	"sys.uptime.latest",
   ]
#   # host_metric_exclude = [] ## Nothing excluded by default
#   # host_instances = true ## true by default
#
#   ## Clusters
    cluster_include = [ "/**"]
    cluster_metric_include = ["cpu.usagemhz.average","cpu.usage.average","mem.consumed.average",] ## if omitted or empty, all metrics are collected
#    cluster_metric_exclude = ["*"] ## Nothing excluded by default
#   # cluster_instances = true ## true by default
#
#   ## Datastores
    datastore_include = [ "/**"]
    datastore_metric_include = ["disk.used.latest","disk.capacity.latest",] ## if omitted or empty, all metrics are collected
#    datastore_metric_exclude = ["*"] ## Nothing excluded by default
#   # datastore_instances = false ## false by default for Datastores only
#
#   ## Datacenters
    datacenter_include = [ "/**"]
#   datacenter_metric_include = [] ## if omitted or empty, all metrics are collected
    datacenter_metric_exclude = [ "*" ] ## Datacenters are not collected by default.
#   # datacenter_instances = false ## false by default for Datastores only
#
#   ## Plugin Settings
#   ## separator character to use for measurement and field names (default: "_")
#    separator = "_"
#
#   ## number of objects to retreive per query for realtime resources (vms and hosts)
#   ## set to 64 for vCenter 5.5 and 6.0 (default: 256)
#    max_query_objects = 256
#
#   ## number of metrics to retreive per query for non-realtime resources (clusters and datastores)
#   ## set to 64 for vCenter 5.5 and 6.0 (default: 256)
#    max_query_metrics = 256
#
#   ## number of go routines to use for collection and discovery of objects and metrics
    collect_concurrency = 15
    discover_concurrency = 5
#
#   ## whether or not to force discovery of new objects on initial gather call before collecting metrics
#   ## when true for large environments this may cause errors for time elapsed while collecting metrics
#   ## when false (default) the first collection cycle may result in no or limited metrics while objects are discovered
    force_discover_on_init = true
#
#   ## the interval before (re)discovering objects subject to metrics collection (default: 300s)
#    object_discovery_interval = "300s"
#
#   ## timeout applies to any of the api request made to vcenter
#   # timeout = "20s"
#
#   ## Optional SSL Config
#   # ssl_ca = "/path/to/cafile"
#   # ssl_cert = "/path/to/certfile"
#   # ssl_key = "/path/to/keyfile"
#   ## Use SSL but skip chain & host verification
    insecure_skip_verify = true

System info:

OS: Ubuntu 18.04.2 LTS
telegraf: 1.10.4
VMware: vCenter 6.5

Steps to reproduce:

1. Telegraf (1.9.5)
   

2. Telegraf (1.10.4)
   

Expected behavior:

Before telegraf upgrade from 1.9.5 to 1.10.4 "dsname" values i saw datastore name.

Actual behavior:

After upgrade "dsname" values i see as "LUN". The "dsname" values are the same as "lun" values.

Additional info:

I think it's a bug.

[glinton]

What were the results of the same query used in the 1.9.5 screenshot FROM default vsphere_host_datastore WHERE dsname =~ /FX250_ANT_LOC_*/?

[radekhuda]

@glinton
The results of the query were none, because instead the datastore names there are lun id.

This screanshots are better:
Telegraf(1.9.5):

Telegraf(1.10.4):

[prydin]

@vikramraman Can you have a look at this? Seems like a regression. Please let me know if you need my help!

[prydin]

Confirmed regression. We are investigating.

[prydin]

Fixed in PR #5920

[danielnelson]

Closed in #5920

[playground-io]

Hi all, Is this issue, back again in version 1.15.1-1?
I'm having it, LUN and DSNAME have the same content.