Wrong "username" parameter in Snowflake sample configuration file example

[jmrprieto]

===============
Agent (v7.29.1)
===============

  Status date: 2021-07-21 10:01:40.128 UTC (1626861700128)
  Agent start: 2021-07-21 10:01:00.103 UTC (1626861660103)
  Pid: 378
  Go Version: go1.15.11
  Python Version: 3.8.10
  Build arch: amd64
  Agent flavor: agent
  Check Runners: 4
  Log Level: info

  Paths
  =====
    Config File: /etc/datadog-agent/datadog.yaml
    conf.d: /etc/datadog-agent/conf.d
    checks.d: /etc/datadog-agent/checks.d

  Clocks
  ======
    NTP offset: 107.715ms
    System time: 2021-07-21 10:01:40.128 UTC (1626861700128)

  Host Info
  =========
    bootTime: 2021-07-21 05:50:29 UTC (1626846629000)
    kernelArch: x86_64
    kernelVersion: 5.8.0-59-generic
    os: linux
    platform: ubuntu
    platformFamily: debian
    platformVersion: 21.04
    procs: 173
    uptime: 4h10m43s
    virtualizationRole: guest
    virtualizationSystem: docker

  Hostnames
  =========
    hostname: flask
    socket-fqdn: d03fa0e12212
    socket-hostname: d03fa0e12212
    hostname provider: container
    unused hostname providers:
      aws: not retrieving hostname from AWS: the host is not an ECS instance and other providers already retrieve non-default hostnames
      azure: azure_hostname_style is set to 'os'
      configuration/environment: hostname is empty
      gce: unable to retrieve hostname from GCE: Get "http://169.254.169.254/computeMetadata/v1/instance/hostname": dial tcp 169.254.169.254:80: i/o timeout (Client.Timeout exceeded while awaiting headers)

  Metadata
  ========
    hostname_source: container

=========
Collector
=========

  Running Checks
  ==============

    cpu
    ---
      Instance ID: cpu [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/cpu.d/conf.yaml.default
      Total Runs: 2
      Metric Samples: Last Run: 9, Total: 11
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 0s
      Last Execution Date : 2021-07-21 10:01:26 UTC (1626861686000)
      Last Successful Execution Date : 2021-07-21 10:01:26 UTC (1626861686000)


    disk (4.3.0)
    ------------
      Instance ID: disk:e5dffb8bef24336f [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/disk.d/conf.yaml.default
      Total Runs: 2
      Metric Samples: Last Run: 232, Total: 464
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 42ms
      Last Execution Date : 2021-07-21 10:01:33 UTC (1626861693000)
      Last Successful Execution Date : 2021-07-21 10:01:33 UTC (1626861693000)


    docker
    ------
      Instance ID: docker [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/docker.d/conf.yaml.default
      Total Runs: 1
      Metric Samples: Last Run: 25, Total: 25
      Events: Last Run: 1, Total: 1
      Service Checks: Last Run: 1, Total: 1
      Average Execution Time : 17ms
      Last Execution Date : 2021-07-21 10:01:25 UTC (1626861685000)
      Last Successful Execution Date : 2021-07-21 10:01:25 UTC (1626861685000)


    file_handle
    -----------
      Instance ID: file_handle [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/file_handle.d/conf.yaml.default
      Total Runs: 2
      Metric Samples: Last Run: 5, Total: 10
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 0s
      Last Execution Date : 2021-07-21 10:01:32 UTC (1626861692000)
      Last Successful Execution Date : 2021-07-21 10:01:32 UTC (1626861692000)


    io
    --
      Instance ID: io [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/io.d/conf.yaml.default
      Total Runs: 2
      Metric Samples: Last Run: 260, Total: 340
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 1ms
      Last Execution Date : 2021-07-21 10:01:39 UTC (1626861699000)
      Last Successful Execution Date : 2021-07-21 10:01:39 UTC (1626861699000)


    load
    ----
      Instance ID: load [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/load.d/conf.yaml.default
      Total Runs: 2
      Metric Samples: Last Run: 6, Total: 12
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 1ms
      Last Execution Date : 2021-07-21 10:01:31 UTC (1626861691000)
      Last Successful Execution Date : 2021-07-21 10:01:31 UTC (1626861691000)


    memory
    ------
      Instance ID: memory [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/memory.d/conf.yaml.default
      Total Runs: 2
      Metric Samples: Last Run: 18, Total: 36
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 0s
      Last Execution Date : 2021-07-21 10:01:38 UTC (1626861698000)
      Last Successful Execution Date : 2021-07-21 10:01:38 UTC (1626861698000)


    network (2.1.2)
    ---------------
      Instance ID: network:e0204ad63d43c949 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/network.d/conf.yaml.default
      Total Runs: 2
      Metric Samples: Last Run: 55, Total: 110
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 13ms
      Last Execution Date : 2021-07-21 10:01:30 UTC (1626861690000)
      Last Successful Execution Date : 2021-07-21 10:01:30 UTC (1626861690000)


    ntp
    ---
      Instance ID: ntp:d884b5186b651429 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/ntp.d/conf.yaml.default
      Total Runs: 1
      Metric Samples: Last Run: 1, Total: 1
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 1, Total: 1
      Average Execution Time : 179ms
      Last Execution Date : 2021-07-21 10:01:11 UTC (1626861671000)
      Last Successful Execution Date : 2021-07-21 10:01:11 UTC (1626861671000)


    snowflake (3.0.2)
    -----------------
      Instance ID: snowflake:5fb59b59abbcf162 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/snowflake.d/conf.yml
      Total Runs: 1
      Metric Samples: Last Run: 0, Total: 0
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 1, Total: 1
      Average Execution Time : 13.417s
      Last Execution Date : 2021-07-21 10:01:24 UTC (1626861684000)
      Last Successful Execution Date : 2021-07-21 10:01:24 UTC (1626861684000)
      metadata:
        version.major: 5
        version.minor: 28
        version.patch: 0
        version.raw: 5.28.0
        version.scheme: semver


    uptime
    ------
      Instance ID: uptime [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/uptime.d/conf.yaml.default
      Total Runs: 2
      Metric Samples: Last Run: 1, Total: 2
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 0s
      Last Execution Date : 2021-07-21 10:01:37 UTC (1626861697000)
      Last Successful Execution Date : 2021-07-21 10:01:37 UTC (1626861697000)

========
JMXFetch
========

  Information
  ==================
  Initialized checks
  ==================
    no checks

  Failed checks
  =============
    no checks

=========
Forwarder
=========

  Transactions
  ============
    Clusters: 0
    CronJobs: 0
    DaemonSets: 0
    Deployments: 0
    Dropped: 0
    DroppedOnInput: 0
    Jobs: 0
    Nodes: 0
    Pods: 0
    ReplicaSets: 0
    Requeued: 0
    Retried: 0
    RetryQueueSize: 0
    Services: 0

  Transaction Successes
  =====================
    Total number: 7
    Successes By Endpoint:
      check_run_v1: 2
      intake: 3
      series_v1: 2

  API Keys status
  ===============
    API key ending with 76b51: API Key valid

==========
Endpoints
==========
  https://app.datadoghq.com - API Key ending with:
      - 76b51

==========
Logs Agent
==========


  Logs Agent is not running

=========
APM Agent
=========
  Status: Running
  Pid: 373
  Uptime: 39 seconds
  Mem alloc: 8,019,960 bytes
  Hostname: flask
  Receiver: 0.0.0.0:8126
  Endpoints:
    https://trace.agent.datadoghq.com

  Receiver (previous minute)
  ==========================
    No traces received in the previous minute.
    Default priority sampling rate: 100.0%

  Writer (previous minute)
  ========================
    Traces: 0 payloads, 0 traces, 0 events, 0 bytes
    Stats: 0 payloads, 0 stats buckets, 0 bytes

=========
Aggregator
=========
  Checks Metric Sample: 1,049
  Dogstatsd Metric Sample: 255
  Event: 2
  Events Flushed: 2
  Number Of Flushes: 2
  Series Flushed: 405
  Service Check: 22
  Service Checks Flushed: 21
=========
DogStatsD
=========
  Event Packets: 0
  Event Parse Errors: 0
  Metric Packets: 254
  Metric Parse Errors: 0
  Service Check Packets: 0
  Service Check Parse Errors: 0
  Udp Bytes: 19,477
  Udp Packet Reading Errors: 0
  Udp Packets: 196
  Uds Bytes: 0
  Uds Origin Detection Errors: 0
  Uds Packet Reading Errors: 0
  Uds Packets: 0
  Unterminated Metric Errors: 0

Agent deployed as docker image on Ubuntu 20 LTS

Steps to reproduce the issue:

1. Use the provided example Snowflake configuration file as is in https://github.com/DataDog/integrations-core/blob/master/snowflake/datadog_checks/snowflake/data/conf.yaml.example
2. Get error message
   Python Check Loader: could not configure check instance for python check snowflake: could not invoke 'snowflake' python check constructor. New constructor API returned: Traceback (most recent call last): File "/opt/datadog-agent/embedded/lib/python3.8/site-packages/datadog_checks/snowflake/check.py", line 39, in __init__ self._config = Config(self.instance) File "/opt/datadog-agent/embedded/lib/python3.8/site-packages/datadog_checks/snowflake/config.py", line 51, in __init__ raise ConfigurationError('Must specify a user') datadog_checks.base.errors.ConfigurationError: Must specify a user Deprecated constructor API returned: Traceback (most recent call last): File "/opt/datadog-agent/embedded/lib/python3.8/site-packages/datadog_checks/snowflake/check.py", line 39, in __init__ self._config = Config(self.instance) File "/opt/datadog-agent/embedded/lib/python3.8/site-packages/datadog_checks/snowflake/config.py", line 51, in __init__ raise ConfigurationError('Must specify a user') datadog_checks.base.errors.ConfigurationError: Must specify a user
3. Change the parameter "username" by "user" and retry. This time the connection succeeds.

Describe the results you received:
I got a quite cryptic error message.

Describe the results you expected:
I connection stablished to my Snowflake instance.

Additional information you deem important (e.g. issue happens only occasionally):

[FlorianVeaux]

Hey @jmrprieto
I think you used the wrong version of the conf.yaml.example file for your agent version. You are using Agent 7.29 so you should be looking at this version of the example file: https://github.com/DataDog/integrations-core/blob/7.29.0/snowflake/datadog_checks/snowflake/data/conf.yaml.example

If you look inside the agent package on your host, the agent ships the correct version of the file /etc/datadog-agent/conf.d/snowflake.d/conf.yaml.example for your version of the snowflake integration (with user and not username)

What happened is that the very latest snowflake integration now recommends "username" for consistency with other integrations (but continue to support user in the config for backwards compatibility). The change was introduced here cc @ian28223
Older version of the snowflake integration only supported "user", so the example config of newer snowflake integration versions cannot be used with older versions of that integration.

As a generic guidance, always look at the conf.yaml.example file that is shipped with your agent to know what configuration options are available to your version of the integration.

Hope it clears things up :)

[jmrprieto]

Hey @jmrprieto
I think you used the wrong version of the conf.yaml.example file for your agent version. You are using Agent 7.29 so you should be looking at this version of the example file: https://github.com/DataDog/integrations-core/blob/7.29.0/snowflake/datadog_checks/snowflake/data/conf.yaml.example

If you look inside the agent package on your host, the agent ships the correct version of the file /etc/datadog-agent/conf.d/snowflake.d/conf.yaml.example for your version of the snowflake integration (with user and not username)

What happened is that the very latest snowflake integration now recommends "username" for consistency with other integrations (but continue to support user in the config for backwards compatibility). The change was introduced here cc @ian28223
Older version of the snowflake integration only supported "user", so the example config of newer snowflake integration versions cannot be used with older versions of that integration.

As a generic guidance, always look at the conf.yaml.example file that is shipped with your agent to know what configuration options are available to your version of the integration.

Hope it clears things up :)

Definitely I used the wrong configuration file and I should've used the example provided with the agent itself. The problem is that this is not clearly stated in the configuration steps I followed [https://docs.datadoghq.com/integrations/snowflake/] here. So it may be a good idea to clarify that document a bit further?

Thanks for the quick reply anyway.