FR: VAULT: Expose Configuration Option to Disable Vault Version Check

[adawalli]

Nomad version

Nomad v0.12.0 (8f7fbc8e7b5a4ed0d0209968faf41b238e6d5817)

Issue

At the group level in a job spec, Nomad will check the vault version. In the default case, this is a good idea. However, some customer choose to restrict the Version info from the /health endpoints. Obfuscating the Vault version is just one more way to prevent information an attacker might need if your Vault endpoint is publicly exposed.

Unfortunately, this will cause a jobspec to fail even when a perfectly working Vault instance is up and running. Even more interestingly, is that Nomad has already

• Talked to the /sys/init /sys/health endpoint successfully (thanks to Skip checking Vault health #8524)
• Validated the Server token
• Successfully renewed it's token (and posted the next renewal time period in the logs)

all before deciding that it won't fetch a token for a client against a vault policy.

This means you will the following error

nomad job plan whoami.hcl                                                                      
+/- Job: "testing"                                                                                
+/- Stop: "true" => "false"                                                                       
+/- Task Group: "tester" (1 create)                                                               
  +/- Task: "whoami" (forces in-place update)                                                     
    + Constraint {                                                                                
      + LTarget: "${attr.vault.version}"                                                          
      + Operand: "regexp"                                                                         
      + RTarget: ".*"                                                                             
      }                                                                                           
                                                                                                  
Scheduler dry-run:                                                                                
- WARNING: Failed to place all allocations.                                                       
  Task Group "tester" (failed to place 1 allocation):                                             
    * Constraint "${attr.vault.version} semver >= 0.6.1": 1 nodes excluded by filter    

Now, in my case, I can thankfully add

    constraint {
      attribute = "${attr.vault.version}"
      operator = "is_not_set"
    }

At the group level and bypass this limitation, but I think it would be clearer for the users to expose a config option (which by default, represents current behavior) that allows us to disable this constraint check.

Thoughts?

[notnoop]

Hi @adawalli ! Thanks for raising the issue. We'd like to get more input, as we haven't run into this scenario in the past, and not being able to detect the Vault version may hinder us from coping with backward incompatible changes in Vault effectively. I'm glad that you found an alternative.

Would love to get more prospective on the motivation behind masking the value and what threat model you are considering here. Curious how you are masking the version and what is it set at? Did you considering setting an arbitrary large version (e.g. 99.99.99)? I also realize that these decisions might be out of your control ;-).

[adawalli]

I don't have any information on the thread model, but the last time I ask our OPs folks about this, they pointed this out as an example
https://github.com/hashicorp/vault/blob/master/CHANGELOG.md#142-may-21st-2020

I was also told
It's common practice to avoid exposing versions of software being run on public services to avoid vulnerability scans and potential targeted attacks

As far as the Version hiding, I believe it's done through some gateway magic, but I am not sure offhand.

[YohannHammad]

Hello, any news here ?