vault kv put on secrets engine requires read access to /sys/internal/ui/mounts/*

[meangrape]

Describe the bug
Unable to write secret via vault kv put web/foo value="bar" even with correct capabilities. vault returns an HTTP 403 error.

merlot:~ jayed$ vault kv put web/foo value=bar
Error making API request.

URL: GET http://127.0.0.1:8200/v1/sys/internal/ui/mounts/web/foo
Code: 403. Errors:

* permission denied

To Reproduce
Create a new vault installation. Mount a secrets engine on /web. Try to insert a key.

Expected behavior
I did not expect the following capabilities to required in my policy.

path "sys/internal/ui/mounts/*" {
  capabilities = ["read"]
}

Environment:

• Vault Server Version (retrieve with vault status): 0.10.0
• Vault CLI Version (retrieve with vault version): Vault v0.10.0 ('5dd7f25f5c4b541f2da62d70075b6f82771a650d')
• Server Operating System/Architecture: FreeBSD 11.1-RELEASE

Vault server configuration file(s):

disable_mlock = "true"

listener "tcp" {
  address = "127.0.0.1:8200"
  tls_disable = 1
}

listener "tcp" {
  address = "XXXXXX:8200"
  tls_cert_file = "/usr/local/etc/ssl/sentenai.net.crt"
  tls_key_file = "/usr/local/etc/ssl/sentenai.net.key"
  tls_min_version = "tls12"
  tls_client_ca_file = "/etc/ssl/sentenai-ca.crt"
}

storage "file" {
  path = "/mnt/vault/data"
}

[briankassouf]

Thanks for the report! That error message is a little misleading, it is causing a 403 because you don't have access to the web/ mount. The sys/internal/ui/mounts/* endpoint in unauthenticated (therefore does not need to be part of a policy) but will return 403 if the token doesn't have access to the specified mount.

Could you make sure your token has the proper permissions to write to web/foo and try again?

I will add this the the next milestone so we can make that error more clear.

Also are you sure the server and client are both 0.10.0? I believe this call wasn't introduced until 0.10.1

[meangrape]

Here's a GPG encrypted log of the token creation & capabilities.

Here's a screencast of my vault version info.

If I toggle this section of my policy file then my ability to read/write from the web kv store goes away.

path "sys/internal/ui/mounts/*" {
  capabilities = ["read"]
}

Here's my entire dev-policy.hcl file:

path "web" {                                                                                                                                                 
    capabilities = ["list"]                                                                                                                                  
}                                                                                                                                                            
                                                                                                                                                             
path "dev" {                                                                                                                                                 
    capabilities = ["list"]                                                                                                                                  
}

path "web/*" {
    capabilities = ["create", "update", "delete", "list", "read"]
}

path "dev/*" {
    capabilities = ["create", "update", "delete", "list", "read"]
}

path "auth/token/lookup-self" {
  capabilities = ["read"]
}

path "auth/token/renew" {
  capabilities = ["update", "create"]
}

path "auth/approle/role/observatory/secret-id" {
  capabilities = ["read", "create", "update", "list"]
}

path "sys/internal/ui/mounts/*" {
  capabilities = ["read"]
}

[briankassouf]

So that endpoint didn't exist until 0.10.1, which is why the server is returning a 403. But it's confusing why your client is even making the request since only 0.10.1 CLIs will query that endpoint. see #4430.

[jefferai]

Cannot reproduce:

$ vault policy write dev-policy - <<EOF
path "web" {                                                                                                                                                 
    capabilities = ["list"]                                                                                                                                  
}                                                                                                                                                            
                                                                                                                                                             
path "dev" {                                                                                                                                                 
    capabilities = ["list"]                                                                                                                                  
}

path "web/*" {
    capabilities = ["create", "update", "delete", "list", "read"]
}

path "dev/*" {
    capabilities = ["create", "update", "delete", "list", "read"]
}

path "auth/token/lookup-self" {
  capabilities = ["read"]
}

path "auth/token/renew" {
  capabilities = ["update", "create"]
}

path "auth/approle/role/observatory/secret-id" {
  capabilities = ["read", "create", "update", "list"]
}
> EOF
Success! Uploaded policy: dev-policy

$ vault secrets enable -path=web kv
Success! Enabled the kv secrets engine at: web

$ vault token create -policy=dev-policy
Key                Value
---                -----
token              bb1574d2-bc17-522d-a333-1d02d392f1b3
token_accessor     94e7ad7d-2bf7-117d-ed53-a9cc8fb2f668
token_duration     768h
token_renewable    true
token_policies     [default dev-policy]

$ VAULT_TOKEN=bb1574d2-bc17-522d-a333-1d02d392f1b3 vault kv put web/foo value=bar
Success! Data written to: web/foo

$ vault secrets tune -options=version=2 web
Success! Tuned the secrets engine at: web/

$ VAULT_TOKEN=bb1574d2-bc17-522d-a333-1d02d392f1b3 vault kv put web/foo value=bar
Key              Value
---              -----
created_time     2018-05-23T23:52:38.081252968Z
deletion_time    n/a
destroyed        false
version          2

Note that the policy I uploaded does not grant explicit access to sys/internal/ui/mounts. It is very likely that you are not running the version(s) of Vault that you think you are.

[meangrape]

@briankassouf You are correct! I wasn't checking the vault binary on my client.
It's version is 0.10.1 — Vault v0.10.1 ('756fdc4587350daf1c65b93647b2cc31a6f119cd')

[jefferai]

There are some issues with vault kv when the server and client version don't match. If you use a 0.10.1 CLI I think it will solve your issue.

[meangrape]

Thanks!

…

On Wed, May 23, 2018 at 9:14 PM Jeff Mitchell ***@***.***> wrote: There are some issues with vault kv when the server and client version don't match. If you use a 0.10.1 CLI I think it will solve your issue. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#4620 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAje0iWG0quieZraiqjPYOLYylJfGuVks5t1gmBgaJpZM4ULB1S> .

[jefferai]

Closing for now then!