Vault Agent 1.13.0 requires a listener block when its documented as optional

[danlsgiga]

Describe the bug
In Vault 1.13.0, not defining a listener block in a vault agent configuration will cause the following error at startup:

error validating configuration: configuring the api_proxy requires at least 1 listener to be defined

This was introduced here

This is a specially opinionated decision given the fact the api_proxy is also not declared in the config and both directives are documented as optional here

To Reproduce
Steps to reproduce the behavior:

1. Run a vault agent without the listener and api_proxy stanzas defined

Expected behavior
Vault agent will start without failures and not allocate a listener

Environment:

Vault server configuration file(s):

# Paste your Vault config here.
# Be sure to scrub any sensitive values

Additional context
In our use case we use Vault Agent to render certificates and configuration file templates but we don't need any listener binding since we don't have any other processes talking to Vault. In this case, the vault agent is acting more like a client rather than a proxy. So for our specific use case, this is a breaking change.

[VioletHynes]

Hi there!

I wasn't able to reproduce this issue. The error validating configuration: configuring the api_proxy requires at least 1 listener to be defined should only occur if you have an API proxy or API proxy related configuration.

I was able to start Agent just fine with the following config, with no API proxy configuration or listeners:

pid_file = "/tmp/pidfile"

vault {
  address = "http://127.0.0.1:8200"
  tls_skip_verify = true
  retry {
    num_retries = 10
  }
}

template {
  destination  = "/tmp/agent/render-content.txt"
  contents     = "{{ with secret \"auth/token/lookup-self\" }}{{ .Data.id }}{{ end }}"
  exec {
	command = ["touch", "/tmp/agent/my,file"]
  }
}

auto_auth {
  method {
    type      = "token_file"
    config = {
      token_file_path = "/Users/violet/.vault-token"
    }
  }
  sink {
    type = "file"
    config = {
      path = "/Users/violet/vault/token"
    }
  }
}

I think I know the source of your issue, though, you've likely configured the API proxy through the cache stanza, like maybe something like this. It would be helpful to see your full config, though, if this isn't the case:

cache {
  use_auto_auth_token = true
}

It's worth noting that the above config has been moved to the api_stanza as of 1.13, though the config in cache has remained for backwards compatibility.

The above configuration is API proxy configuration, and does nothing without a defined listener. In other words, there's no reason to configure an API proxy if you're not using that use case.

[danlsgiga]

Ah, that's great info @VioletHynes! Indeed I have the cache stanza in my config and that explains why its failing for me.

Thanks for looking into this issue! We can close it now! I'd suggest to add a note in the release notes or docs to avoid other folks hitting this scenario. Its not clear that having the cache stanza alone will trigger this new startup failure behaviour!

[VioletHynes]

Thanks for confirming! I'll close it, and thanks for the feedback! I'll think about how best to improve this experience.

[itspngu]

Sorry for commenting on a closed issue, but this is somewhat confusing. Are we meant to define a listener block when running the agent as a sidecar container in Kubernetes and want to take advantage of the persistent caching feature? We're just using the agent to render templates and would like to avoid having to open sockets to get persistence in 1.13+ :(

[F21]

Just wanted to chime in on this as I ran into this issue as well. Do we need a listener if we just want to use the cache?

I have this in my config and the vault agent also fails to start with error validating configuration: enabling the cache requires at least 1 template or 1 listener to be defined:

cache {
  // An empty cache stanza still enables caching
}

In my case, I don't plan to clear the cache using the api, so having to define a listener seems extraneous.

[itspngu]

I was hoping that #20934 fixed this, but it didn't.

It appears that we are indeed supposed to define listener blocks. At least that's what the Agent Injector webhook does when using annotations: https://github.com/hashicorp/vault-k8s/blob/main/agent-inject/agent/config.go#L269-L278

Excerpt with relevant lines from a working deployment:

# Set up caching for leased secrets between Vault Agent init and runtime containers
# We need to define a`listener` block for this to work, even if nothing is meant to talk to the Agent.
listener "unix" {
  address = "/run/vault.sock"
  tls_disable = true
}
cache {
  use_auto_auth_token = true
  persist "kubernetes" {
    path = "/vault/agent-cache"
  }
}

[VioletHynes]

Hey! Sorry if this is still causing some kind of issue. I'd love to learn more about what's going on.

You do need either a listener or templating for the cache to do anything, so we require either block -- this isn't new, as far as I can tell and remember.

Just to confirm, are you saying you're getting errors enabling the cache when you do have templating in your config? Could you post what your full config is (do redact sensitive stuff!) This definitely sounds like a bug, though, you shouldn't need to add a listener and it certainly shouldn't be required for the cache.

[VioletHynes]

Oh, I just noticed that you had use_auto_auth_token = true in your config. I missed that before posting my above comment. This config solely affects the listener, so naturally, you need to have a listener to configure, when your config includes listener-specific config.

The reason use_auto_auth_token is in the cache stanza is for historical reasons (it's not related to cache behaviour), but it's not required for the cache and it affects listener behaviour. If you're not using the api proxy functionality, there's no need to use it. @itspngu can you confirm?

[itspngu]

@VioletHynes hey, thanks for the input. I'll revisit this tomorrow and report back.

[VioletHynes]

Going to close this one, as I think I re-opened this erroneously and the error was valid. In the config posted above, use_auto_auth_token is defined, which configures the API proxy. As a result, you need a listener for the config to make sense. If you're not using the API proxy or require a listener, there's no need to configure use_auto_auth_token at all, and it shouldn't affect any behaviour other than listener behaviour.

In newer versions of Agent, and our docs, we have a separate stanza, api_proxy, for API proxy configuration to occur, moving it out of the cache stanza where it was confusing. We understand that API proxy configuration being in the cache stanza is confusing.

listener "unix" {
  address = "/run/vault.sock"
  tls_disable = true
}
cache {
  use_auto_auth_token = true # listener configuration
  persist "kubernetes" {
    path = "/vault/agent-cache"
  }
}

Feel free to comment after I've closed if you think I've misunderstood in some way, or you're still getting some kind of errors even after removing the API proxy specific configuration.

[itspngu]

Sorry for the late reply, I wasn't able to get around to testing this earlier. The changes proposed by you work, the whole "listener config in the cache stanza" thing is confusing indeed. Thank you!

Everything works just fine with the listener stanza and use_auto_auth_token removed:

cache {
  persist "kubernetes" {
    path = "/vault/agent-cache"
  }
}