-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
_nomad_si token getting expired or revoked. #24354
Comments
@danm-talkscriber this is described in the docs for configuring your auth method: https://developer.hashicorp.com/nomad/docs/integrations/consul/acl#consul-auth-method
Closing as duplicate of #20185 |
@tgross , thanks for so rapid reply. Going through docs now. I got damn good federated cluster setup, and that is only one issue that just killing me once a month or so. |
From your email:
You can have whatever TTL you'd like for the Nomad agent's token, so long as you have the means to renew it and reload the Nomad agent out-of-band. Nomad does not renew this token on its own.
The binding rules you created in Consul will configure this. The Consul docs on Service Identity get into the default policies. But the TTL is determined by the Consul Auth Method config (which you should have created as described here). You should not configure a TTL on those tokens. If you insist on doing so, the TTL needs to exceed the allocation lifetime.
During allocation startup, the Nomad client "logs in" to Consul using the allocation's Workload Identity, and receives the SI token in exchange. The Nomad client uses this SI token to register the workload's services and bootstrap the Envoy proxy. See the Consul ACLs integration docs. (Aside: in the future, I'd appreciate it if you asked question in GitHub or Discuss, rather than sending email to my personal email address. Thanks!) |
Nomad version
Nomad v1.8.4
Operating system and Environment details
Debian 12
Issue
Issue affect consul-connect enabled job. To establish connection between connect-proxies nomad is requesting token from consul - _nomad_si token. Periodically that token is getting expired or revoked, causing service outage since communication between services is becoming broken.
Reproduction steps
There is no reproduction step, just need to wait certain period of time, when token is getting expired/revoked.
Expected Result
Either token need to be renewed or replaced with new one.
Actual Result
Getting service outage.
Job file (if appropriate)
Nomad Server logs (if appropriate)
Nomad Client logs (if appropriate)
The text was updated successfully, but these errors were encountered: