Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

operator-hostname: SDL update doesn't update K8s ingresses upon http_options change #248

Closed
andy108369 opened this issue Aug 12, 2024 · 1 comment
Assignees
Labels
repo/provider Akash provider-services repo issues

Comments

@andy108369
Copy link
Contributor

SDL update doesn't update K8s ingresses upon http_options change.
Example -- increased http_options.next_timeout from 60000 (60 seconds) to 600000 (600 seconds),
manifest got updated in the K8s, but the ingress wasn't updated.

Haven't spotted anything specific in the operator-hostname pod logs.

arno@x1:~$ kubectl -n lease get manifest $ns -o yaml | grep -B4 -A8 http_options
        external_port: 80
        global: true
        hosts:
        - siteXYZ.com
        http_options:
          max_body_size: 1073741824
          next_cases:
          - error
          - timeout
          next_timeout: 600000
          next_tries: 3
          read_timeout: 600000
          send_timeout: 600000
--
      expose:
      - endpoint_sequence_number: 0
        external_port: 8000
        global: true
        http_options:
          max_body_size: 1048576
          next_cases:
          - error
          - timeout
          next_tries: 3
          read_timeout: 60000
          send_timeout: 60000
        port: 8000
arno@x1:~$ kubectl -n $ns get ing -o yaml | grep -E 'nginx.ingress.kubernetes.io|host'
      nginx.ingress.kubernetes.io/proxy-body-size: "1048576"
      nginx.ingress.kubernetes.io/proxy-next-upstream: error timeout
      nginx.ingress.kubernetes.io/proxy-next-upstream-timeout: "0"
      nginx.ingress.kubernetes.io/proxy-next-upstream-tries: "3"
      nginx.ingress.kubernetes.io/proxy-read-timeout: "60"
      nginx.ingress.kubernetes.io/proxy-send-timeout: "60"
    - host: siteXYZ.com
      nginx.ingress.kubernetes.io/proxy-body-size: "1048576"
      nginx.ingress.kubernetes.io/proxy-next-upstream: error timeout
      nginx.ingress.kubernetes.io/proxy-next-upstream-timeout: "0"
      nginx.ingress.kubernetes.io/proxy-next-upstream-tries: "3"
      nginx.ingress.kubernetes.io/proxy-read-timeout: "60"
      nginx.ingress.kubernetes.io/proxy-send-timeout: "60"
    - host: REDACTED.ingress.h100.mon.obl.akash.pub
arno@x1:~$ 

The only way to update the values was to either:

  1. redeploy from scratch;
  2. OR remove the accept: directive and add it back again along with the new http_options params
  3. OR add a dummy hostname under the accept: directive and then remove it
@chainzero
Copy link
Collaborator

Issue was resolved via Akash Provider release 0.6.3. Provider ingress rules are now updated when the deployment's HTTP options are updatedd.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
repo/provider Akash provider-services repo issues
Projects
None yet
Development

No branches or pull requests

3 participants