First request with 0 replicas sometimes takes ~20s

[abatilo]

Report

Sometimes when I send a request to an application that I have configured with scale to 0, the first request will take around 20 seconds to respond, even though the pod has been scheduled and ready after only 2-3 seconds.

Expected Behavior

I would expect that if I send a request to an application with 0 replicas, that once the pod is up and healthy that the interceptor will return immediately

Actual Behavior

Instead, it might take between 20-25 seconds before a response is made

Steps to Reproduce the Problem

I have a website up that hosts just a stock boilerplate React application + Golang backed service. Feel free to hit it. It's a domain/codebase that I like to use for testing things.

curl https://friendlyfaces.app

Below is the entire YAML that I use for deploying the site:

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.kubernetes.io/topology-aware-hints: auto
  name: friendlyfaces
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 8080
  selector:
    app: friendlyfaces
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: friendlyfaces
  name: friendlyfaces
spec:
  revisionHistoryLimit: 1
  selector:
    matchLabels:
      app: friendlyfaces
  template:
    metadata:
      labels:
        app: friendlyfaces
    spec:
      containers:
      - command:
        - chamber
        - exec
        - honeycomb
        - --
        - friendlyfaces
        - --static-assets-path
        - /static
        image: 911907402684.dkr.ecr.us-west-2.amazonaws.com/friendlyfaces:03db570bbd7677cc5a45f1939de03dc474ee03da
        lifecycle:
          preStop:
            exec:
              command:
              - sleep
        name: friendlyfaces
        ports:
        - containerPort: 8080
        readinessProbe:
          httpGet:
            path: /
            port: 8080
        resources:
          limits:
            memory: 128Mi
          requests:
            cpu: 0.1
            memory: 128Mi
      serviceAccountName: friendlyfaces
      topologySpreadConstraints:
      - labelSelector:
          matchLabels:
            app: friendlyfaces
        maxSkew: 1
        topologyKey: topology.kubernetes.io/zone
        whenUnsatisfiable: ScheduleAnyway
      - labelSelector:
          matchLabels:
            app: friendlyfaces
        maxSkew: 1
        topologyKey: kubernetes.io/hostname
        whenUnsatisfiable: ScheduleAnyway
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: friendlyfaces
spec:
  dnsNames:
  - friendlyfaces.app
  - www.friendlyfaces.app
  issuerRef:
    kind: ClusterIssuer
    name: letsencrypt
  secretName: friendlyfaces-cert
---
apiVersion: http.keda.sh/v1alpha1
kind: HTTPScaledObject
metadata:
  name: friendlyfaces
spec:
  host: friendlyfaces.app
  replicas:
    max: 5
    min: 0
  scaleTargetRef:
    deployment: friendlyfaces
    port: 80
    service: friendlyfaces
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: friendlyfaces
spec:
  entryPoints:
  - websecure
  routes:
  - kind: Rule
    match: Host(`friendlyfaces.app`, `www.friendlyfaces.app`)
    middlewares:
    - name: friendlyfaces
    - name: security-headers
      namespace: traefik
    services:
    - name: keda-add-ons-http-interceptor-proxy
      namespace: keda
      port: 8080
  tls:
    secretName: friendlyfaces-cert
---
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: friendlyfaces
spec:
  headers:
    customRequestHeaders:
      host: friendlyfaces.app

Logs from KEDA HTTP operator

No logs about this as far as I can tell

What version of the KEDA HTTP Add-on are you running?

v0.4.0

Kubernetes Version

1.24

Platform

Amazon Web Services

Anything else?

I'd love to be a contributor and try to debug this on my own. I'm going to do my best to figure out what's happening but if anyone could give me ideas or point me in the right direction that'd be amazing too.

Thank you all for this project. It's really neat and I like it a lot.

[abatilo]

Okay, something that I've noticed is that it appears that if I restart the interceptors, it's fast again:

k -n keda rollout restart deployment/keda-add-ons-http-interceptor

So maybe something with caching? Hmm

[JorTurFer]

Hi,
Thanks for reporting :)
Are you willing to contribute with the fix?

[abatilo]

Hi @JorTurFer!

Yes, sorry, I should have highlighted that more in my original issue submission:

I'd love to be a contributor and try to debug this on my own. I'm going to do my best to figure out what's happening but if anyone could give me ideas or point me in the right direction that'd be

I'd love to contribute the fix. I'm working through trying to find the problem. If you have any tips or ideas, I'd love to hear them though.

[JorTurFer]

oh, sorry, I read it fast and I didn't get that part :(
In this case, I'd check what component needs the restart (In this case, it seems that is the interceptor). Restarting the interceptor, you are removing caches but also enforcing new connections between interceptor and metrics server. I'd start with this communication, maybe adding some logs to the interceptor/metrics server to detect possible dropped connections.

TBH, I'm not sure about what kind of cache could be in the interceptor

[abatilo]

@JorTurFer We did it!

#598

I found the problem that was causing this and opened a PR to fix the problem. Please take a look when you can!