How can I ensure cloud-sql-proxy sidecar container is running before other containers connecting to cloudsql from k8s?

[chardch]

Question

I have a k8s pod with 2 containers

• cloud-sql-proxy
• a container from third party which connects to my cloudsql private-ip instance

I want the cloud-sql-proxy to start running before the container connecting to my cloudsql instance. I experimented with using lifecycle.postStart in the cloud-sql-proxy container spec, but the container image doesn't seem to include any commands I can viably use to ensure other containers start after it.

This must be a common desired behavior, so is there any suggestion here?

Code

No response

Additional Details

No response

[jackwotherspoon]

We recently added support for a wait command #2041 and I think it is what you are looking for.

To help ensure the Proxy is up and ready, this commit adds a wait command with an optional --max flag to set the maximum time to wait.

By default when invoking this command:

./cloud-sql-proxy wait

The Proxy will wait up to the maximum time for the /startup endpoint to respond. This command requires that the Proxy be started in another process with the HTTP health check enabled. If an alternate health check port or address is used, as in:

./cloud-sql-proxy <INSTANCE_CONNECTION_NAME> \
  --http-address 0.0.0.0 \
  --http-port 9191

Then the wait command must also be told to use the same custom values:

./cloud-sql-proxy wait \
  --http-address 0.0.0.0 \
  --http-port 9191

By default the wait command will wait 30 seconds. To alter this value, use:

./cloud-sql-proxy wait --max 10s

@chardch does that work for your use-case?

[enocom]

This works in combination with a poststart hook. Context: https://medium.com/@marko.luksa/delaying-application-start-until-sidecar-is-ready-2ec2d21a7b74

[enocom]

Going to close this. Feel free to reopen if you’re still stuck on this.

[chardch]

Got it, thank you for this info! The wait command is exactly what I was looking for.

[cfstras]

For anyone coming here, please note that you'll also have to pass --health-check to the proxy invocation. Otherwise you'll just get the log below, and your cloud-sql-proxy wait will time out.
2024/04/04 10:05:39 Ignoring --http-port because --prometheus or --health-check was not set

[joaoviana]

This works in combination with a poststart hook. Context: https://medium.com/@marko.luksa/delaying-application-start-until-sidecar-is-ready-2ec2d21a7b74

Hi @enocom, I was wondering if you could share how it should work with a postStart hook and wait?

Version: cloud-sql-proxy:2.11.4

lifecycle:                                                                                                                                                                                                     
   postStart:                                                                                                                                                                                                        
     exec:                                                                                                                                                                                                      
       command:                                                                                                                                                                                                 
            - /bin/sh          
            - -c                                                                                                                                                                                                     
            - ./cloud-sql-proxy wait --http-address 0.0.0.0 --http-port XXXX   

also tried

postStart:                                                                                                                                                                        
   exec:                                                                                                                                                                           
     command:                                                                                                                                                                      
        - /cloud_sql_proxy                                                                                                                                                            
        - wait                                                                                                                                                                        
        - --http-address                                                                                                                                                              
        - 0.0.0.0                                                                                                                                                                     
        - --http-port                                                                                                                                                                 │
        - "XXXX" 

and just

postStart:                                                                                                                                                                        
   exec:                                                                                                                                                                           
     command:                                                                                                                                                                                                                                                                                       
        - wait         

On the above ⬆️ also tried with and without the 2 flags: --http-address --http-port

and I set the right envs (I believe):

  env:                                                                                                                                                                                                           │                                                                                                                                                                                           │
│     - name: CSQL_PROXY_HEALTH_CHECK                                                                                                                                                                                │
│       value: "true"                                                                                                                                                                                                │
│     - name: CSQL_PROXY_HTTP_PORT                                                                                                                                                                                   │
│       value: "XXXX"                                                                                                                                                                                                │
│     - name: CSQL_PROXY_HTTP_ADDRESS                                                                                                                                                                                │
│       value: 0.0.0.0                                                                                                                                                                                               │
│     - name: CSQL_PROXY_EXIT_ZERO_ON_SIGTERM                                                                                                                                                                        │
│       value: "true"                                                                                                                                                                                                │
│     - name: CSQL_PROXY_STRUCTURED_LOGS                                                                                                                                                                             │
│       value: "true"                 

And I keep getting a PostStartHook failed error.
I also grabbed the startupProbe and livenessProbe from the healthcheck examples, and added them to my set up.
Thanks 😃

EDIT:

was obviously the command flags that weren't set properly 🤦 thanks anyways!

[frko]

@joaoviana Could you perhaps share the implementation of the setup that was successful for you? It's unclear to me what your final k8s post start hook exactly looks like and would be very grateful If you'd be able to share that.

Or would you perhaps be able to provide a working example @enocom ?

[mdering]

yes please post what you got working! @joaoviana

[enocom]

Take a look at #128 (comment) -- that uses the new sidecar containers.

#2063 (comment) would be another good option if you can't use the new feature.