Feature/SK-658 | Add gRPC healthchecking #515
Conversation
| @@ -31,7 +31,8 @@ | |||
| "plotly", | |||
| "pandas", | |||
| "bokeh<3.0.0", | |||
| "networkx" | |||
| "networkx", | |||
| "grpcio-health-checking" | |||
There was a problem hiding this comment.
this must be pinned, since grpcio is pinned at v1.57
Dockerfile
Outdated
| @@ -13,7 +13,13 @@ COPY config/settings-reducer.yaml.template /app/config/settings-reducer.yaml | |||
| COPY $REQUIREMENTS /app/config/requirements.txt | |||
|
|
|||
| # Install developer tools (needed for psutil) | |||
| RUN apt-get update && apt-get install -y python3-dev gcc | |||
| RUN apt-get update && apt-get install -y python3-dev gcc wget | |||
There was a problem hiding this comment.
installing binary wget not good, remove executable after it has been used. (we should have a build layer in the image really)
Dockerfile
Outdated
| RUN apt-get update && apt-get install -y python3-dev gcc wget | ||
|
|
||
| # Install grpc health probe checker | ||
| RUN GRPC_HEALTH_PROBE_VERSION=v0.4.24 && \ |
There was a problem hiding this comment.
Ok, but we won't need this binary for k8s. It's good for compose but other than that it just add more maintenance. For example, this version maybe need to be in sync with grpcio version.
There was a problem hiding this comment.
This image is for docker-compose not k8s?
|
one possible solution is to introduce an ARG health_checking=0 in the dockerfile. Then download probe only if health_checking=1. Then we set this arg in compose file to be 1. I.e by default it will not be in the image, but the healthcheck endpoint will still be there for clients to use in what ever way they desire. |
Description
This PR introduces the standardized gRPC HealthChecking protocol for the combiner server.
This fixes the issue that intermittently when spinning up the pseudo-distributed sandbox with docker-compose ,the clients fail with UnMatchedConfig error due to the combiner not being ready yet.
We can also now use this for Kubernetes: https://kubernetes.io/blog/2018/10/01/health-checking-grpc-servers-on-kubernetes/