Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] Support gRPC health checking protocol #489

Closed
2 of 13 tasks
honnix opened this issue Aug 28, 2020 · 0 comments
Closed
2 of 13 tasks

[Feature] Support gRPC health checking protocol #489

honnix opened this issue Aug 28, 2020 · 0 comments
Assignees
@honnix
Labels
enhancement New feature or request

Comments

@honnix
Copy link
Member

honnix commented Aug 28, 2020

Motivation: Why do you think this is important?
Now both flyteadmin and datacatalog only support HTTP health check. It would be great that gRPC health check is also supported so client that only talks gRPC can also check service health. Also, grpc-health-probe is commonly used to check gRPC server/service health, so it would be nice to support it.

gRPC health checking protocol is defined by https://github.com/grpc/grpc/blob/master/doc/health-checking.md and implemented in go gRPC lib.

Goal: What should the final outcome look like, ideally?
Both flyteadmin and datacatalog support gRCP health check.

Describe alternatives you've considered
As described in motivation section, HTTP health check could be used however that requires client to talk plain HTTP as well.

Flyte component

  • Overall
  • Flyte Setup and Installation scripts
  • Flyte Documentation
  • Flyte communication (slack/email etc)
  • FlytePropeller
  • FlyteIDL (Flyte specification language)
  • Flytekit (Python SDK)
  • FlyteAdmin (Control Plane service)
  • FlytePlugins
  • DataCatalog
  • FlyteStdlib (common libraries)
  • FlyteConsole (UI)
  • Other

[Optional] Propose: Link/Inline
The implementation is very straight forward:

	healthServer := health.NewServer()
	healthServer.SetServingStatus("", grpc_health_v1.HealthCheckResponse_SERVING)
	grpc_health_v1.RegisterHealthServer(grpcServer, healthServer)

Additional context

Is this a blocker for you to adopt Flyte
Nope.
@honnix honnix added enhancement New feature or request untriaged This issues has not yet been looked at by the Maintainers labels Aug 28, 2020
@honnix honnix self-assigned this Aug 28, 2020
honnix added a commit to flyteorg/flyteadmin that referenced this issue Aug 28, 2020
@honnix
Support gRPC health checking
302b0e4 
This addresses flyteorg/flyte#489.

For now only server level health checking is supported but we could
extend to service level when needed later.
@honnix honnix mentioned this issue Aug 28, 2020
Merged
8 tasks
honnix added a commit to honnix/datacatalog that referenced this issue Aug 28, 2020
@honnix
Support gRPC health checking
1065dbb 
This addresses flyteorg/flyte#489.

For now only server level health checking is supported but we could
extend to service level when needed later.
@honnix honnix mentioned this issue Aug 28, 2020
Merged
8 tasks
honnix added a commit to flyteorg/flyteadmin that referenced this issue Aug 28, 2020
@honnix
Grpc health checking (#118)
bf159e5 
* Delete and ignore vim swap file

* Support gRPC health checking

This addresses flyteorg/flyte#489.

For now only server level health checking is supported but we could
extend to service level when needed later.
@honnix honnix removed the untriaged This issues has not yet been looked at by the Maintainers label Sep 2, 2020
@honnix honnix closed this as completed Sep 2, 2020
eapolinario pushed a commit to eapolinario/flyte that referenced this issue Dec 6, 2022
Skipping of cached task outputs via execution config (flyteorg#489)
01ec214 
* Implemented skipping of cached task results via execution config
CatalogClient.Put can now create or update/overwrite artifacts and their data

Signed-off-by: Nick Müller <[email protected]>

* Renamed SkipCache flag to OverwriteCache
Updated to latest released versions of flyteidl and flytestdlib

Signed-off-by: Nick Müller <[email protected]>

* datacatalog client now handles NotFound errors gracefully while overwriting artifact

Signed-off-by: Nick Müller <[email protected]>

* Refactored updating of artifact data into separate Update method of datacatalog client

Signed-off-by: Nick Müller <[email protected]>

* Updated to latest released flyteplugins version

Signed-off-by: Nick Müller <[email protected]>

* Use go 1.18 for CodeQL GHA

Signed-off-by: Nick Müller <[email protected]>

Signed-off-by: Nick Müller <[email protected]>
eapolinario pushed a commit to eapolinario/flyte that referenced this issue Dec 6, 2022
@manuelrombach
Remove shard prefix from upload location (flyteorg#489)
0e1cc8b 
Signed-off-by: Manuel Rombach <[email protected]>

Signed-off-by: Manuel Rombach <[email protected]>
eapolinario pushed a commit to eapolinario/flyte that referenced this issue Dec 20, 2022
@samhita-alla
add kftensorflow example (flyteorg#489)
4458c9e 
* add kftensorflow example

Signed-off-by: Samhita Alla <[email protected]>

* fix conf

Signed-off-by: Samhita Alla <[email protected]>

* set number of gpus to 1

Signed-off-by: Samhita Alla <[email protected]>

* gpu

Signed-off-by: Samhita Alla <[email protected]>

* increase cpu request

Signed-off-by: Samhita Alla <[email protected]>

* increase cpu request

Signed-off-by: Samhita Alla <[email protected]>

* mem

Signed-off-by: Samhita Alla <[email protected]>

* mem

Signed-off-by: Samhita Alla <[email protected]>

* mem

Signed-off-by: Samhita Alla <[email protected]>

* remove cache

Signed-off-by: Samhita Alla <[email protected]>

* mem

Signed-off-by: Samhita Alla <[email protected]>

* storage

Signed-off-by: Samhita Alla <[email protected]>

* storage

Signed-off-by: Samhita Alla <[email protected]>

* storage

Signed-off-by: Samhita Alla <[email protected]>

* storage

Signed-off-by: Samhita Alla <[email protected]>

* modify

Signed-off-by: Samhita Alla <[email protected]>

* modify

Signed-off-by: Samhita Alla <[email protected]>

* modify

Signed-off-by: Samhita Alla <[email protected]>

* increase memory limit

Signed-off-by: Samhita Alla <[email protected]>

* working

Signed-off-by: Samhita Alla <[email protected]>

* fix resources

Signed-off-by: Samhita Alla <[email protected]>

* final version

Signed-off-by: Samhita Alla <[email protected]>

* nit

Signed-off-by: Samhita Alla <[email protected]>

* rephrase GPU section

Signed-off-by: Samhita Alla <[email protected]>

* fix workflow

Signed-off-by: Samhita Alla <[email protected]>

* incorporate comments by @cosmicBboy

Signed-off-by: Samhita Alla <[email protected]>
pingsutw pushed a commit to pingsutw/flyte-monorepo that referenced this issue Apr 4, 2023
@honnix
Grpc health checking (#118)
0c4b0a5 
* Delete and ignore vim swap file

* Support gRPC health checking

This addresses flyteorg/flyte#489.

For now only server level health checking is supported but we could
extend to service level when needed later.
eapolinario pushed a commit to eapolinario/flyte that referenced this issue Aug 9, 2023
Skipping of cached task outputs via execution config (flyteorg#489)
cbffd45 
* Implemented skipping of cached task results via execution config
CatalogClient.Put can now create or update/overwrite artifacts and their data

Signed-off-by: Nick Müller <[email protected]>

* Renamed SkipCache flag to OverwriteCache
Updated to latest released versions of flyteidl and flytestdlib

Signed-off-by: Nick Müller <[email protected]>

* datacatalog client now handles NotFound errors gracefully while overwriting artifact

Signed-off-by: Nick Müller <[email protected]>

* Refactored updating of artifact data into separate Update method of datacatalog client

Signed-off-by: Nick Müller <[email protected]>

* Updated to latest released flyteplugins version

Signed-off-by: Nick Müller <[email protected]>

* Use go 1.18 for CodeQL GHA

Signed-off-by: Nick Müller <[email protected]>

Signed-off-by: Nick Müller <[email protected]>
eapolinario pushed a commit to eapolinario/flyte that referenced this issue Aug 21, 2023
@manuelrombach
Remove shard prefix from upload location (flyteorg#489)
1398328 
Signed-off-by: Manuel Rombach <[email protected]>

Signed-off-by: Manuel Rombach <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@honnix honnix

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@honnix