Add endpoint exposing runtime metrics (flyteorg#524)

* metrics manager working for task nodes Signed-off-by: Daniel Rammer <[email protected]> * dynamic tasks working Signed-off-by: Daniel Rammer <[email protected]> * subworkflow node working Signed-off-by: Daniel Rammer <[email protected]> * refactored to allow branch and gate node parsing Signed-off-by: Daniel Rammer <[email protected]> * added node transition times Signed-off-by: Daniel Rammer <[email protected]> * sorting node and task executions Signed-off-by: Daniel Rammer <[email protected]> * added metrics metrics ... inception Signed-off-by: Daniel Rammer <[email protected]> * fixed duplicate metrics Signed-off-by: Daniel Rammer <[email protected]> * branch node working? Signed-off-by: Daniel Rammer <[email protected]> * implemented gate node Signed-off-by: Daniel Rammer <[email protected]> * working with partial completions and failures and cache hits Signed-off-by: Daniel Rammer <[email protected]> * added unit tests Signed-off-by: Daniel Rammer <[email protected]> * fixed most lint issues Signed-off-by: Daniel Rammer <[email protected]> * fixed lint issues Signed-off-by: Daniel Rammer <[email protected]> * added docs Signed-off-by: Daniel Rammer <[email protected]> * updated flyteidl dependency Signed-off-by: Daniel Rammer <[email protected]> * fixed unit tests Signed-off-by: Daniel Rammer <[email protected]> * chnaged to local flyteidl for testing Signed-off-by: Daniel Rammer <[email protected]> * using task event reported_at timestamps for updated_at in task execution models Signed-off-by: Daniel Rammer <[email protected]> * updated flyteidl Signed-off-by: Daniel Rammer <[email protected]> * fixed lint issues Signed-off-by: Daniel Rammer <[email protected]> * using reported_at for node executions Signed-off-by: Daniel Rammer <[email protected]> * updated flyteidl Signed-off-by: Daniel Rammer <[email protected]> * fixes unit tests and lint issues Signed-off-by: Daniel Rammer <[email protected]> * updated flyteidl Signed-off-by: Daniel Rammer <[email protected]> * bumped flyteidl deps Signed-off-by: Daniel Rammer <[email protected]> * using consts for start-node and end-node Signed-off-by: Daniel Rammer <[email protected]> --------- Signed-off-by: Daniel Rammer <[email protected]>
eapolinario · Mar 27, 2023 · cca9e17 · cca9e17
1 parent 680cac3
commit cca9e17
Show file tree

Hide file tree

Showing 12 changed files with 1,920 additions and 19 deletions.
diff --git a/go.mod b/go.mod
@@ -13,7 +13,7 @@ require (
 	github.com/cloudevents/sdk-go/v2 v2.8.0
 	github.com/coreos/go-oidc v2.2.1+incompatible
 	github.com/evanphx/json-patch v4.12.0+incompatible
-	github.com/flyteorg/flyteidl v1.3.13
+	github.com/flyteorg/flyteidl v1.3.14
 	github.com/flyteorg/flyteplugins v1.0.40
 	github.com/flyteorg/flytepropeller v1.1.70
 	github.com/flyteorg/flytestdlib v1.0.15

diff --git a/go.sum b/go.sum
@@ -312,8 +312,8 @@ github.com/fatih/structs v1.0.0/go.mod h1:9NiDSp5zOcgEDl+j00MP/WkGVPOlPRLejGD8Ga
 github.com/fatih/structs v1.1.0/go.mod h1:9NiDSp5zOcgEDl+j00MP/WkGVPOlPRLejGD8Ga6PJ7M=
 github.com/felixge/httpsnoop v1.0.1 h1:lvB5Jl89CsZtGIWuTcDM1E/vkVs49/Ml7JJe07l8SPQ=
 github.com/felixge/httpsnoop v1.0.1/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
-github.com/flyteorg/flyteidl v1.3.13 h1:jOjiHl6jmSCOGC094QaRdSjjhThhzYPm0jHSxwAZ6UM=
-github.com/flyteorg/flyteidl v1.3.13/go.mod h1:Pkt2skI1LiHs/2ZoekBnyPhuGOFMiuul6HHcKGZBsbM=
+github.com/flyteorg/flyteidl v1.3.14 h1:o5M0g/r6pXTPu5PEurbYxbQmuOu3hqqsaI2M6uvK0N8=
+github.com/flyteorg/flyteidl v1.3.14/go.mod h1:Pkt2skI1LiHs/2ZoekBnyPhuGOFMiuul6HHcKGZBsbM=
 github.com/flyteorg/flyteplugins v1.0.40 h1:RTsYingqmqr13qBbi4CB2ArXDHNHUOkAF+HTLJQiQ/s=
 github.com/flyteorg/flyteplugins v1.0.40/go.mod h1:qyUPqVspLcLGJpKxVwHDWf+kBpOGuItOxCaF6zAmDio=
 github.com/flyteorg/flytepropeller v1.1.70 h1:/d1qqz13rdVADM85ST70eerAdBstJJz9UUB/mNSZi0w=