graph level metadata #1803

RafalSkolasinski · 2020-05-06T20:09:07Z

Initial implementation of graph-level metadata. Closes #1728

Implementation around following new structs defined in executor

type MetadataTensor struct {
	DataType string `json:"datatype,omitempty"`
	Name     string `json:"name,omitempty"`
	Shape    []int  `json:"shape,omitempty"`
}

type ModelMetadata struct {
	Name     string           `json:"name,omitempty"`
	Platform string           `json:"platform,omitempty"`
	Versions []string         `json:"versions,omitempty"`
	Inputs   []MetadataTensor `json:"inputs,omitempty"`
	Outputs  []MetadataTensor `json:"outputs,omitempty"`
}

type GraphMetadata struct {
	Name         string                   `json:"name,omitempty"`
	Models       map[string]ModelMetadata `json:"models,omitempty"`
	GraphInputs  []MetadataTensor         `json:"graphinputs,omitempty"`
	GraphOutputs []MetadataTensor         `json:"graphoutputs,omitempty"`
}

note that struct GraphMetadata defines output format of graph level metadata

seldondev · 2020-05-06T20:10:13Z

Wed May 6 20:10:12 UTC 2020
The logs for [pr-build] [1] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1803/1.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1803 --build=1

seldondev · 2020-05-06T20:10:22Z

Wed May 6 20:10:21 UTC 2020
The logs for [lint] [2] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1803/2.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1803 --build=2

seldondev · 2020-05-07T15:05:05Z

Thu May 7 15:05:04 UTC 2020
The logs for [pr-build] [3] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1803/3.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1803 --build=3

seldondev · 2020-05-07T15:05:13Z

Thu May 7 15:05:12 UTC 2020
The logs for [lint] [4] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1803/4.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1803 --build=4

seldondev · 2020-05-12T16:33:46Z

Tue May 12 16:33:45 UTC 2020
The logs for [pr-build] [5] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1803/5.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1803 --build=5

seldondev · 2020-05-12T16:33:46Z

Tue May 12 16:33:45 UTC 2020
The logs for [lint] [6] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1803/6.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1803 --build=6

seldondev · 2020-05-13T16:37:12Z

Wed May 13 16:37:11 UTC 2020
The logs for [lint] [8] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1803/8.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1803 --build=8

seldondev · 2020-05-13T16:37:16Z

Wed May 13 16:37:15 UTC 2020
The logs for [pr-build] [7] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1803/7.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1803 --build=7

glindsell

Good stuff!
A few nit picks in the code.
The most important area to me to look at more closely before approving is the situations where nil, nil is returned from GetShapeFromGraph function.
Understand there is a design doc here so will take a look at that.

executor/predictor/metadata.go

executor/predictor/metadata_test.go

seldondev · 2020-05-18T15:14:38Z

Mon May 18 15:14:37 UTC 2020
The logs for [pr-build] [11] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1803/11.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1803 --build=11

seldondev · 2020-05-18T15:14:38Z

Mon May 18 15:14:37 UTC 2020
The logs for [lint] [12] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1803/12.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1803 --build=12

seldondev · 2020-05-19T11:30:06Z

Tue May 19 11:30:05 UTC 2020
The logs for [lint] [14] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1803/14.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1803 --build=14

seldondev · 2020-05-19T11:30:09Z

Tue May 19 11:30:08 UTC 2020
The logs for [pr-build] [13] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1803/13.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1803 --build=13

seldondev · 2020-05-19T12:05:34Z

Tue May 19 12:05:33 UTC 2020
The logs for [pr-build] [15] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1803/15.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1803 --build=15

seldondev · 2020-05-19T12:05:44Z

Tue May 19 12:05:43 UTC 2020
The logs for [lint] [16] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1803/16.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1803 --build=16

seldondev · 2020-05-19T15:06:22Z

Tue May 19 15:06:20 UTC 2020
The logs for [pr-build] [17] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1803/17.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1803 --build=17

seldondev · 2020-05-19T15:06:24Z

Tue May 19 15:06:23 UTC 2020
The logs for [lint] [18] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1803/18.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1803 --build=18

RafalSkolasinski · 2020-05-19T19:00:24Z

/test integration

seldondev · 2020-05-19T19:07:10Z

Tue May 19 19:07:09 UTC 2020
The logs for [lint] [21] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1803/21.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1803 --build=21

seldondev · 2020-05-22T11:51:00Z

Fri May 22 11:50:59 UTC 2020
The logs for [pr-build] [50] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1803/50.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1803 --build=50

seldondev · 2020-05-22T11:51:04Z

Fri May 22 11:51:02 UTC 2020
The logs for [lint] [51] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1803/51.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1803 --build=51

seldondev · 2020-05-22T11:51:51Z

Fri May 22 11:51:48 UTC 2020
The logs for [notebooks] [53] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1803/53.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1803 --build=53

seldondev · 2020-05-22T11:52:08Z

Fri May 22 11:52:04 UTC 2020
The logs for [integration] [54] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1803/54.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1803 --build=54

seldondev · 2020-05-22T11:53:26Z

Fri May 22 11:53:25 UTC 2020
The logs for [lint] [55] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1803/55.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1803 --build=55

seldondev · 2020-05-22T11:55:01Z

Fri May 22 11:54:59 UTC 2020
The logs for [pr-build] [52] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1803/52.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1803 --build=52

RafalSkolasinski · 2020-05-22T13:43:34Z

/retest

RafalSkolasinski · 2020-05-22T13:43:49Z

/test notebooks

seldondev · 2020-05-22T13:44:50Z

Fri May 22 13:44:48 UTC 2020
The logs for [lint] [57] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1803/57.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1803 --build=57

seldondev · 2020-05-22T13:44:53Z

Fri May 22 13:44:52 UTC 2020
The logs for [pr-build] [56] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1803/56.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1803 --build=56

seldondev · 2020-05-22T13:45:17Z

Fri May 22 13:45:16 UTC 2020
The logs for [notebooks] [58] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1803/58.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1803 --build=58

RafalSkolasinski · 2020-05-22T13:53:09Z

/retest

seldondev · 2020-05-22T13:54:29Z

Fri May 22 13:54:28 UTC 2020
The logs for [pr-build] [59] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1803/59.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1803 --build=59

RafalSkolasinski · 2020-05-22T15:34:50Z

The failing notebook is now passing locally without troubles - have not a single clue what is going wrong here.

RafalSkolasinski · 2020-05-24T19:06:25Z

/test integration
/test notebooks

seldondev · 2020-05-24T19:07:53Z

Sun May 24 19:07:53 UTC 2020
The logs for [integration] [60] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1803/60.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1803 --build=60

seldondev · 2020-05-24T19:08:16Z

Sun May 24 19:08:12 UTC 2020
The logs for [notebooks] [61] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1803/61.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1803 --build=61

RafalSkolasinski · 2020-05-26T08:53:40Z

/test integration

seldondev · 2020-05-26T08:54:37Z

Tue May 26 08:54:36 UTC 2020
The logs for [pr-build] [62] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1803/62.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1803 --build=62

seldondev · 2020-05-26T08:54:51Z

Tue May 26 08:54:50 UTC 2020
The logs for [integration] [64] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1803/64.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1803 --build=64

seldondev · 2020-05-26T08:55:39Z

Tue May 26 08:55:38 UTC 2020
The logs for [lint] [63] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1803/63.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1803 --build=63

RafalSkolasinski · 2020-05-26T09:25:16Z

/retest

seldondev · 2020-05-26T09:28:54Z

Tue May 26 09:28:49 UTC 2020
The logs for [integration] [65] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1803/65.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1803 --build=65

seldondev · 2020-05-26T15:04:39Z

@RafalSkolasinski: The following tests failed, say /retest to rerun them all:

Test name	Commit	Details	Rerun command
notebooks	`0c14de6`	link	`/test notebooks`
integration	`6470cd3`	link	`/test integration`

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

RafalSkolasinski · 2020-05-26T17:15:49Z

@axsaucedo all tests passed locally, shall we merge manually?

seldondev · 2021-05-13T06:32:13Z

failed to trigger Pull Request pipeline

failed to create agent
failed to calculate in repo config
failed to load trigger config for repository SeldonIO/seldon-core for ref 6470cd3
failed to find any lighthouse configuration files in repo SeldonIO/seldon-core at sha 6470cd3
failed to process repo SeldonIO/seldon-core refref 6470cd3
failed to list files in directory /var/tmp/gitrepo660600678/.lighthouse
open /var/tmp/gitrepo660600678/.lighthouse
no such file or directory

seldondev added do-not-merge/work-in-progress size/L labels May 6, 2020

seldondev added size/XL and removed size/L labels May 7, 2020

RafalSkolasinski mentioned this pull request May 13, 2020

Is it possible to access /health/status and /metadata endpoints defined in seldon through ambassador #1822

Closed

glindsell suggested changes May 14, 2020

View reviewed changes

seldondev assigned glindsell May 14, 2020

seldondev added size/XXL and removed size/XL labels May 18, 2020

RafalSkolasinski requested a review from ukclivecox May 18, 2020 15:37

RafalSkolasinski changed the title ~~WIP: graph level metadata~~ graph level metadata May 19, 2020

seldondev removed the do-not-merge/work-in-progress label May 19, 2020

RafalSkolasinski added 2 commits May 26, 2020 09:50

Add graph-level metadata support. Closes 1728.

173f137

housekeeping in CI-tested notebooks

6470cd3

RafalSkolasinski merged commit e7b1d8f into SeldonIO:master May 26, 2020

axsaucedo unassigned glindsell May 13, 2021

graph level metadata #1803

graph level metadata #1803

Conversation

RafalSkolasinski commented May 6, 2020 • edited Loading

seldondev commented May 6, 2020

seldondev commented May 6, 2020

seldondev commented May 7, 2020

seldondev commented May 7, 2020

seldondev commented May 12, 2020

seldondev commented May 12, 2020

seldondev commented May 13, 2020

seldondev commented May 13, 2020

glindsell left a comment

Choose a reason for hiding this comment

seldondev commented May 18, 2020

seldondev commented May 18, 2020

seldondev commented May 19, 2020

seldondev commented May 19, 2020

seldondev commented May 19, 2020

seldondev commented May 19, 2020

seldondev commented May 19, 2020

seldondev commented May 19, 2020

RafalSkolasinski commented May 19, 2020

seldondev commented May 19, 2020

seldondev commented May 22, 2020

seldondev commented May 22, 2020

seldondev commented May 22, 2020

seldondev commented May 22, 2020

seldondev commented May 22, 2020

seldondev commented May 22, 2020

RafalSkolasinski commented May 22, 2020

RafalSkolasinski commented May 22, 2020

seldondev commented May 22, 2020

seldondev commented May 22, 2020

seldondev commented May 22, 2020

RafalSkolasinski commented May 22, 2020

seldondev commented May 22, 2020

RafalSkolasinski commented May 22, 2020

RafalSkolasinski commented May 24, 2020

seldondev commented May 24, 2020

seldondev commented May 24, 2020

RafalSkolasinski commented May 26, 2020

seldondev commented May 26, 2020

seldondev commented May 26, 2020

seldondev commented May 26, 2020

RafalSkolasinski commented May 26, 2020

seldondev commented May 26, 2020

seldondev commented May 26, 2020

RafalSkolasinski commented May 26, 2020

seldondev commented May 13, 2021

RafalSkolasinski commented May 6, 2020 •

edited

Loading