Add KEDA support to seldon-core #2506

anggao · 2020-09-30T17:08:22Z

What this PR does / why we need it:
Add KEDA support to seldon-core

Which issue(s) this PR fixes:

Special notes for your reviewer:
As part of this work, I also upgraded golang version to 1.14 as this is required for knative.dev/pkg module used by KEDA

Does this PR introduce a user-facing change?:

Added KEDA support to seldon-core

seldondev · 2020-09-30T17:08:27Z

Hi @anggao. Thanks for your PR.

I'm waiting for a SeldonIO member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the jenkins-x/lighthouse repository.

anggao · 2020-09-30T17:11:40Z

/assign @cliveseldon

ukclivecox · 2020-10-01T10:50:12Z

@anggao Can you add an example notebook to illustrate in the /examples folder?

ukclivecox · 2020-10-01T10:50:23Z

/ok-to-test

seldondev · 2020-10-01T10:52:09Z

Thu Oct 1 10:52:07 UTC 2020
The logs for [lint] [2] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2506/2.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2506 --build=2

seldondev · 2020-10-01T10:52:15Z

Thu Oct 1 10:52:14 UTC 2020
The logs for [pr-build] [1] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2506/1.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2506 --build=1

anggao · 2020-10-01T11:05:31Z

sure I will do and fix the tests

anggao · 2020-10-01T15:33:23Z

/retest

seldondev · 2020-10-01T15:34:39Z

Thu Oct 1 15:34:38 UTC 2020
The logs for [lint] [4] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2506/4.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2506 --build=4

seldondev · 2020-10-01T15:34:42Z

Thu Oct 1 15:34:41 UTC 2020
The logs for [pr-build] [3] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2506/3.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2506 --build=3

anggao · 2020-10-01T15:49:20Z

@cliveseldon can I run these tests locally ?

ukclivecox · 2020-10-01T16:32:35Z

The CI steps are defined at top level of project. The failed on would be: https://github.com/SeldonIO/seldon-core/blob/master/jenkins-x.yml

Not sure why lint is pending.

You can run locally these steps. See what they do in the jenkins definitions

seldondev · 2020-10-02T11:32:02Z

Fri Oct 2 11:32:01 UTC 2020
The logs for [pr-build] [5] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2506/5.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2506 --build=5

seldondev · 2020-10-02T11:32:19Z

Fri Oct 2 11:32:18 UTC 2020
The logs for [lint] [6] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2506/6.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2506 --build=6

seldondev · 2020-10-02T11:46:19Z

Fri Oct 2 11:46:16 UTC 2020
The logs for [lint] [8] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2506/8.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2506 --build=8

seldondev · 2020-10-02T11:46:19Z

Fri Oct 2 11:46:14 UTC 2020
The logs for [pr-build] [7] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2506/7.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2506 --build=7

anggao · 2020-10-02T17:20:14Z

@cliveseldon I added example and fixed the build

seldondev · 2020-10-02T17:21:10Z

Fri Oct 2 17:21:04 UTC 2020
The logs for [lint] [10] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2506/10.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2506 --build=10

seldondev · 2020-10-02T17:21:17Z

Fri Oct 2 17:21:16 UTC 2020
The logs for [pr-build] [9] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2506/9.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2506 --build=9

ukclivecox · 2020-10-06T12:38:57Z

/test integration

seldondev · 2020-10-06T12:40:44Z

Tue Oct 6 12:40:38 UTC 2020
The logs for [integration] [11] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2506/11.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2506 --build=11

seldondev · 2020-10-06T13:14:05Z

Tue Oct 6 13:14:04 UTC 2020
The logs for [lint] [13] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2506/13.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2506 --build=13

seldondev · 2020-10-06T13:14:07Z

Tue Oct 6 13:14:06 UTC 2020
The logs for [pr-build] [12] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2506/12.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2506 --build=12

ukclivecox · 2020-10-12T13:09:27Z

I don't see changes to split_resources for the new helm value, e.g. addition to

seldon-core/operator/helm/split_resources.py

Lines 33 to 59 in 2e62a3a

    
           HELM_ENV_SUBST = { 
        
               "AMBASSADOR_ENABLED": "ambassador.enabled", 
        
               "AMBASSADOR_SINGLE_NAMESPACE": "ambassador.singleNamespace", 
        
               "ENGINE_SERVER_GRPC_PORT": "engine.grpc.port", 
        
               "ENGINE_CONTAINER_IMAGE_PULL_POLICY": "engine.image.pullPolicy", 
        
               "ENGINE_LOG_MESSAGES_EXTERNALLY": "engine.logMessagesExternally", 
        
               "ENGINE_SERVER_PORT": "engine.port", 
        
               "ENGINE_PROMETHEUS_PATH": "engine.prometheus.path", 
        
               "ENGINE_CONTAINER_USER": "engine.user", 
        
               "ENGINE_CONTAINER_SERVICE_ACCOUNT_NAME": "engine.serviceAccount.name", 
        
               "ISTIO_ENABLED": "istio.enabled", 
        
               "ISTIO_GATEWAY": "istio.gateway", 
        
               "ISTIO_TLS_MODE": "istio.tlsMode", 
        
               "PREDICTIVE_UNIT_SERVICE_PORT": "predictiveUnit.port", 
        
               "PREDICTIVE_UNIT_DEFAULT_ENV_SECRET_REF_NAME": "predictiveUnit.defaultEnvSecretRefName", 
        
               "PREDICTIVE_UNIT_METRICS_PORT_NAME": "predictiveUnit.metricsPortName", 
        
               "USE_EXECUTOR": "executor.enabled", 
        
               "EXECUTOR_CONTAINER_IMAGE_PULL_POLICY": "executor.image.pullPolicy", 
        
               "EXECUTOR_SERVER_PORT": "executor.port", 
        
               "EXECUTOR_SERVER_METRICS_PORT_NAME": "executor.metricsPortName", 
        
               "EXECUTOR_PROMETHEUS_PATH": "executor.prometheus.path", 
        
               "EXECUTOR_CONTAINER_USER": "executor.user", 
        
               "EXECUTOR_CONTAINER_SERVICE_ACCOUNT_NAME": "executor.serviceAccount.name", 
        
               "MANAGER_CREATE_RESOURCES": "managerCreateResources", 
        
               "EXECUTOR_REQUEST_LOGGER_DEFAULT_ENDPOINT": "executor.requestLogger.defaultEndpoint", 
        
               "DEFAULT_USER_ID": "defaultUserID", 
        
           }

helm-charts/seldon-core-operator/templates/deployment_seldon-controller-manager.yaml

seldondev · 2020-10-12T13:54:10Z

Mon Oct 12 13:54:04 UTC 2020
The logs for [pr-build] [35] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2506/35.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2506 --build=35

seldondev · 2020-10-12T13:54:14Z

Mon Oct 12 13:54:13 UTC 2020
The logs for [lint] [36] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2506/36.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2506 --build=36

seldondev · 2020-10-12T14:00:07Z

Mon Oct 12 14:00:00 UTC 2020
The logs for [lint] [38] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2506/38.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2506 --build=38

seldondev · 2020-10-12T14:00:07Z

Mon Oct 12 14:00:00 UTC 2020
The logs for [pr-build] [37] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2506/37.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2506 --build=37

ukclivecox · 2020-10-12T14:22:58Z

/test notebooks

ukclivecox · 2020-10-12T14:23:06Z

/approve

seldondev · 2020-10-12T14:24:46Z

Mon Oct 12 14:24:41 UTC 2020
The logs for [notebooks] [39] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2506/39.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2506 --build=39

anggao · 2020-10-12T15:51:29Z

/retest

RafalSkolasinski · 2020-10-14T17:12:33Z

@anggao the test in question (that tests custom metrics) seems to have passed in other PR that was also failing
Could you please make sure you are rebased onto master and then trigger the notebook tests again?

seldondev · 2020-10-14T18:25:27Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cliveseldon

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [cliveseldon]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

seldondev · 2020-10-14T18:26:37Z

Wed Oct 14 18:26:32 UTC 2020
The logs for [lint] [41] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2506/41.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2506 --build=41

seldondev · 2020-10-14T18:26:37Z

Wed Oct 14 18:26:32 UTC 2020
The logs for [pr-build] [40] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2506/40.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2506 --build=40

anggao · 2020-10-14T18:26:47Z

@RafalSkolasinski sure just did it

anggao · 2020-10-14T18:26:58Z

/test notebooks

seldondev · 2020-10-14T18:29:28Z

Wed Oct 14 18:29:26 UTC 2020
The logs for [notebooks] [42] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2506/42.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2506 --build=42

seldondev · 2020-10-14T20:56:56Z

@anggao: The following test failed, say /retest to rerun them all:

Test name	Commit	Details	Rerun command
notebooks	`ce0d57b`	link	`/test notebooks`

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the jenkins-x/lighthouse repository. I understand the commands that are listed here.

ukclivecox · 2020-10-15T07:40:06Z

Thanks @groszewn ! We will investigate flaky tests in master.

seldondev added needs-ok-to-test size/XXL labels Sep 30, 2020

seldondev assigned ukclivecox Sep 30, 2020

seldondev added ok-to-test and removed needs-ok-to-test labels Oct 1, 2020

anggao force-pushed the add-keda-support branch from fadff4c to 2da4cf2 Compare October 1, 2020 15:33

anggao force-pushed the add-keda-support branch from 3a69e0a to 3e74c0a Compare October 6, 2020 13:13

ukclivecox reviewed Oct 12, 2020

View reviewed changes

helm-charts/seldon-core-operator/templates/deployment_seldon-controller-manager.yaml Outdated Show resolved Hide resolved

anggao force-pushed the add-keda-support branch from 76d3b0d to f55578e Compare October 12, 2020 13:57

seldondev added the approved label Oct 12, 2020

anggao added 7 commits October 14, 2020 19:24

Add KEDA support for seldon-core

980ebf2

fix comment

1d749e8

fix build

8296193

fix lint

ea785d1

add keda example

5cfe690

add keda example to notebook end-to-end tests

c027522

fix helm chart

ce0d57b

anggao force-pushed the add-keda-support branch from f55578e to ce0d57b Compare October 14, 2020 18:24

ukclivecox merged commit dd3f71f into SeldonIO:master Oct 15, 2020

Add KEDA support to seldon-core #2506

Add KEDA support to seldon-core #2506

Conversation

anggao commented Sep 30, 2020

seldondev commented Sep 30, 2020

anggao commented Sep 30, 2020

ukclivecox commented Oct 1, 2020

ukclivecox commented Oct 1, 2020

seldondev commented Oct 1, 2020

seldondev commented Oct 1, 2020

anggao commented Oct 1, 2020

anggao commented Oct 1, 2020

seldondev commented Oct 1, 2020

seldondev commented Oct 1, 2020

anggao commented Oct 1, 2020

ukclivecox commented Oct 1, 2020

seldondev commented Oct 2, 2020

seldondev commented Oct 2, 2020

seldondev commented Oct 2, 2020

seldondev commented Oct 2, 2020

anggao commented Oct 2, 2020

seldondev commented Oct 2, 2020

seldondev commented Oct 2, 2020

ukclivecox commented Oct 6, 2020

seldondev commented Oct 6, 2020

seldondev commented Oct 6, 2020

seldondev commented Oct 6, 2020

ukclivecox commented Oct 12, 2020

seldondev commented Oct 12, 2020

seldondev commented Oct 12, 2020

seldondev commented Oct 12, 2020

seldondev commented Oct 12, 2020

ukclivecox commented Oct 12, 2020

ukclivecox commented Oct 12, 2020

seldondev commented Oct 12, 2020

anggao commented Oct 12, 2020

RafalSkolasinski commented Oct 14, 2020

seldondev commented Oct 14, 2020

seldondev commented Oct 14, 2020

seldondev commented Oct 14, 2020

anggao commented Oct 14, 2020

anggao commented Oct 14, 2020

seldondev commented Oct 14, 2020

seldondev commented Oct 14, 2020

ukclivecox commented Oct 15, 2020