Possible enhance ambassador customization in seldon deployment

[yufengshan]

If one wants to customize the ambassador configuration for a certain Seldon deployment, the user needs to add an annotation to the seldondeployment yaml file by following the link at Seldon document page:
https://docs.seldon.io/projects/seldon-core/en/latest/ingress/ambassador.html#custom-amabassador-configuration

For example, if a user wants to set the num_retries of ambassador for a certain backend service, he/she needs to include an annotation in the seldondeployment yaml file (an example is at the bottom of this page). The annotation looks verbose and easy to make errors. Can Seldon team work on an easy way to customize the ambassador? maybe parameterize some params?

One more question here, I checked the envoy document online, the default value for num_retries at envoy is set to 1. At Seldon, it is set to 3 as default. Are there any reason for choosing 3?

https://www.envoyproxy.io/learn/automatic-retries
By default, Envoy will set the number of retries to one with num_retries. There’s little downside to increasing this to three, especially for relatively short requests, as Envoy will limit the total time spent to the overall request timeout, including the initial request and all retries.

======================= seldondeployment yaml file annotation =======

kind: SeldonDeployment
metadata:
  labels:
    app: seldon
  name: monitor-backend-service-amba
  namespace: seldon-monitor
spec:
  annotations:
    seldon.io/headless-svc: "true"
    seldon.io/ambassador-config: "apiVersion: ambassador/v1\n\
      kind: Mapping\n\
      name: seldon_seldon-monitor_monitor-backend-service-amba_main_rest_mapping\n\
      prefix: /seldon/seldon-monitor/monitor-backend-service-amba/\n\
      service: monitor-backend-service-amba-monitor-backend-service-amba-main.seldon-monitor:8000\n\
      timeout_ms: 3000\n\
      weight: 100\n\
      retry_policy:\n\
      \  retry_on: connect-failure\n\
      \  num_retries: 1\n\
      --- \n\
      apiVersion: ambassador/v1\n\
      kind: Mapping\n\
      name: seldon_seldon-monitor_monitor-backend-service-amba_main_grpc_mapping\n\
      grpc: true\n\
      prefix: /seldon.protos.Seldon\n\
      rewrite: /seldon.protos.Seldon\n\
      service: monitor-backend-service-amba-monitor-backend-service-amba-main.seldon-monitor:5001\n\
      timeout_ms: 3000\n\
      weight: 100\n\
      headers:\n\
      \  namespace: seldon-monitor\n\
      \  seldon: monitor-backend-service-amba\n\
      retry_policy:\n\
      \  retry_on: connect-failure\n\
      \  num_retries: 2\n"
  name: monitor-backend-service-amba

[ukclivecox]

Could we extend the annotations available to cover some core basic Ambassador configs : https://docs.seldon.io/projects/seldon-core/en/latest/ingress/ambassador.html ?

Any alternative suggestions?

[ukclivecox]

I suggest we change to 1 reties following envoy recommendation?

[yufengshan]

Hi Clive Can we just completely turn off retries and add method to turn it on (with num_retries) if needed.  For our use cases, we try to minimize the service delay for our serving models. I believe this is also a typical use cases for other companies.  thanksYufeng On Wednesday, January 29, 2020, 06:55:03 AM EST, cliveseldon <[email protected]> wrote: I suggest we change to 1 reties following envoy recommendation? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[ukclivecox]

@yufengshan can you review the PR #1383 ?

[yufengshan]

HI Clive, The code looks good to me. Thanks for the quick response.  RegarsYufeng On Thursday, January 30, 2020, 03:21:23 AM EST, cliveseldon <[email protected]> wrote: @yufengshan can you review the PR #1383 ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[divyadilip91]

Hi,
Is it possible to customize the seldon ambassador url?
I know that the url prefix can be changed from the default.
By default it is http:///seldon///api/v0.1/predictions.
I have seen cases where we can change the prefix upto api.
Is it possible to completely change the url according to user requirement?That is without the portion /api/v0.1/predictions?
Kindly help