Skip to content

Commit

Permalink
Update helm/docs per changes in supported task discovery
Browse files Browse the repository at this point in the history
Signed-off-by: Jason Parraga <[email protected]>
  • Loading branch information
Sovietaced committed Aug 26, 2024
1 parent 7d59f10 commit db28350
Show file tree
Hide file tree
Showing 16 changed files with 12 additions and 93 deletions.
4 changes: 2 additions & 2 deletions charts/flyte-core/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -195,11 +195,11 @@ helm install gateway bitnami/contour -n flyte
| flyteadmin.serviceMonitor.scrapeTimeout | string | `"30s"` | Sets the timeout after which request to scrape metrics will time out |
| flyteadmin.tolerations | list | `[]` | tolerations for Flyteadmin deployment |
| flyteagent.enabled | bool | `false` | |
| flyteagent.plugin_config.plugins.agent-service | object | `{"defaultAgent":{"endpoint":"dns:///flyteagent.flyte.svc.cluster.local:8000","insecure":true},"supportedTaskTypes":["sensor"]}` | Agent service configuration for propeller. |
| flyteagent.plugin_config.plugins.agent-service | object | `{"defaultAgent":{"endpoint":"dns:///flyteagent.flyte.svc.cluster.local:8000","insecure":true},"supportedTaskTypes":[]}` | Agent service configuration for propeller. |
| flyteagent.plugin_config.plugins.agent-service.defaultAgent | object | `{"endpoint":"dns:///flyteagent.flyte.svc.cluster.local:8000","insecure":true}` | The default agent service to use for plugin tasks. |
| flyteagent.plugin_config.plugins.agent-service.defaultAgent.endpoint | string | `"dns:///flyteagent.flyte.svc.cluster.local:8000"` | The agent service endpoint propeller should connect to. |
| flyteagent.plugin_config.plugins.agent-service.defaultAgent.insecure | bool | `true` | Whether the connection from propeller to the agent service should use TLS. |
| flyteagent.plugin_config.plugins.agent-service.supportedTaskTypes | list | `["sensor"]` | The task types supported by the default agent. |
| flyteagent.plugin_config.plugins.agent-service.supportedTaskTypes | list | `[]` | The task types supported by the default agent. As of #5460 these are discovered automatically and don't need to be configured. |
| flyteagent.podLabels | object | `{}` | Labels for flyteagent pods |
| flyteconsole.affinity | object | `{}` | affinity for Flyteconsole deployment |
| flyteconsole.enabled | bool | `true` | |
Expand Down
8 changes: 3 additions & 5 deletions charts/flyte-core/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -287,11 +287,9 @@ flyteagent:
endpoint: "dns:///flyteagent.flyte.svc.cluster.local:8000"
# -- Whether the connection from propeller to the agent service should use TLS.
insecure: true
# -- The task types supported by the default agent.
supportedTaskTypes:
- sensor
# -- Uncomment to enable task type that uses Flyte Agent
# - bigquery_query_job_task
# -- The task types supported by the default agent. As of #5460 these are discovered automatically and don't
# need to be configured.
supportedTaskTypes: []
# -- Labels for flyteagent pods
podLabels: {}

Expand Down
4 changes: 2 additions & 2 deletions docker/sandbox-bundled/manifests/complete-agent.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -817,7 +817,7 @@ type: Opaque
---
apiVersion: v1
data:
haSharedSecret: dkJZdnpKQ0FYZkhWano2eg==
haSharedSecret: ZHg2VFNJWXNrdG9YVEhvZA==
proxyPassword: ""
proxyUsername: ""
kind: Secret
Expand Down Expand Up @@ -1414,7 +1414,7 @@ spec:
metadata:
annotations:
checksum/config: 8f50e768255a87f078ba8b9879a0c174c3e045ffb46ac8723d2eedbe293c8d81
checksum/secret: 4c91f059d15ecfed81f3906fb24896c41fde9103a61ace577327d080409423da
checksum/secret: 07edf9259ce1c6d595c50929a13b9b06a91655293ac394fe9cead67afe377506
labels:
app: docker-registry
release: flyte-sandbox
Expand Down
4 changes: 2 additions & 2 deletions docker/sandbox-bundled/manifests/complete.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -797,7 +797,7 @@ type: Opaque
---
apiVersion: v1
data:
haSharedSecret: WGJQZFpzb2ZDSkU5dmJReQ==
haSharedSecret: WVZOdERqQktEWWdvM1c0VA==
proxyPassword: ""
proxyUsername: ""
kind: Secret
Expand Down Expand Up @@ -1361,7 +1361,7 @@ spec:
metadata:
annotations:
checksum/config: 8f50e768255a87f078ba8b9879a0c174c3e045ffb46ac8723d2eedbe293c8d81
checksum/secret: 20145f8b7e37f104163904f86eeb0a46444c157de19f8e675128b04d16598ee4
checksum/secret: 32b31fbb2d7281ddbac891c87734245482c5fce18d11c589f1aa38b2bd28f75b
labels:
app: docker-registry
release: flyte-sandbox
Expand Down
4 changes: 2 additions & 2 deletions docker/sandbox-bundled/manifests/dev.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -499,7 +499,7 @@ metadata:
---
apiVersion: v1
data:
haSharedSecret: R2RwSGJNOERJN2NSWXNQNg==
haSharedSecret: SXRvUlE0N1B4eGZqcExXMw==
proxyPassword: ""
proxyUsername: ""
kind: Secret
Expand Down Expand Up @@ -934,7 +934,7 @@ spec:
metadata:
annotations:
checksum/config: 8f50e768255a87f078ba8b9879a0c174c3e045ffb46ac8723d2eedbe293c8d81
checksum/secret: a110328cf7fce9dfe57fe25438d4902fc3cc661346782bb261c0b6b80fb783d1
checksum/secret: a5ded0b6e6f09135a5e9fd69cb5f2a044fcca223bf902a269f6ff1b165ae55a8
labels:
app: docker-registry
release: flyte-sandbox
Expand Down
9 changes: 0 additions & 9 deletions docs/deployment/agents/airflow.rst
Original file line number Diff line number Diff line change
Expand Up @@ -35,11 +35,6 @@ Specify agent configuration
- container_array: k8s-array
- airflow: agent-service
plugins:
agent-service:
supportedTaskTypes:
- airflow
.. group-tab:: Flyte core

Create a file named ``values-override.yaml`` and add the following configuration to it.
Expand All @@ -63,10 +58,6 @@ Specify agent configuration
sidecar: sidecar
container_array: k8s-array
airflow: agent-service
plugins:
agent-service:
supportedTaskTypes:
- airflow
Upgrade the Flyte Helm release
Expand Down
9 changes: 0 additions & 9 deletions docs/deployment/agents/bigquery.rst
Original file line number Diff line number Diff line change
Expand Up @@ -42,11 +42,6 @@ Specify agent configuration
- container: container
- container_array: k8s-array
- bigquery_query_job_task: agent-service
plugins:
agent-service:
supportedTaskTypes:
- bigquery_query_job_task
.. group-tab:: Flyte core

Expand All @@ -71,10 +66,6 @@ Specify agent configuration
sidecar: sidecar
container_array: k8s-array
bigquery_query_job_task: agent-service
plugins:
agent-service:
supportedTaskTypes:
- bigquery_query_job_task
Ensure that the propeller has the correct service account for BigQuery.

Expand Down
4 changes: 0 additions & 4 deletions docs/deployment/agents/chatgpt.rst
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,6 @@ Specify agent configuration
plugins:
agent-service:
supportedTaskTypes:
- chatgpt
# Configuring the timeout is optional.
# Tasks like using ChatGPT with a large model might require a longer time,
# so we have the option to adjust the timeout setting here.
Expand Down Expand Up @@ -70,8 +68,6 @@ Specify agent configuration
chatgpt: agent-service
plugins:
agent-service:
supportedTaskTypes:
- chatgpt
# Configuring the timeout is optional.
# Tasks like using ChatGPT with a large model might require a longer time,
# so we have the option to adjust the timeout setting here.
Expand Down
12 changes: 0 additions & 12 deletions docs/deployment/agents/databricks.rst
Original file line number Diff line number Diff line change
Expand Up @@ -153,10 +153,6 @@ Specify agent configuration
- sidecar
- k8s-array
- agent-service
plugins:
agent-service:
supportedTaskTypes:
- spark
.. group-tab:: Helm chart

Expand All @@ -176,10 +172,6 @@ Specify agent configuration
- container: container
- container_array: k8s-array
- spark: agent-service
plugins:
agent-service:
supportedTaskTypes:
- spark
.. group-tab:: Flyte core

Expand All @@ -201,10 +193,6 @@ Specify agent configuration
sidecar: sidecar
container_array: k8s-array
spark: agent-service
plugins:
agent-service:
supportedTaskTypes:
- spark
Add the Databricks access token
-------------------------------
Expand Down
2 changes: 0 additions & 2 deletions docs/deployment/agents/mmcloud.rst
Original file line number Diff line number Diff line change
Expand Up @@ -71,8 +71,6 @@ Enable the MMCloud agent by adding the following config to the relevant YAML fil
mmcloud-agent:
endpoint: <AGENT_ENDPOINT>
insecure: true
supportedTaskTypes:
- mmcloud_task
agentForTaskTypes:
- mmcloud_task: mmcloud-agent
Expand Down
8 changes: 0 additions & 8 deletions docs/deployment/agents/openai_batch.rst
Original file line number Diff line number Diff line change
Expand Up @@ -32,10 +32,6 @@ Specify agent configuration
- container: container
- container_array: k8s-array
- openai-batch: agent-service
plugins:
agent-service:
supportedTaskTypes:
- openai-batch
.. group-tab:: Flyte core

Expand All @@ -58,10 +54,6 @@ Specify agent configuration
sidecar: sidecar
container_array: k8s-array
openai-batch: agent-service
plugins:
agent-service:
supportedTaskTypes:
- openai-batch
Add the OpenAI API token
------------------------
Expand Down
10 changes: 0 additions & 10 deletions docs/deployment/agents/sagemaker_inference.rst
Original file line number Diff line number Diff line change
Expand Up @@ -33,11 +33,6 @@ Specify agent configuration
- container_array: k8s-array
- boto: agent-service
- sagemaker-endpoint: agent-service
plugins:
agent-service:
supportedTaskTypes:
- boto
- sagemaker-endpoint
.. group-tab:: Flyte core

Expand All @@ -61,11 +56,6 @@ Specify agent configuration
container_array: k8s-array
boto: agent-service
sagemaker-endpoint: agent-service
plugins:
agent-service:
supportedTaskTypes:
- boto
- sagemaker-endpoint
AWS credentials
---------------
Expand Down
9 changes: 0 additions & 9 deletions docs/deployment/agents/sensor.rst
Original file line number Diff line number Diff line change
Expand Up @@ -69,11 +69,6 @@ Enable the sensor agent by adding the following config to the relevant YAML file
- container: container
- container_array: k8s-array
- sensor: agent-service
plugins:
agent-service:
supportedTaskTypes:
- sensor
.. group-tab:: Flyte core

Expand All @@ -98,10 +93,6 @@ Enable the sensor agent by adding the following config to the relevant YAML file
sidecar: sidecar
container_array: k8s-array
sensor: agent-service
plugins:
agent-service:
supportedTaskTypes:
- sensor
Upgrade the deployment
Expand Down
9 changes: 0 additions & 9 deletions docs/deployment/agents/snowflake.rst
Original file line number Diff line number Diff line change
Expand Up @@ -49,11 +49,6 @@ Specify agent configuration
- container_array: k8s-array
- snowflake: agent-service
plugins:
agent-service:
supportedTaskTypes:
- snowflake
.. group-tab:: Flyte core

Create a file named ``values-override.yaml`` and add the following configuration to it.
Expand All @@ -77,10 +72,6 @@ Specify agent configuration
sidecar: sidecar
container_array: k8s-array
snowflake: agent-service
plugins:
agent-service:
supportedTaskTypes:
- snowflake
Ensure that the propeller has the correct service account for Snowflake.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -994,9 +994,6 @@ agent-service (`agent.Config`_)
Value: 50
ProjectScopeResourceConstraint:
Value: 100
supportedTaskTypes:
- task_type_1
- task_type_2
webApi:
caching:
maxSystemFailures: 5
Expand Down
6 changes: 1 addition & 5 deletions docs/flyte_agents/developing_agents.md
Original file line number Diff line number Diff line change
Expand Up @@ -197,10 +197,6 @@ you can route particular task requests to designated agent services by adjusting
```yaml
plugins:
agent-service:
supportedTaskTypes:
- bigquery_query_job_task
- default_task
- custom_task
# By default, all requests will be sent to the default agent.
defaultAgent:
endpoint: "dns:///flyteagent.flyte.svc.cluster.local:8000"
Expand All @@ -224,4 +220,4 @@ you can route particular task requests to designated agent services by adjusting
agentForTaskTypes:
# It will override the default agent for custom_task, which means propeller will send the request to this agent.
- custom_task: custom_agent
```
```

0 comments on commit db28350

Please sign in to comment.