Revising matchable resource documentation #257

wild-endeavor · 2020-04-12T20:36:55Z

When first working with the matchable resource and configuration endpoints I was a little confused. This is my attempt to alleviate the confusion another newcomer may have with this topic.

As an example to point to in the documentation, this also adds the K8s namespace resourcequota construct to the sandbox deployment. The default settings given are:

        production:
          - projectQuotaCpu:
              value: "5"
          - projectQuotaMemory:
              value: "4000Mi"
        staging:
          - projectQuotaCpu:
              value: "1"
          - projectQuotaMemory:
              value: "2000Mi"
        development:
          - projectQuotaCpu:
              value: "1"
          - projectQuotaMemory:
              value: "2000Mi"

These are intentionally pretty low but should be sufficient to run the examples we have.

katrogan · 2020-04-16T17:28:05Z

kustomize/overlays/sandbox/admindeployment/kustomization.yaml

@@ -16,6 +16,7 @@ configMapGenerator:
  files:
 # Files are read in alphabetical order. To ensure that we create the namespace first, prefix the file name with "aa".
    - clusterresource-templates/aa_namespace.yaml
+    - clusterresource-templates/ab_project-resource-quota.yaml


nit: doesn't need the ab_ we just do that so that namespaces get created first (golang reads the files in alphabetically)

Hmm... when these are done as config maps then, what order are they run in?

the config maps are mounted as files here, no?

katrogan · 2020-04-16T18:24:00Z

rsts/administrator/install/managing_customizable_resources.rst


-In the absence of custom override values, templateData from the `flyteadmin config <https://github.com/lyft/flyteadmin/blob/6a64f00315f8ffeb0472ae96cbc2031b338c5840/flyteadmin_config.yaml#L154,L159>`_ is used as a default.
+Note that these settings can only take on domain, or a project and domain specificity. Project & domain together in Flyte form Kubernetes namespaces. Since Flyte has not tied in the notion of a workflow or a launch plan to any Kubernetes constructs, specifying a workflow or launch plan name doesn't make any sense.


Project & domain together in Flyte form Kubernetes namespaces.

this is not always true - this is now configurable (e.g. for the L5 use case namespaces are now domain only)

That is a good point, let me remove that.

katrogan · 2020-04-16T18:25:11Z

rsts/administrator/install/managing_customizable_resources.rst


 All other inbound CreateExecution requests would use the default values specified in the flyteadmin config (if any).

 *********
 Debugging
 *********

-Use the `get <https://github.com/lyft/flyteidl/blob/ba13965bcfbf7e7bfce40664800aaf1f2a1088a1/protos/flyteidl/service/admin.proto#L395>`_ endpoint
+Use the `get <https://github.com/lyft/flyteidl/blob/ba13965bcfbf7e7bfce40664800aaf1f2a1088a1/protos/flyteidl/service/admin.proto#L395>`__ endpoint


yikes, thanks for correcting all of these 🤦‍♀️

katrogan

Thank you so much for adding all these docs!

wild-endeavor · 2020-04-16T21:10:44Z

make end2end passes locally even though the GitHub workflow is failing, we are still in the process of investigating...

Signed-off-by: Samhita Alla <[email protected]>

Signed-off-by: Haytham Abuelfutuh <[email protected]>

* overriding OutputWriter Signed-off-by: Daniel Rammer <[email protected]> * updated comments Signed-off-by: Daniel Rammer <[email protected]> * added unit test Signed-off-by: Daniel Rammer <[email protected]> * fixed lint issues Signed-off-by: Daniel Rammer <[email protected]>

* Migrate register from docker to flytectl Signed-off-by: yuvraj <[email protected]> * Small fix Signed-off-by: Yuvraj <[email protected]> * added docs for flytectl in readme Signed-off-by: Yuvraj <[email protected]> * remove register command from conatiner Signed-off-by: Yuvraj <[email protected]> * use register command outside the container Signed-off-by: Yuvraj <[email protected]> * remove register from container makefile for all example Signed-off-by: Yuvraj <[email protected]> * fix typo Signed-off-by: Yuvraj <[email protected]> * revert conatiner change Signed-off-by: Yuvraj <[email protected]>

* Revert "Migrate register from docker to flytectl (flyteorg#257)" This reverts commit 1d2f279.

* Adding flytectl.rst to index Signed-off-by: Prafulla Mahindrakar <[email protected]> * Update index.rst Co-authored-by: Niels Bantilan <[email protected]>

Signed-off-by: Pianist038801 <[email protected]> Co-authored-by: Pianist038801 <[email protected]>

# [0.39.0](http://github.com/lyft/flyteconsole/compare/v0.38.0...v0.39.0) (2022-01-11) ### Features * load 100 executions on each filter ([flyteorg#257](http://github.com/lyft/flyteconsole/issues/257)) ([1a0c3ce](http://github.com/lyft/flyteconsole/commit/1a0c3cecbcde382bb0386e46249708d284f2c188))

Signed-off-by: Samhita Alla <[email protected]>

Signed-off-by: Haytham Abuelfutuh <[email protected]>

* overriding OutputWriter Signed-off-by: Daniel Rammer <[email protected]> * updated comments Signed-off-by: Daniel Rammer <[email protected]> * added unit test Signed-off-by: Daniel Rammer <[email protected]> * fixed lint issues Signed-off-by: Daniel Rammer <[email protected]>

* Adding flytectl.rst to index Signed-off-by: Prafulla Mahindrakar <[email protected]> * Update index.rst Co-authored-by: Niels Bantilan <[email protected]>

Yee Hing Tong added 9 commits April 12, 2020 13:36

adding text

3c4e619

formatting nits in other files

1318c0c

add an example resource quota

7c113cd

attempting to add config for local sandbox resource quota as an example

cbe3568

more doc changes

e81c760

rename file

563fb43

nits

70bdb44

remove domain domain

c024505

more text

db31dc4

katrogan reviewed Apr 16, 2020

View reviewed changes

katrogan previously approved these changes Apr 16, 2020

View reviewed changes

pr comment

16ead7c

wild-endeavor dismissed katrogan’s stale review via 16ead7c April 16, 2020 21:07

katrogan approved these changes Apr 16, 2020

View reviewed changes

wild-endeavor merged commit 25d79e3 into master Apr 16, 2020

eapolinario pushed a commit to eapolinario/flyte that referenced this pull request Dec 6, 2022

Updated boilerplate code (flyteorg#257)

2d18760

Signed-off-by: Samhita Alla <[email protected]>

eapolinario pushed a commit to eapolinario/flyte that referenced this pull request Dec 6, 2022

Update propeller to 0.14.2 to pickup compiler change (flyteorg#257)

d9f09e4

Signed-off-by: Haytham Abuelfutuh <[email protected]>

eapolinario pushed a commit to eapolinario/flyte that referenced this pull request Dec 20, 2022

Revert "Migrate register from docker to flytectl" (flyteorg#315)

0b37a26

* Revert "Migrate register from docker to flytectl (flyteorg#257)" This reverts commit 1d2f279.

eapolinario pushed a commit to eapolinario/flyte that referenced this pull request Dec 20, 2022

feat: load 100 executions on each filter (flyteorg#257)

1a0c3ce

Signed-off-by: Pianist038801 <[email protected]> Co-authored-by: Pianist038801 <[email protected]>

eapolinario pushed a commit to eapolinario/flyte that referenced this pull request Aug 9, 2023

Updated boilerplate code (flyteorg#257)

2e28dbb

Signed-off-by: Samhita Alla <[email protected]>

eapolinario pushed a commit to eapolinario/flyte that referenced this pull request Aug 21, 2023

Update propeller to 0.14.2 to pickup compiler change (flyteorg#257)

8262f78

Signed-off-by: Haytham Abuelfutuh <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Revising matchable resource documentation #257

Revising matchable resource documentation #257

wild-endeavor commented Apr 12, 2020 •

edited

Loading

katrogan Apr 16, 2020 •

edited

Loading

wild-endeavor Apr 16, 2020

katrogan Apr 16, 2020

katrogan Apr 16, 2020 •

edited

Loading

wild-endeavor Apr 16, 2020

katrogan Apr 16, 2020

katrogan left a comment

wild-endeavor commented Apr 16, 2020


		In the absence of custom override values, templateData from the `flyteadmin config <https://github.com/lyft/flyteadmin/blob/6a64f00315f8ffeb0472ae96cbc2031b338c5840/flyteadmin_config.yaml#L154,L159>`_ is used as a default.
		Note that these settings can only take on domain, or a project and domain specificity. Project & domain together in Flyte form Kubernetes namespaces. Since Flyte has not tied in the notion of a workflow or a launch plan to any Kubernetes constructs, specifying a workflow or launch plan name doesn't make any sense.

Revising matchable resource documentation #257

Revising matchable resource documentation #257

Conversation

wild-endeavor commented Apr 12, 2020 • edited Loading

katrogan Apr 16, 2020 • edited Loading

Choose a reason for hiding this comment

wild-endeavor Apr 16, 2020

Choose a reason for hiding this comment

katrogan Apr 16, 2020

Choose a reason for hiding this comment

katrogan Apr 16, 2020 • edited Loading

Choose a reason for hiding this comment

wild-endeavor Apr 16, 2020

Choose a reason for hiding this comment

katrogan Apr 16, 2020

Choose a reason for hiding this comment

katrogan left a comment

Choose a reason for hiding this comment

wild-endeavor commented Apr 16, 2020

wild-endeavor commented Apr 12, 2020 •

edited

Loading

katrogan Apr 16, 2020 •

edited

Loading

katrogan Apr 16, 2020 •

edited

Loading