-
Notifications
You must be signed in to change notification settings - Fork 302
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Signed-off-by: wild-endeavor <[email protected]> Signed-off-by: Ketan Umare <[email protected]>
- Loading branch information
1 parent
f310696
commit 9c748c9
Showing
45 changed files
with
957 additions
and
634 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -26,5 +26,5 @@ dist | |
| .python-version | ||
| _build/ | ||
| docs/source/generated/ | ||
| .pytest-flyte | ||
| .pytest_flyte | ||
| htmlcov | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,55 +1,100 @@ | ||
| .. _design-control-plane: | ||
|
|
||
| ############################ | ||
| Control Plane Objects | ||
| ############################ | ||
| For those who require programmatic access to the control plane, certain APIs are available through "control plane classes". | ||
| ################################################### | ||
| FlyteRemote: A Programmatic Control Plane Interface | ||
| ################################################### | ||
|
|
||
| .. warning:: | ||
| For those who require programmatic access to the control plane, the :mod:`~flytekit.remote` module enables you to perform | ||
| certain operations in a python runtime environment. | ||
|
|
||
| The syntax of this section, while it will continue to work, is subject to change. | ||
| Since this section naturally deals with the control plane, this discussion is only relevant for those who have a Flyte | ||
| backend set up and have access to it (a :std:ref:`local sandbox <flyte:deployment-sandbox>` will suffice as well). | ||
|
|
||
| Since this section naturally deals with the control plane, this discussion is only relevant for those who have a Flyte backend set up, and have access to it (a local backend will suffice as well of course). These objects do not rely on the underlying code they represent being locally available. | ||
| *************************** | ||
| Create a FlyteRemote Object | ||
| *************************** | ||
|
|
||
| ******* | ||
| Setup | ||
| ******* | ||
| Similar to the CLIs, this section requires proper configuration. Please follow the setup guide there. | ||
| The :class:`~flytekit.remote.remote.FlyteRemote` class is the entrypoint for programmatically performing operations in a python | ||
| runtime. There are two ways of creating a remote object. | ||
|
|
||
| Unlike the CLI case however, you may need to explicitly target the configuration file like so :: | ||
| **Initialize directly** | ||
|
|
||
| from flytekit.configuration.common import CONFIGURATION_SINGLETON | ||
| CONFIGURATION_SINGLETON.reset_config("/Users/full/path/to/config") | ||
| .. code-block:: python | ||
| from flytekit.remote import FlyteRemote | ||
| remote = FlyteRemote( | ||
| default_project="project", | ||
| default_domain="domain", | ||
| flyte_admin_url="<url>", | ||
| insecure=True, | ||
| ) | ||
| **Initialize from flyte config** | ||
|
|
||
| .. TODO: link documentation to flyte config and environment variables | ||
| This will initialize a :class:`~flytekit.remote.remote.FlyteRemote` object from your flyte config file or environment variable | ||
| overrides: | ||
|
|
||
| ******* | ||
| Classes | ||
| ******* | ||
| This is not an exhaustive list of the objects available, but should provide the reader with the wherewithal to further ascertain for herself additional functionality. | ||
| .. code-block:: python | ||
| Task | ||
| ====== | ||
| To fetch a Task :: | ||
| remote = FlyteRemote.from_config() | ||
| from flytekit.common.tasks.task import SdkTask | ||
| SdkTask.fetch('flytetester', 'staging', 'recipes.core_basic.task.square', '49b6c6bdbb86e974ffd9875cab1f721ada8066a7') | ||
| ***************************** | ||
| Fetching Flyte Admin Entities | ||
| ***************************** | ||
|
|
||
| .. code-block:: python | ||
| Workflow | ||
| ======== | ||
| To inspect a Workflow :: | ||
| flyte_task = remote.fetch_task(name="my_task", version="v1") | ||
| flyte_workflow = remote.fetch_workflow(name="my_workflow", version="v1") | ||
| flyte_launch_plan = remote.fetch_launch_plan(name="my_launch_plan", version="v1") | ||
| from flytekit.common.workflow import SdkWorkflow | ||
| wf = SdkWorkflow.fetch('flytetester', 'staging', 'recipes.core_basic.basic_workflow.my_wf', '49b6c6bdbb86e974ffd9875cab1f721ada8066a7') | ||
| ****************** | ||
| Executing Entities | ||
| ****************** | ||
|
|
||
| WorkflowExecution | ||
| ================= | ||
| This class represents one run of a workflow. The ``execution_name`` used here is just the tail end of the URL you see in your browser when looking at a workflow run. | ||
| You can execute all of these flyte entities, which returns a :class:`~flytekit.remote.workflow_execution.FlyteWorkflowExecution` object. | ||
| For more information on flyte entities, see the See the :ref:`remote flyte entities <remote-flyte-execution-objects>` | ||
| reference. | ||
|
|
||
| .. code-block:: python | ||
| from flytekit.common.workflow_execution import SdkWorkflowExecution | ||
| flyte_entity = ... # one of FlyteTask, FlyteWorkflow, or FlyteLaunchPlan | ||
| execution = remote.execute(flyte_entity, inputs={...}) | ||
| ******************************** | ||
| Waiting for Execution Completion | ||
| ******************************** | ||
|
|
||
| You can use the :meth:`~flytekit.remote.remote.FlyteRemote.wait` method to synchronously wait for the execution to complete: | ||
|
|
||
| .. code-block:: python | ||
| completed_execution = remote.wait(execution) | ||
| You can also pass in ``wait=True`` to the :meth:`~flytekit.remote.remote.FlyteRemote.execute` method. | ||
|
|
||
| .. code-block:: python | ||
| completed_execution = remote.execute(flyte_entity, inputs={...}, wait=True) | ||
| ******************** | ||
| Syncing Remote State | ||
| ******************** | ||
|
|
||
| Use the :meth:`~flytekit.remote.remote.FlyteRemote.sync` method to sync the entity object's state with the remote state | ||
|
|
||
| .. code-block:: python | ||
| synced_execution = remote.sync(execution) | ||
| e = SdkWorkflowExecution.fetch('project_name', 'development', 'execution_name') | ||
| e.sync() | ||
| **************************** | ||
| Inspecting Execution Objects | ||
| **************************** | ||
|
|
||
| As a workflow is made up of nodes (each of which can contain a task, a subworkflow, or a launch plan), a workflow execution is made up of node executions (each of which can contain a task execution, a subworkflow execution, or a launch plan execution). | ||
| At any time you can inspect the inputs, outputs, completion status, error status, and other aspects of a workflow | ||
| execution object. See the :ref:`remote execution objects <remote-flyte-execution-objects>` reference for a list | ||
| of all the available attributes. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.