Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add PERIAN Job Platform Agent example #1746

Merged
merged 1 commit into from
Oct 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion _example_template/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
FROM python:3.8-slim-buster
LABEL org.opencontainers.image.source https://github.com/flyteorg/flytesnacks
LABEL org.opencontainers.image.source=https://github.com/flyteorg/flytesnacks

WORKDIR /root
ENV VENV /opt/venv
Expand Down
3 changes: 3 additions & 0 deletions docs/integrations/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,8 @@ Native backend plugins can be executed without any external service dependencies
- Execute tasks using the MemVerge Memory Machine Cloud agent.
* - {doc}`OpenAI Batch </auto_examples/openai_batch_agent/index>`
- Submit requests for asynchronous batch processing on OpenAI.
* - {doc}`PERIAN Job Platform agent </auto_examples/perian_agent/index>`
- Execute tasks on PERIAN Job Platform.
* - {doc}`Sensor agent </auto_examples/sensor/index>`
- Run sensor jobs in your workflows with the sensor agent.
* - {doc}`Snowflake agent </auto_examples/snowflake_agent/index>`
Expand Down Expand Up @@ -235,6 +237,7 @@ ChatGPT agent </auto_examples/chatgpt_agent/index>
Databricks agent </auto_examples/databricks_agent/index>
Memory Machine Cloud agent </auto_examples/mmcloud_agent/index>
OpenAI batch agent </auto_examples/openai_batch_agent/index>
PERIAN Job Platform agent </auto_examples/perian_agent/index>
Sensor agent </auto_examples/sensor/index>
Snowflake agent </auto_examples/snowflake_agent/index>
```
Expand Down
31 changes: 31 additions & 0 deletions examples/perian_agent/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
FROM python:3.8-slim-buster
LABEL org.opencontainers.image.source=https://github.com/flyteorg/flytesnacks

WORKDIR /root
ENV VENV /opt/venv
ENV LANG C.UTF-8
ENV LC_ALL C.UTF-8
ENV PYTHONPATH /root

# This is necessary for opencv to work
RUN apt-get update && apt-get install -y libsm6 libxext6 libxrender-dev ffmpeg build-essential curl

WORKDIR /root

ENV VENV /opt/venv
# Virtual environment
RUN python3 -m venv ${VENV}
ENV PATH="${VENV}/bin:$PATH"

# Install Python dependencies
COPY requirements.in /root
RUN pip install -r /root/requirements.in
RUN pip freeze

# Copy the actual code
COPY . /root

# This tag is supplied by the build script and will be used to determine the version
# when registering tasks, workflows, and launch plans
ARG tag
ENV FLYTE_INTERNAL_IMAGE $tag
21 changes: 21 additions & 0 deletions examples/perian_agent/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
```{eval-rst}
.. tags:: Cloud, GPU, Integration, Advanced
```

(perian_agent)=

# PERIAN Job Platform Agent

The PERIAN Flyte Agent enables you to execute Flyte tasks on the [PERIAN Sky Platform](https://perian.io/). PERIAN allows the execution of any task on servers aggregated from multiple cloud providers.

Example usage:

```{auto-examples-toc}
example
```

To get started with PERIAN, see the [PERIAN documentation](https://perian.io/docs/overview) and the [PERIAN Flyte Agent documentation](https://perian.io/docs/flyte-getting-started).

## Agent setup

Consult the [PERIAN Flyte Agent setup guide](https://perian.io/docs/flyte-setup-guide).
Empty file.
40 changes: 40 additions & 0 deletions examples/perian_agent/perian_agent/example.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# %% [markdown]
# (example)=
# # PERIAN agent example usage
#
# This example shows how to use the PERIAN agent to execute tasks on PERIAN Job Platform.

# %%
from flytekit import ImageSpec, task, workflow
from flytekitplugins.perian_job import PerianConfig

image_spec = ImageSpec(
name="flyte-test",
registry="my-registry",
python_version="3.11",
apt_packages=["wget", "curl", "git"],
packages=[
"flytekitplugins-perian-job",
],
)


# %% [markdown]
# `PerianConfig` configures `PerianTask`. Tasks specified with `PerianConfig` will be executed on PERIAN Job Platform.


# %%
@task(
container_image=image_spec,
task_config=PerianConfig(
accelerators=1,
accelerator_type="A100",
),
)
def perian_hello(name: str) -> str:
return f"hello {name}!"


@workflow
def my_wf(name: str = "world") -> str:
return perian_hello(name=name)
5 changes: 5 additions & 0 deletions examples/perian_agent/requirements.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
flytekit>=1.7.0
wheel
matplotlib
flytekitplugins-deck-standard
flytekitplugins-perian-job
Loading