Skip to content

Commit

Permalink
Add PERIAN Job Platform example
Browse files Browse the repository at this point in the history
Signed-off-by: Omar Tarabai <[email protected]>
  • Loading branch information
otarabai committed Oct 16, 2024
1 parent 1be1081 commit ed85d99
Show file tree
Hide file tree
Showing 7 changed files with 101 additions and 1 deletion.
2 changes: 1 addition & 1 deletion _example_template/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
FROM python:3.8-slim-buster
LABEL org.opencontainers.image.source https://github.com/flyteorg/flytesnacks
LABEL org.opencontainers.image.source=https://github.com/flyteorg/flytesnacks

WORKDIR /root
ENV VENV /opt/venv
Expand Down
3 changes: 3 additions & 0 deletions docs/integrations/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,8 @@ Native backend plugins can be executed without any external service dependencies
- Execute tasks using the MemVerge Memory Machine Cloud agent.
* - {doc}`OpenAI Batch </auto_examples/openai_batch_agent/index>`
- Submit requests for asynchronous batch processing on OpenAI.
* - {doc}`PERIAN Job Platform agent </auto_examples/perian_agent/index>`
- Execute tasks on PERIAN Job Platform.
* - {doc}`Sensor agent </auto_examples/sensor/index>`
- Run sensor jobs in your workflows with the sensor agent.
* - {doc}`Snowflake agent </auto_examples/snowflake_agent/index>`
Expand Down Expand Up @@ -235,6 +237,7 @@ ChatGPT agent </auto_examples/chatgpt_agent/index>
Databricks agent </auto_examples/databricks_agent/index>
Memory Machine Cloud agent </auto_examples/mmcloud_agent/index>
OpenAI batch agent </auto_examples/openai_batch_agent/index>
PERIAN Job Platform agent </auto_examples/perian_agent/index>
Sensor agent </auto_examples/sensor/index>
Snowflake agent </auto_examples/snowflake_agent/index>
```
Expand Down
31 changes: 31 additions & 0 deletions examples/perian_agent/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
FROM python:3.8-slim-buster
LABEL org.opencontainers.image.source=https://github.com/flyteorg/flytesnacks

WORKDIR /root
ENV VENV /opt/venv
ENV LANG C.UTF-8
ENV LC_ALL C.UTF-8
ENV PYTHONPATH /root

# This is necessary for opencv to work
RUN apt-get update && apt-get install -y libsm6 libxext6 libxrender-dev ffmpeg build-essential curl

WORKDIR /root

ENV VENV /opt/venv
# Virtual environment
RUN python3 -m venv ${VENV}
ENV PATH="${VENV}/bin:$PATH"

# Install Python dependencies
COPY requirements.in /root
RUN pip install -r /root/requirements.in
RUN pip freeze

# Copy the actual code
COPY . /root

# This tag is supplied by the build script and will be used to determine the version
# when registering tasks, workflows, and launch plans
ARG tag
ENV FLYTE_INTERNAL_IMAGE $tag
21 changes: 21 additions & 0 deletions examples/perian_agent/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
```{eval-rst}
.. tags:: Cloud, GPU, Integration, Advanced
```

(perian_agent)=

# PERIAN Job Platform Agent

The PERIAN Flyte Agent enables you to execute Flyte tasks on the [PERIAN Sky Platform](https://perian.io/). PERIAN allows the execution of any task on servers aggregated from multiple cloud providers.

Example usage:

```{auto-examples-toc}
example
```

To get started with PERIAN, see the [PERIAN documentation](https://perian.io/docs/overview) and the [PERIAN Flyte Agent documentation](https://perian.io/docs/flyte-getting-started).

## Agent setup

Consult the [PERIAN Flyte Agent setup guide](https://perian.io/docs/flyte-setup-guide).
Empty file.
40 changes: 40 additions & 0 deletions examples/perian_agent/perian_agent/example.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# %% [markdown]
# (example)=
# # PERIAN agent example usage
#
# This example shows how to use the PERIAN agent to execute tasks on PERIAN Job Platform.

# %%
from flytekit import ImageSpec, task, workflow
from flytekitplugins.perian_job import PerianConfig

image_spec = ImageSpec(
name="flyte-test",
registry="my-registry",
python_version="3.11",
apt_packages=["wget", "curl", "git"],
packages=[
"flytekitplugins-perian-job",
],
)


# %% [markdown]
# `PerianConfig` configures `PerianTask`. Tasks specified with `PerianConfig` will be executed on PERIAN Job Platform.


# %%
@task(
container_image=image_spec,
task_config=PerianConfig(
accelerators=1,
accelerator_type="A100",
),
)
def perian_hello(name: str) -> str:
return f"hello {name}!"


@workflow
def my_wf(name: str = "world") -> str:
return perian_hello(name=name)
5 changes: 5 additions & 0 deletions examples/perian_agent/requirements.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
flytekit>=1.7.0
wheel
matplotlib
flytekitplugins-deck-standard
flytekitplugins-perian-job

0 comments on commit ed85d99

Please sign in to comment.