diff --git a/docs/source/guides/clients.rst b/docs/source/guides/clients.rst index 3adf456df19..55702f14d2d 100644 --- a/docs/source/guides/clients.rst +++ b/docs/source/guides/clients.rst @@ -133,6 +133,50 @@ Alternatively, use the ``server_ready_timeout`` parameter to specify the maximum client.close() +.. _call-a-task-endpoint: + +Call a task endpoint +-------------------- + +You can create clients to interact with Services defined with :ref:`task ` endpoints by submitting inputs and then asynchronously checking for results at a later time. This model is particularly useful for “fire-and-forget" operations where the client does not need to actively wait for the task to complete. Both synchronous (``SyncHTTPClient``) and asynchronous (``AsyncHTTPClient``) clients can be used to call a task endpoint. + +To submit a task: + +.. code-block:: python + + import bentoml + import numpy as np + + input_data = np.array([...]) # Some numpy array + + client = bentoml.SyncHTTPClient('http://localhost:3000') + # The arguments are the same as the Service method, just call with `.submit()` + task = client.long_running_task.submit(input=input_data) + print("Task submitted, ID:", task.id) + +Check the task status and retrieve the result at a later time: + +.. code-block:: python + + status = task.get_status() + if status.value == 'success': + print("The task runs successfully. The result is", task.get()) + elif status.value == 'failure': + print("The task run failed.") + else: + print("The task is still running.") + +Use ``retry()`` if a task fails or you need to rerun the task with the same parameters: + +.. code-block:: python + + status = task.get_status() + if status.value == 'failure': + print("Task failed, retrying...") + new_task = task.retry() + new_status = new_task.get_status() + print("New task status:", new_status.value) + Input and output ---------------- diff --git a/docs/source/guides/services.rst b/docs/source/guides/services.rst index 038f7bab335..b8c97fa904e 100644 --- a/docs/source/guides/services.rst +++ b/docs/source/guides/services.rst @@ -140,7 +140,7 @@ BentoML's lifecycle hooks provide a way to insert custom logic at specific stage You use decorators to set lifecycle hooks. For details, see :doc:`/guides/lifecycle-hooks`. Synchronous and asynchronous APIs ----------------------------------- +--------------------------------- APIs in a BentoML Service can be defined as either synchronous functions or asynchronous coroutines in Python. @@ -259,6 +259,30 @@ However, directly calling synchronous blocking functions within an asynchronous In this example, the ``.to_async`` property converts synchronous methods (``txt2img`` and ``synthesize`` of ``SDXLTurboService`` and ``XTTSService`` respectively) into their asynchronous versions, enabling the ``generate_card`` method to perform multiple asynchronous operations concurrently with ``asyncio.gather``. +.. _bentoml-tasks: + +Tasks +----- + +Tasks in BentoML allow you to execute long-running operations in the background, managed via a task queue style API. These background tasks are ideal for scenarios like batch processing and image or video generation where you don't need the results immediately or synchronously. + +To define a task endpoint, use the ``@bentoml.task`` decorator in the Service constructor. Here's an example: + +.. code-block:: python + + import bentoml + import numpy as np + + @bentoml.service + class MyService: + @bentoml.task + def long_running_task(self, input: np.ndarray) -> np.ndarray: + # Simulated long-running process + # Placeholder for actual processing logic + return result + +BentoML automatically exposes several endpoints for clients to manage the task, such as task submission, status retrieval, and cancellation. For more information, see :ref:`call-a-task-endpoint`. + Convert legacy Runners to a Service -----------------------------------