Skip to content

Commit

Permalink
[FEAUTURE] argilla: working with webhooks (#5502)
Browse files Browse the repository at this point in the history
# Description
<!-- Please include a summary of the changes and the related issue.
Please also include relevant motivation and context. List any
dependencies that are required for this change. -->

This PR adds the `argilla.webhooks` module to work with webhooks from
the SDK. With the changes on this PR, users can easily create webhook
listeners using the python SDK. For example, listening changes can be
defined as follows:

```python
import argilla as rg


@rg.webhook_listener(events="response.updated")
async def on_response_updated(response: rg.UserResponse, **kwargs):
   ... # do your work 

@rg.webhook_listener(events=["dataset.created", "dataset.updated", "dataset.published"])
async def on_dataset_event(
    type: str,
    timestamp: datetime,
    dataset: rg.Dataset,
    **kwargs,
):
    print(f"Event type {type} at {timestamp}")
    print(dataset.settings)

```

You can find a fully basic example using webhooks
[here](https://github.com/argilla-io/argilla/tree/feat/argilla/working-with-webhooks/examples/webhooks/basic-webhooks)

This is still a draft PR and the final feature may change.


Refs: #4658

**Type of change**
<!-- Please delete options that are not relevant. Remember to title the
PR according to the type of change -->

- New feature (non-breaking change which adds functionality)

**How Has This Been Tested**
<!-- Please add some reference about how your feature has been tested.
-->

**Checklist**
<!-- Please go over the list and make sure you've taken everything into
account -->

- I added relevant documentation
- I followed the style guidelines of this project
- I did a self-review of my code
- I made corresponding changes to the documentation
- I confirm My changes generate no new warnings
- I have added tests that prove my fix is effective or that my feature
works
- I have added relevant notes to the CHANGELOG.md file (See
https://keepachangelog.com/)

---------

Co-authored-by: David Berenstein <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Sara Han <[email protected]>
Co-authored-by: José Francisco Calvo <[email protected]>
  • Loading branch information
5 people authored Oct 8, 2024
1 parent 8a90dcc commit 22dfb92
Show file tree
Hide file tree
Showing 23 changed files with 3,015 additions and 23 deletions.
18 changes: 17 additions & 1 deletion argilla/docs/how_to_guides/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,22 @@ These guides provide step-by-step instructions for common scenarios, including d

<div class="grid cards" markdown>

- __Use webhooks to respond to server events__

---

Learn how to use Argilla webhooks to receive notifications about events in your Argilla Server.

[:octicons-arrow-right-24: How-to guide](webhooks.md)

- __Webhooks internals__

---

Learn how Argilla webhooks are implented under the hood and the structure of the different events.

[:octicons-arrow-right-24: How-to guide](webhooks_internals.md)

- __Use Markdown to format rich content__

---
Expand All @@ -98,4 +114,4 @@ These guides provide step-by-step instructions for common scenarios, including d

[:octicons-arrow-right-24: How-to guide](migrate_from_legacy_datasets.md)

</div>
</div>
160 changes: 160 additions & 0 deletions argilla/docs/how_to_guides/webhooks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@
---
description: In this section, we will provide a step-by-step guide to create a webhook in Argilla.
---

# Use Argilla webhooks

This guide provides an overview of how to create and use webhooks in Argilla.

A **webhook** allows an application to submit real-time information to other applications whenever a specific event occurs. Unlike traditional APIs, you won’t need to poll for data very frequently in order to get it in real time. This makes webhooks much more efficient for both the provider and the consumer.

## Creating a webhook listener in Argilla

The python SDK provides a simple way to create a webhook in Argilla. It allows you to focus on the use case of the webhook and not on the implementation details. You only need to create your event handler function with the `webhook_listener` decorator.

```python
import argilla as rg

from datetime import datetime
from argilla import webhook_listener

@webhook_listener(events="dataset.created")
async def my_webhook_handler(dataset: rg.Dataset, type: str, timestamp: datetime):
print(dataset, type, timestamp)
```

In the example above, we have created a webhook that listens to the `dataset.created` event.
> You can find the list of events in the [Events](#events) section.
The python SDK will automatically create a webhook in Argilla and listen to the specified event. When the event is triggered,
the `my_webhook_handler` function will be called with the event data. The SDK will also parse the incoming webhook event into
a proper resource object (`rg.Dataset`, `rg.Record`, and `rg.Response`). The SDK will also take care of request authentication and error handling.

## Running the webhook server

Under the hood, the SDK uses the `FastAPI` framework to create the webhook server and the POST endpoint to receive the webhook events.

To run the webhook, you need to define the webhook server in your code and start it using the `uvicorn` command.

```python
# my_webhook.py file
from argilla import get_webhook_server

server = get_webhook_server()
```

```bash
uvicorn my_webhook:server
```

You can explore the Swagger UI to explore your defined webhooks by visiting `http://localhost:8000/docs`.


The `uvicorn` command will start the webhook server on the default port `8000`.

By default, the Python SDK will register the webhook using the server URL `http://127.0.0.1:8000/`. If you want to use a different server URL, you can set the `WEBHOOK_SERVER_URL` environment variable.

```bash
export WEBHOOK_SERVER_URL=http://my-webhook-server.com
```

All incoming webhook events will be sent to the specified server URL.

## Webhooks management

The Python SDK provides a simple way to manage webhooks in Argilla. You can create, list, update, and delete webhooks using the SDK.

### Create a webhook

To create a new webhook in Argilla, you can define it in the `Webhook` class and then call the `create` method.

```python
import argilla as rg

client = rg.Argilla(api_url="<api_url>", api_key="<api_key>")

webhook = rg.Webhook(
url="http://127.0.0.1:8000",
events=["dataset.created"],
description="My webhook"
)

webhook.create()

```

### List webhooks

You can list all the existing webhooks in Argilla by accessing the `webhooks` attribute on the Argilla class and iterating over them.

```python
import argilla as rg

client = rg.Argilla(api_url="<api_url>", api_key="<api_key>")

for webhook in client.webhooks:
print(webhook)

```

### Update a webhook

You can update a webhook using the `update` method.

```python
import argilla as rg

client = rg.Argilla(api_url="<api_url>", api_key="<api_key>")

webhook = rg.Webhook(
url="http://127.0.0.1:8000",
events=["dataset.created"],
description="My webhook"
).create()

webhook.events = ["dataset.updated"]
webhook.update()

```
> You should use IP address instead of localhost since the webhook validation expect a Top Level Domain (TLD) in the URL.
### Delete a webhook

You can delete a webhook using the `delete` method.

```python
import argilla as rg

client = rg.Argilla(api_url="<api_url>", api_key="<api_key>")

for webhook in client.webhooks:
webhook.delete()

```

## Deploying a webhooks server in a Hugging Face Space

You can deploy your webhook in a Hugging Face Space. You can visit this [link](https://huggingface.co/spaces/argilla/argilla-webhooks/tree/main) to explore an example of a webhook server deployed in a Hugging Face Space.


## Events

The following is a list of events that you can listen to in Argilla, grouped by resource type.

### Dataset events

- `dataset.created`: The Dataset resource was created.
- `dataset.updated`: The Dataset resource was updated.
- `dataset.deleted`: The Dataset resource was deleted.
- `dataset.published`: The Dataset resource was published.

### Record events
- `record.created`: The Record resource was created.
- `record.updated`: The Record resource was updated.
- `record.deleted`: The Record resource was deleted.
- `record.completed`: The Record resource was completed (status="completed").

### Response events
- `response.created`: The Response resource was created.
- `response.updated`: The Response resource was updated.
- `response.deleted`: The Response resource was deleted.
Loading

0 comments on commit 22dfb92

Please sign in to comment.