Skip to content

Commit

Permalink
Some more reading improvements
Browse files Browse the repository at this point in the history
  • Loading branch information
nataliaElv committed Jun 21, 2024
1 parent b1289a7 commit ec15a7d
Showing 1 changed file with 13 additions and 15 deletions.
28 changes: 13 additions & 15 deletions argilla/docs/how_to_guides/annotate.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ description: In this section, we will provide a step-by-step guide to show how t
# Annotate your dataset

!!! note ""
To experience the UI features firsthand, you can take a look to the [Demo ↗](https://demo.argilla.io/sign-in?auth=ZGVtbzoxMjM0NTY3OA==).
To experience the UI features firsthand, you can take a look at the [Demo ↗](https://demo.argilla.io/sign-in?auth=ZGVtbzoxMjM0NTY3OA==).

Argilla UI offers many functions to help you manage your annotation workflow, aiming to provide the most flexible approach to fit the wide variety of use cases handled by the community.

Expand All @@ -21,12 +21,13 @@ The app is responsive, which enables you to adapt your workspace from two to one
At the right side of the navigation breadcrumb, you can customize the dataset settings and edit your profile.

=== "Left pane"
This area displays the **control panel** on the top. The control panel is used for performing keyword-based search, applying filters, and sorting the results.

This area is displayed on top of **the control panel** for performing searches, applying filters and sorting results. The record card(s) are displayed one by one or in a vertical list depending on the active view: **Focus view** or **Bulk view.** A card includes one or many fields and an ellipsis menu to mainly access the record extra info like the metadata.
Below the control panel, the record card(s) are displayed one by one (Focus view) or in a vertical list (Bulk view).

=== "Right pane"

This is where you annotate your dataset. Simply fill it out as a form, then choose to Submit, Save as Draft, or Discard your response to send the records to their corresponding queues.
This is where you annotate your dataset. Simply fill it out as a form, then choose to Submit, Save as Draft, or Discard to send the records to their corresponding queues.

=== "Left bottom panel"

Expand Down Expand Up @@ -75,12 +76,9 @@ If you are starting an annotation effort, all the records are initially kept in

### Suggestions

If your dataset includes model predictions, you will see them represented by a sparkle icon `` in the label or value button. We call them “Suggestions” and they appear in the form as pre-filled responses. If you agree with the suggestion, you just need to click on the Submit button, and they will be considered as your response.
If your dataset includes model predictions, you will see them represented by a sparkle icon `` in the label or value button. We call them “Suggestions” and they appear in the form as pre-filled responses. If confidence scores are provided, they will be shown with the label or value as well. Additionally, admins can choose to always show suggested labels at the beginning of the list. This can be configured from the dataset settings.

If the suggestion is incorrect, you can modify it and submit your response.

The score per suggested label/value is displayed for MultiLabelQuestion and RankingQuestion.
For label questions, admins can also choose the ordering of labels in the datasets settings. Additionally, admins can choose to always show suggested labels at the beginning of the list.
If you agree with the suggestions, you just need to click on the Submit button, and they will be considered as your response. If the suggestion is incorrect, you can modify it and submit your final response.

### Focus view

Expand All @@ -91,7 +89,7 @@ This is the default view to annotate your dataset linearly, displaying one recor
!!! tip
You should use this when getting acquainted with a dataset or when the annotation team is very diverse, the topic is generic.

Once you submit your first response, the next record will appear automatically. To see again your submitted record, just click on prev.
Once you submit your first response, the next record will appear automatically. To see again your submitted record, just click on `Prev`.

**Navigating through the records**

Expand All @@ -105,21 +103,21 @@ Each time the page is fully refreshed, the records with modified statuses (Pendi

The bulk view is designed to speed up the annotation and get a quick overview of the whole dataset.

It displays the records in a vertical list. Once the view is active, some functions from the **control panel** will be available to optimize the records reading. You define the number of records to display by page between `10`, `25`, `50``100` and the option to fix the card height by selecting `Expand records` or `Collapse records`.
The bulk view displays the records in a vertical list. Once this view is active, some functions from the control panel will activate to optimize the view. You can define the number of records to display by page between `10`, `25`, `50`, `100` and whether records are shown with a fixed (`Collapse records`) or their natural height (`Expand records`).

!!! tip
You should use this when you have a good understanding of your data and want to apply your knowledge based on things like similarity search, filter patterns, and suggestion score thresholds.

With multiple questions, think about using the bulk view to annotate massively one question. Then, you can complete the annotation per records from the draft queue.
With multiple questions, think about using the bulk view to annotate massively one question. Then, you can complete the annotation per record from the draft queue.

!!! note
You should consider that suggestions are not shown in bulk view (except for Spans) and that you will need to save as a draft when you are not providing responses to all required questions.

### Annotation progress

On the dataset list, the global progress of the annotation task from all users is displayed. This is indicated in the `Global progress` column, which shows the number of records still to be annotated, along with a progress bar. The progress bar displays the percentage and number of records submitted, conflicting (i.e., those with both submitted and discarded responses), discarded, and pending by hovering your mouse over it.
The global progress of the annotation task from all users is displayed in the dataset list. This is indicated in the `Global progress` column, which shows the number of records still to be annotated, along with a progress bar. The progress bar displays the percentage and number of records submitted, conflicting (i.e., those with both submitted and discarded responses), discarded and pending by hovering your mouse over it.

You can track your annotation progress in real time from the righ-bottom panel inside the dataset page. This means that, while you are annotating, the progress bar updates as you submit or discard a record. Expanding the panel, the distribution of `Pending``Draft`, `Submitted` and `Discarded` responses is displayed in a donut chart.
You can track your annotation progress in real time from the righ-bottom panel inside the dataset page. This means that, while you are annotating, the progress bar updates as you submit or discard a record. Expanding the panel, the distribution of `Pending``Draft`, `Submitted` and `Discarded` responses is displayed in a donut chart.

## Discover patterns and speed up your annotation

Expand All @@ -139,7 +137,7 @@ You can retrieve records based on their similarity to another record if vectors
!!! note
Consult these guides to know how to add vectors to your [dataset](dataset.md) and [records](record.md).

To initiate a semantic search, click on `Find similar` within the record you wish to use as a reference. If multiple vectors are available, select the desired vector or choose whether to retrieve the most or least similar records.
To initiate a semantic search, click on `Find similar` within the record you wish to use as a reference. If multiple vectors are available, select the desired vector. You can also choose whether to retrieve the most or least similar records.

The retrieved records are then ordered by similarity, with the similarity score displayed on each record card.

Expand Down Expand Up @@ -186,4 +184,4 @@ The suggestion scores, response, and suggestion values for rating questions and
As an `owner` or `admin`, you can edit the guidelines as much as you need from the icon settings on the header. Markdown format is enabled.

!!! tip
If you want further guidance on good practices for guidelines during the project development, check our [blog post](https://argilla.io/blog/annotation-guidelines-practices/).
If you want further guidance on good practices for guidelines during the project development, check this [blog post](https://argilla.io/blog/annotation-guidelines-practices/).

0 comments on commit ec15a7d

Please sign in to comment.