ANNOTATIONS USING CVAT (GUIDE)

Website and user login

• The annotations tool and tasks are available at the website http://label.plyzer.ai/. CVAT must be accessed using the Google Chrome Browser. Any other browser is not supported.

• Lahore user login credentials

  • Username: lahore
  • The password will be given via Slack chat

Note: You will be working in parallel with the same user. Since labeling work is not saved and updated in real time for other login sessions, you must distribute and assign tasks or even images in a task between every person performing annotations. This way, you won't label the same image twice. In order for the labeling work done in parallel to appear to the other sessions you must save the work and reload the page.

Introduction

The images used for annotations have been automatically rectified to recover the parallelism between lines and therefore remove the projection distorsion. Bounding-boxes are pre-computed using a deep learning model (SKU110K) and labeled as UNANOTATED_OBJECT since the model can not predict the product label yet.

The model computes bounding-boxes for each product it can see indepently of its trademark. This means that products that must not be labeled may have a bounding-box and vice versa, some products that must be labeled may not have a bounding-box. We are interested on DAMM group products. Following there is a list of the trademarks we are going to be working with. An example picture of products is provided, but this does not mean that just the listed products are going to appear on the datasset, there will be new products of the same trademarks and some given examples may never appear.

The official CVAT user guide is available here. Our instructions and use case guide is the following:

Focus trademarks

1. Estrella Damm can bottle can pack bottle pack
2. Free Damm can can pack bottle pack
3. Free Damm Lemon can can pack
4. Voll Damm can bottle can pack bottle pack
5. Xibeca can bottle can pack bottle pack
6. Damm Lemon can bottle can pack bottle pack
7. Daura can bottle can pack bottle pack
8. Daura Marzen bottle bottle pack
9. Turia can can packbottle pack
10. Complot bottle bottle pack
11. Innis & Gunn Blood red sky bottle pack
12. Malquerida bottle pack
13. AK Damm bottle pack
14. Inedit bottle pack
15. Bock Damm can bottle pack
16. Sazz bottle pack
17. Shofferhoffer can bottle pack
18. Skol Shandy can
19. Skol can bottle pack
20. Skol Sin can
21. Estrella Levante can bottle can pack bottle pack
22. Estrella Levante sin can bottle bottle pack
23. Estrella Sur can bottle bottle pack
24. Estrella Sur sin can bottlebottle pack
25. Punta Este bottlebottle pack
26. Victoria de Málaga can bottle can pack bottle pack
27. Oro Bilbao can bottle bottle pack
28. Keler can bottle can pack bottle pack
29. Verna can bottle
30. Radeberger bottlebottle pack

Labeling process

In order to perform the image labeling while dealing with false positives (products that should not be labeled) and false negatives (products that should be labeled but are not) we propose the following method.

1. Delete all bounding-boxes of objects you are 100% sure they do not belong to DAMM products. Here we list some popuar companies and their trademarks that should not be labeled:

   1. Heineken
      1. El Águila
      2. Desperados
   2. Coronita
   3. Cruzcampo
   4. Amstel
   5. Mahou 5. Cervezas Alhambra
   6. Paulaner
   7. Erdinger
   8. Leffe
   9. Ambar
   10. Carlsberg
   11. Budweiser
   12. Guinness

   This can be done using the Del key (Fn + Del on Mac).

2. Label the remaining boxes with its corresponding product. The labeling is designed in the following way: Trademark_PackageType_ProductType_NumberUnits_UnitVolume

   • Trademark: trademark of the product (one of the listed in the first list)
   • PackageType: Either SINGLE (refering to single products as for example a bottle) or PACK (a pack of products e.g. bottle box)
   • ProductType: Either BOT (refering to bottles) or LATA (refering to cans, LATA is the spanish for can).
   • NumberUnits: Can be 1UNIT for single objects or 6UNIT, 10UNIT, 12UNIT and 24UNIT referencing the number of products found in a pack. In case of packs the number of products it contains should be written on the packaging or possible to be counted, if it's not possible to determine the number of units leave the product as an UNANOTATED_OBJECT.
   • UnitVolume: Defines the volume of the product. For the case of bottles the possible volumes are 25CL, 33CL, 66CL and 1L. Cans can be 33CL or 50CL. If it's not possible to determine the volume leave the product as an UNANOTATED_OBJECT.

   By clicking on the box you want to annotate the selected item will appear highlighted on top of the object list. The object label has to be chosen from the list shown under each object.

   

   Since there are a lot of labels and there are no possible keyboard shortcuts to assign to all of the labels. What can be done in order to speed up the labeling process when there are some objects with the same label is:

   • Label one of those products.

   

   • Delete the rest of the objects of the same product with the UNANOTATED_OBJECT label.

   

   • Copy (Ctrl + C) and paste (Ctrl + V) the labeled shape into the position of the other objects with the same label.

   

3. Adjust the boxes so they outline the product with no remaining space (i.e. there should be no remaining space at the top or the sides, and the surface where the object sits shouldn't appear) and just the frontal face of the product is inside the box, this is meant to keep boxes rectangular, ignoring the sides of the product (e.g. in can-packs that are not seen from the front)

1. At the end of each image you can go to the labels tab and hide all the labels exept the UNANOTATED_OBJECT to check that you didn't leave any product unlabeled except for those that do not have their label available on CVAT.

REMEMBER TO SAVE THE WORK ONCE IN A WHILE

To store the annotations into the server work needs to be saved. Everything that is not saved explicitly will not be kept for the next time you open the job and you will have to start over.

Save work button is into the Menu that can be accessed by the Open Menu button.

Other useful Menu options

The Menu also has a Help tab that will show you all the avaiable keyboard shortcuts. You may find something usefull there in addition to the shortcuts already explained in this file.

Special cases

Six can packs

Six can packs are obiosuly composed of individual cans tied together with some plastic rings. By the moment the detection/classification system is not able to detect a six can pack and it identifies each can individualy, therefore the classification will be performed on each separate can. This is why we must keep both labels, the six can pack label (e.g. ESTRELLADAMM_PACK_LATA_6UNIT_33CL) and the individual can label (ESTRELLADAMM_SINGLE_LATA_1UNIT_33CL). To do this a new shape must be created for the six pack label and then annotate it.

In order to work with both labels without them overlaping and not allowing you to work some labels can be hidden by pressing the circle next to each label in the labels tab.

Partially occluded objects

The automatic projective correction and the nature of the images may cause some objects to be covered by others or not to be completely shown in the image. In order to deal with this situations CVAT offers the occluded property for any shape. This property is turned on/off for the bounding box under the cursor by pressing the Q key. We are only interested in occluded objects that can give any product classification information, other objects should be left unannotated. Objects croped due to the perspective correction (at the bottom or top of the image) should also be annotated as occluded. Basicaly any bounding box that is not the perfect case for that label should be annotated as occluded.

Objects deep in the shelf

The goal of this model is intented to classify objects that are found on the front of the shelf, what is also called the facing. We are not interessed on those products found in the middle or the back of the shelf. This is why we will label just the ones at the front of the shelf, forgetting about the ones that can aso be seen but are not on the fron of the shelf, as it can be seen in the following image.

Note how the cans seen at the back of the shelf are not labeled

Usage tips

New annotations

New annotations can be created using N key. When in the new annotation mode you have to click at one corner of the box and then click again at the oposite corner. The created annotation will have the default label selected at the moment. This default label is the one that appears at the bottom right corner.

Color by label

Colors shown on each object depend on the Color by atribute at the bottom of the page. This can be either Instance (each object with a different color), Group (each object group with a different color) or Label (objects with the same label have the same color). The Color by label option can be useful to better visualise the ongoing work and to see what elements are left to label.