Comparison Between Quantized Yolov8n model and Unquantized Yolov8n Model by using MSCOCO Validation Dataset

[BeyzaSimsekk]

I quantized yolov8n model using post-training static quantization method with ONNX Runtime on Kaggle. I can do inference operation for both models, i can see that in the image. Now, i want to compare their performance without visualizing, still on Kaggle environment. I want to validate both of them and see their mAP value, speed etc... I tried to use YOLO's val method but i do not have enough memory space. So, i decided to use simplified MSCOCO dataset. Since i will use pretrained model, i didn't download MSCOCO train2017.zip . I only used val2017 and annotations_trainval2017 folders.
My Questions:

1. I want to simplify my val2017 dataset. How can i do that? It has a lot of data in it. I want to simplify that, like calibration dataset in Calibration part when quantizing.
2. This is my code, but i don't think it is right. It does not work. I think it is because of the definition of my AP value. And im also not sure with the inference part of my code, i removed the vizualization code part but still it does not seem right to me. How can i solve that?
   `
   import onnxruntime as ort
   import numpy as np
   import time
   import cv2 as cv
   from pycocotools.coco import COCO
   from sklearn.metrics import average_precision_score

yolov8n = '/kaggle/working/yolov8n.onnx'
static_quantized = '/kaggle/working/static_quantized.onnx'

def load_model(model_path):
return ort.InferenceSession(model_path)

#preprocess and postprocess operations
def preprocess(frame):
x = cv.resize(frame,(640,640))
image_data = np.array(x).astype(np.float32) / 255.0 #Normalize to 0-1 range
image_data = np.transpose(image_data, (2, 0, 1)) #Converting from HWC to CHW (first one is "Channel"; PyTorch Format)
image_data = np.expand_dims(image_data, axis=0) #Add batch dimension
return image_data

def postprocess(results, img_shape, confidence=0.35, iou=0.35):
img_height, img_width = img_shape[:2]
outputs = np.transpose(np.squeeze(results[0]))
rows = outputs.shape[0]
boxes = []
scores = []
class_ids = []

x_factor = img_width / 640
y_factor = img_height / 640

for i in range(rows):
    classes_scores = outputs[i][4:]
    max_score = np.amax(classes_scores)
    if max_score >= confidence:
        class_id = np.argmax(classes_scores) #Get the class id with the highest score
        x, y, w, h = outputs[i][0], outputs[i][1], outputs[i][2], outputs[i][3] #Extract the bounding box coordinates from the current row
           
        #Calculate the scaled coodinates of the bounding box
        left = int((x - w / 2) * x_factor)
        top = int((y - h / 2) * y_factor)
        width = int(w * x_factor)
        height = int(h * y_factor)
            
        class_ids.append(class_id)
        scores.append(max_score)
        boxes.append([left, top, width, height])
        
#Apply non-maximum suppression to filter out overlapping bounding boxes
indices = cv.dnn.NMSBoxes(boxes, scores, confidence, iou)
filtered_boxes = []
filtered_scores = []

if len(indices) > 0:
    indices = indices.flatten()  # Flatten the array if necessary
    for i in indices:
        box = boxes[i]
        score = scores[i]
        filtered_boxes.append(box)
        filtered_scores.append(score)

return filtered_boxes, filtered_scores

#simply the datasets(val2017) like you did in the calibration dataset***

#Performans Estimation Function
def evaluate_model(model_session, coco_annotation_file):
coco = COCO(coco_annotation_file)
img_ids = coco.getImgIds() # gets the id of all images
all_gt_boxes = [] # a list to store all ground truth boxes
"""
ground truth: the reality you want to model with your supervised machine learning algorithm.
Ground truth is also known as the target for training or validating the model with a labeled dataset.
"""
all_dets = [] # a list to store all the detections made by model
all_scores = [] # a list to store confidence scores of boxes predicted by model

start_time = time.time()

for img_id in img_ids: 
    img_info = coco.loadImgs(img_id)[0] #image informations
    img_path = img_info['file_name']
    
    # Create full path to image
    img_path = f"/kaggle/working/coco/images/val2017/000000571598.jpg"  # DEĞİŞECEK!!!!
    
    img = cv.imread(img_path)
    
    img_data = preprocess(img)
    
    results = model_session.run(None, {"images": img_data}) #inference
    boxes, scores = postprocess(results, img.shape)
    
    #collecting predicted boxes and scores
    all_dets.extend(boxes)
    all_scores.extend(scores)
    #collecting and loading ground truth boxes
    gt_boxes = coco.loadAnns(coco.getAnnIds(imgIds = [img_id]))
    all_gt_boxes.extend(gt_boxes)
    
end_time = time.time()
inference_time = end_time - start_time

#Convert results to format suitable(numpy set) for mAP calculation
dets = np.array(all_dets)
scores = np.array(all_scores)
gt_boxes = [ann['bbox'] for ann in all_gt_boxes] #take only bbox(bounding box) part of the ground truth boxes


#AP FONKSİYONUNU DÜZELT***
ap_scores = []
for i in range(len(dets)):
    if i < len(gt_boxes): #her tahmin için bir gerçek kutu var mı onu kontrol ediyoruz
        gt_box = np.array(gt_boxes[i]) #gerçek kutu verisini numpy dizisine dönüştürür
        dets_for_img = dets[i] #mevcut görüntü için tahmin edilen görüntüleri alır
        scores_for_img = scores[i] #mevcut görüntü için tahmin edilen kutuların confidence skorlarını alır
        
        if len(dets_for_img) > 0:
            #tahminlerin doğruluğunu değerlendirmek için gerçek kutulara ([1] * len(gt_box)) karşılık gelen skorları kullanır.
            ap = average_precision_score([1]*len(gt_box),scores_for_img) 
            ap_scores.append(ap)
            
            
            
mAP = np.mean(ap_scores) if ap_scores else 0
return inference_time, mAP

original_model = load_model(yolov8n)
inference_time_org, mAP_org = evaluate_model(original_model, '/kaggle/working/coco/annotations/annotations/instances_val2017_sampled.json')#DEĞİŞECEK!
print(f'Original Model - Inference Time: {inference_time_org:.2f} seconds, mAP: {mAP_org:.2f}')

`

[glenn-jocher]

@BeyzaSimsekk to simplify your val2017 dataset for validation, you can create a smaller subset similar to how you handle calibration datasets. This involves selecting a representative sample of images and annotations. You can use a script to randomly select a subset of images and their corresponding annotations.

Regarding your code, it seems you are on the right track but may need to refine your approach to calculating mAP and handling the dataset. Ensure that your preprocessing and postprocessing steps align with the model's requirements and that you correctly handle the dataset paths and annotations.

For memory issues, consider using a smaller batch size or reducing the image resolution during validation. This can help manage memory usage more effectively.

If you need further assistance with specific parts of your code or additional guidance on dataset handling, feel free to ask. For more detailed information on TensorRT export and performance optimization, you can refer to the Ultralytics TensorRT documentation.

[glenn-jocher]

@BeyzaSimsekk you're welcome! If you have any more questions or need further assistance with the Ultralytics YOLO models, feel free to ask. The community and the Ultralytics team are here to help.

[BeyzaSimsekk]

Hi @glenn-jocher ! I changed my code and minimaze my dataset. but i have new
issues now. And i wanted to ask you, if that’s okay.
My preprocess and postprocess operations are like this, i changed some
of the parts like i add another output class_ids and for the score part i made it
return float for JSON format when uploading detections.json:

def load_model(model_path):
return ort.InferenceSession(model_path)
#preprocess and postprocess operations
def preprocess(frame):
x = cv.resize(frame,(640,640))
image_data = np.array(x).astype(np.float32) / 255.0 #Normalize to 0-1 range
image_data = np.transpose(image_data, (2, 0, 1)) #Converting from HWC to
CHW (first one is "Channel"; PyTorch Format)
image_data = np.expand_dims(image_data, axis=0) #Add batch dimension
return image_data
def postprocess(results, img_shape, confidence=0.35, iou=0.35):
img_height, img_width = img_shape[:2]
outputs = np.transpose(np.squeeze(results[0]))
rows = outputs.shape[0]
boxes = []
scores = []
class_ids = []
x_factor = img_width / 640
y_factor = img_height / 640
for i in range(rows):
classes_scores = outputs[i][4:]
max_score = np.amax(classes_scores)
if max_score >= confidence:
class_id = np.argmax(classes_scores) #Get the class id with the highest
score
x, y, w, h = outputs[i][0], outputs[i][1], outputs[i][2], outputs[i][3]
#Extract the bounding box coordinates from the current row
#Calculate the scaled coodinates of the bounding box
left = int((x - w / 2) * x_factor)
top = int((y - h / 2) * y_factor)
width = int(w * x_factor)
height = int(h * y_factor)
class_ids.append(class_id)
scores.append(float(max_score)) #burayı değiştirdim***
boxes.append([left, top, width, height])
#Apply non-maximum suppression to filter out overlapping bounding boxes
indices = cv.dnn.NMSBoxes(boxes, scores, confidence, iou)
filtered_boxes = []
filtered_scores = []
filtered_class_ids = []
if len(indices) > 0:
indices = indices.flatten() # Flatten the array if necessary
for i in indices:
box = boxes[i]
score = scores[i]
class_id = class_ids[i]
filtered_boxes.append(box)
filtered_scores.append(score)
filtered_class_ids.append(class_id)
return filtered_boxes, filtered_scores, filtered_class_ids 

I minimize my MSCOCO dataset with supercategory=’vehicle’ including
20 images to test quickly. And this is my evaluation function. I used coco
evaluation method on my quantized yolov8n model and original yolov8n
model.But it returns mAP value as for Yolov8n model = 0.005 (which is definitely
not correct) and for my quantized Yolov8n model = 0.0 which is also not true, i
inference both models on an image with visualize the output and they detected
the objects correctly. but now the mAP values does not seem to correct.

from pycocotools.coco import COCO
from pycocotools.cocoeval import COCOeval
import numpy as np
import os
import json
import cv2 as cv
def get_category_id(coco, category_name):
categories = coco.loadCats(coco.getCatIds())
for cat in categories:
if cat['name'] == category_name:
return cat['id']
return None
def get_class_names(coco):
categories = coco.loadCats(coco.getCatIds())
class_names = {cat['id']: cat['name'] for cat in categories}
return class_names
def evaluate_model(model_session, coco_annotation_file, image_dir):
coco = COCO(coco_annotation_file)
class_names = get_class_names(coco)
img_ids = coco.getImgIds()
all_gt_boxes = []
all_dets = []
for img_id in img_ids:
img_info = coco.loadImgs(img_id)[0]
img_path = os.path.join(image_dir, img_info['file_name'])
img = cv.imread(img_path)
img_data = preprocess(img)
results = model_session.run(None, {"images": img_data})
boxes, scores, class_ids = postprocess(results, img.shape)
# Collect detections
for box, score, class_id in zip(boxes, scores, class_ids):
class_name = class_names.get(class_id, 'unknown') # Get class name
based on class_id
category_id = get_category_id(coco, class_name) # Get category_id
based on class_name
if category_id is not None:
all_dets.append({
'image_id': img_id,
'category_id': category_id, # Use the correct category_id
'bbox': box,
'score': float(score)
})
# Collect ground truth boxes
gt_boxes = coco.loadAnns(coco.getAnnIds(imgIds=[img_id]))
all_gt_boxes.extend([
{'image_id': img_id, 'bbox': ann['bbox'], 'category_id': ann['category_id']}
for ann in gt_boxes
])

# Save detections to a temporary file
with open('/kaggle/working/detections.json', 'w') as f:
json.dump(all_dets, f)
# Save ground truth boxes to a temporary file
with open('/kaggle/working/gt_boxes.json', 'w') as f:
json.dump(all_gt_boxes, f)
coco_dt = coco.loadRes('/kaggle/working/detections.json')
coco_eval = COCOeval(coco, coco_dt, 'bbox')
coco_eval.evaluate()
coco_eval.accumulate()
coco_eval.summarize()
# mAP value
mAP = coco_eval.stats[0] # [email protected]:0.95
return mAP

I also checked my detections.json and gt_boxes.json files and their format
seems correct. for gt_boxes.json: image_id, bbox, category_id ; for
detections.json: image_id, category_id, bbox, score. I dont know how to solve
this problem :”( So, i really appreciate if you would help me.

[BeyzaSimsekk]

while uploading my code block here, my indentations were lost but their indentations are correct in my original code

[glenn-jocher]

Thank you for sharing your detailed code and explanation. It seems like you've made significant progress. To address the mAP discrepancy, ensure that your preprocessing and postprocessing steps align with the model's requirements. Also, verify that the detections.json and gt_boxes.json files are correctly formatted and that the category IDs match between your annotations and detections. If the issue persists, please check if it reproduces with the latest version of the Ultralytics package. This can help identify if there are any underlying issues that have been resolved in recent updates.

[BeyzaSimsekk]

Thank you so much @glenn-jocher ! 😊, Sorry for replying late😞. I improved my code now. I can make a validation operation for one image. Because i validate for 1 image mAP value for Yolov8n model is 25,... and it changes for every image. Now i want to make a validation operation for multiple images such as 50images. For this purpose, I write my evaluation code like this:

from pycocotools.coco import COCO
from pycocotools.cocoeval import COCOeval
import numpy as np
import os
import json
import cv2 as cv

# Retrieve the unique category ID for a given category name from the COCO dataset.
def get_category_id(coco, category_name): 
    categories = coco.loadCats(coco.getCatIds())
    for cat in categories:
        if cat['name'] == category_name:
            return cat['id']
    return None

# Create a dictionary that maps category IDs to their corresponding names from the COCO dataset.
def get_class_names(coco): 
    categories = coco.loadCats(coco.getCatIds())
    class_names = {cat['id']: cat['name'] for cat in categories}
    return class_names


def evaluate_model(inference_instance, coco_annotation_file, image_dir,num_images=50):
    coco = COCO(coco_annotation_file) # Initialize an instance of the COCO API using the specified annotation file
    yolo_class_names = inference_instance.classes  # Class names from YOLO
    coco_class_names = get_class_names(coco) # Class names from COCO
    img_ids = coco.getImgIds() 
    #print(img_ids)
    
    sampled_img_ids = np.random.choice(img_ids, min(num_images, len(img_ids)), replace = False)
    #print(sampled_img_ids)
    
    all_gt_boxes = []
    all_dets = []

    for img_id in sampled_img_ids:  #sampled_img_ids:
        
        img_info = coco.loadImgs(img_id)
        if not img_info:
            print(f"Warning: No image information found for img_id {img_id}")
            continue

        img_info = img_info[0]
        img_path = os.path.join(image_dir, img_info['file_name'])
                       
        img = cv.imread(img_path)

        img_data = preprocessor(img)  
        results = inference_instance.detector(img_data)
        detections = inference_instance.postprocessor(results, img, 0.35, 0.35) #conf_thres & iou_thres = 0.35
        
        # Collect detections
        for det in detections:
            box = det['box']
            score = det['score']
            class_id = det['class_id']
            class_name = yolo_class_names.get(class_id, 'unknown')  # Get class name based on class_id    
            category_id = get_category_id(coco, class_name)  # Get category_id based on class_name   
            if category_id is not None:
                all_dets.append({
                    'image_id': img_id,
                    'category_id': category_id,  # Use the correct category_id
                    'bbox': box,
                    'score': float(score)
                })
        
        # Collect ground truth boxes
        gt_boxes = coco.loadAnns(coco.getAnnIds(imgIds=[img_id]))
        all_gt_boxes.extend([
            {'image_id': img_id, 'bbox': ann['bbox'], 'category_id': ann['category_id']}
            for ann in gt_boxes
        ])

    #print("Sample detections:", all_dets[:5])  # Print a sample of detections
    # Save detections to a temporary file
    with open('/kaggle/working/detections.json', 'w') as f:
        json.dump(all_dets, f)
    
    #print("Sample ground truth boxes:", all_gt_boxes[:5])  # Print a sample of ground truth boxes
    # Save ground truth boxes to a temporary file
    with open('/kaggle/working/gt_boxes.json', 'w') as f:
        json.dump(all_gt_boxes, f)

    coco_dt = coco.loadRes('/kaggle/working/detections.json')
    coco_eval = COCOeval(coco, coco_dt, 'bbox')
    coco_eval.evaluate()
    coco_eval.accumulate()
    coco_eval.summarize()

    # mAP value
    mAP = coco_eval.stats[0]  # [email protected]:0.95
    return mAP

When I run this code using this:

model_path = "/kaggle/working/yolov8n.onnx"
image_dir = "/kaggle/working/coco/images/small_val2017"
coco_annotation_file = "/kaggle/working/coco/images/annotation_small_val/instances_val2017.json"

inference_instance = Inference(model_path, "/kaggle/working/coco/images/small_val2017/000000007816.jpg")
mAP = evaluate_model(inference_instance, coco_annotation_file, image_dir, num_images=50)
print(f"mAP: {mAP}")

I get an error about my detections. json file is empty ; list index out of range.

And when i check my detections json and gt_boxes json file, both of them are empty. because of this part does not work:

for img_id in sampled_img_ids: #sampled_img_ids:

    img_info = coco.loadImgs(img_id)
    if not img_info:
        print(f"Warning: No image information found for img_id {img_id}")
        continue

I checked my sampled_img_ids and they are in the img_ids which i also checked and there is 5000 image ids. but it does not take its img_info. for the output warnings one of them was like this:

Warning: No image information found for img_id 54593

So, i checked this one seperately:

from pycocotools.coco import COCO
from pycocotools.cocoeval import COCOeval

coco_annotation_file = "/kaggle/working/coco/images/annotation_small_val/instances_val2017.json"
coco = COCO(coco_annotation_file)


img_id = 54593

img_info = coco.loadImgs(img_id)

print(img_info)

and it runned perfect:

loading annotations into memory...
Done (t=0.23s)
creating index...
index created!
[{'license': 3, 'file_name': '000000054593.jpg', 'coco_url': 'http://images.cocodataset.org/val2017/000000054593.jpg', 'height': 427, 'width': 640, 'date_captured': '2013-11-23 01:19:17', 'flickr_url': 'http://farm3.staticflickr.com/2731/5846887136_3ced7e71df_z.jpg', 'id': 54593}]

I don't understand why it does not working. And i have no one to ask. I would be really appreciated if you helped me

[BeyzaSimsekk]

I understood my mistake now and solved the issue. also my this approach is not correct i saw that too. Thank you so much for everything, you were great help ☺️

[glenn-jocher]

I'm glad to hear you resolved the issue and found the approach that works for you. If you have any more questions or run into any other issues, feel free to reach out. Happy coding!