Get lower AP using calculated anchors when training VOC person. That's weird!

[Paul0629]

Dear AlexeyAB,
I changed the anchors for the voc person using calc_anchors , I got 42.95% AP at 900 iteration, but the AP is 57.59% when using the author's anchors. That's pretty strange. Should I continue the training and try iteration 10000 or more? @AlexeyAB
PS. When I use the gen_anchors.py, I get 39.02% AP at 900 iteration, lower again.
I don't know why I get much different anchors using gen_anchor and calc_anchors with VOC person.
gen_anchors -->22,55, 39,114, 61,224, 85,124, 112,205, 119,314, 202,351, 204,227, 341,366
0.715856
calc_anchors-->26.8837,57.0492, 52.2642,118.8743, 69.2569,194.7763, 148.4303,173.4197, 102.5254,270.5471, 137.3758,351.0699, 218.5627,265.9619, 235.7346,372.9765, 363.5286,367.1107
the author's-->10,13, 16,30, 33,23, 30,61, 62,45, 59,119, 116,90, 156,198, 373,326

[Paul0629]

Another question. I want to train VOC person and then add COCO person and then add cityperson. Should I use the same anchors from begining to end or change to the adding dataset anchors while I'm adding new dataset. Which do you recommend?

[pascal1129]

Dear AlexeyAB,
I have the similar problem with Paul0629, it is quite difficult to figure out. Thus, i turn to you for help.

YOLOv3 trainde on VOC2007(person class) failed to detect small objects , anchors are changed but have no effect. Here are the results. First one is mine, and the second one is predicted by origin yolov3.

As you can see, in my prediction, several human failde to detected.

I plan to train models based on YOLOv3, in order to realize human detection. I extracted person pictures in VOC2007 to form the dataset. I modified YOLOv3-voc.cfg and other configuration files. Beyond douubt, the trained model realized human detection. However, it failed to detect smanll objects. During training, Region 106 has lots of nan, which i think caused the failure.
In order to solve it, i changed the origin anchors.

Origin anchors in yolov3:
10,13, 16,30, 33,23, 30,61, 62,45, 59,119, 116,90, 156,198, 373,326
New anchors created by kmenas:
24,57, 39,116, 60,217, 86,128, 99,298, 135,203, 173,342, 251,272, 321,382

Now the number of nans in region 106 decreased, so i think it would work. However, while testing, the trained model still failed to detect samll objects. At the same time, the loss after trainning is bigger, comperde with origin anchors.
After that i tried severals sets of anchors, it stilled failed.
So i‘m quite troubled T.T

Here are the logs before and after modified.

yolo3_anchors_modified_log_part.txt
yolo3_unmodified_log_part.txt

[AlexeyAB]

@Paul0629
Author's anchors are calculated for dataset that consists of MSCOCO+PascaVOC2007+PascalVOC2012 datasets.

I changed the anchors for the voc person using calc_anchors , I got 42.95% AP at 900 iteration, but the AP is 57.59% when using the author's anchors.

Did you get test.txt and difficult.txt files to check mAP, or only test.txt?

PascalVOC requires to get two files test.txt (without small objects) and difficult.txt (with small objects) to get the correct mAP, as described here: https://github.com/AlexeyAB/darknet#how-to-calculate-map-on-pascalvoc-2007

---

Also better to use dataset that consists of MSCOCO+PascaVOC2007+PascalVOC2012 datasets.
And get labels for PascaVOC2007+PascalVOC2012 by using this script, just change this line:

darknet/scripts/voc_label.py

Line 37 in 0ae1cd8

if cls not in classes or int(difficult) == 1:

to this:
if cls not in classes:

So it will create labels for (difficult) small objects too.

[AlexeyAB]

@pascal1129

For training - use dataset that consists of PascaVOC2007+PascalVOC2012 datasets.

And get labels for PascaVOC2007+PascalVOC2012 by using this script, just change this line:

darknet/scripts/voc_label.py

Line 37 in 0ae1cd8

if cls not in classes or int(difficult) == 1:

to this:
if cls not in classes:

So it will create labels for (difficult) small objects too.

[pascal1129]

@AlexeyAB

You’re right, it works. I put all difficult pictures into trainning set, and then the trained model is much better then the old one.