Training problem

[nuitvolgit]

I use tensorflow 2.0.0-rc1

I trained using train.py with a default training image: "girl.png"
After training with FLAGS.mode='fit', FLAGS.transfer='none',
I tested it with the same trained image using detect.py
But I got following warnings and couldn't detect any objects as well.

.......
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).layer_with_weights-0.layer_with_weights-0.kernel
W0919 11:04:29.340978 140067676804928 util.py:144] Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).layer_with_weights-0.layer_with_weights-0.kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).layer_with_weights-0.layer_with_weights-1.gamma
W0919 11:04:29.341034 140067676804928 util.py:144] Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).layer_with_weights-0.layer_with_weights-1.gamma
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).layer_with_weights-0.layer_with_weights-1.beta
W0919 11:04:29.341070 140067676804928 util.py:144] Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).layer_with_weights-0.layer_with_weights-1.beta
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).layer_with_weights-0.layer_with_weights-2.kernel
................... and so on..

It looks like weights are not fully restored for some reason.

If I train with FLAGS.mode = 'eager_tf', I don't get such warning when running detect.py

It seems like saving weight using "model.save_weights" does not provide warning but
using "ModelCheckpoint" does provide warning.

I am not sure.

Any ideas?

[nuitvolgit]

I still get the same warning but the detection works.
It turned out that the reason of detection failure was not related to the warnings.
I tried to train a single class detection but the class probability was not trained since the softmax for a single class always gives 1.
So I changed the score value in the yolo_nms function such that if the class number is one, then score is the same as the confidence.

[mmortazavi]

I am having a quick start using this repo. Just read your comments about single class detection! How did you change in score value in the yolo_nms function? Would you mind sharing your modified yolo_nms function here so we could use as well? Thanks

[nuitvolgit]

I changed the following line in yolo_nms function:

scores = confidence * class_probs

to like this:

if classes > 1:
scores = confidence * class_probs
else:
scores = confidence

This is because the class_prob is not trained for a single class case.
So its value is almost like zero from the start to the end of the training.

[nuitvolgit]

So I couldn't fix the warning but everything works fine so I decided to ignore the warning. Thanks. 2019년 9월 20일 (금) 오전 2:55, nivsmall <[email protected]>님이 작성:

…

I had the same issue before,my issue was that the imported tensorflow was version 1 and not 2.0 (even though the V.2 was installed in the environment) did you verify correct version during run? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#70?email_source=notifications&email_token=ACOS23QILI7HDXYWON5X7NTQKO4JJA5CNFSM4IYFSYHKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7EJYJQ#issuecomment-533240870>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACOS23WDZKCTGEJCZX3S753QKO4JJANCNFSM4IYFSYHA> .

[SpontaneousDuck]

This fix for one class should be pushed up to the main! Maybe i'll get around to a PR

[lazerliu]

My detection is also not work,I train with pascal 16 epochs,and then detect nothing!
I wonder what the correct steps to validate the model I trained.thks

[zzh8829]

@lazerliu see this tutorial https://github.com/zzh8829/yolov3-tf2/blob/master/docs/training_voc.md