Error when convert custom model

[stereomatchingkiss]

Thanks, I train the yolov5 model on kaggle(v4.0), copy out the best.pt, use the yolov5 repo to save the state_dict of the model, else the torch.load will throw error 'ModuleNotFoundError: No module named 'models''

import torch

from models.experimental import attempt_load

if __name__ == '__main__':
    model = attempt_load('best.pt', map_location='cpu')  # load FP32 model
    torch.save(model.state_dict(), 'bn.pt')

After that, convert the custom model by yolov5-rt-stack

import torch
from yolort.utils import update_module_state_from_ultralytics

# Update module state from ultralytics
model = update_module_state_from_ultralytics(arch='yolov5s', version='v4.0', custom_path_or_model = torch.load('./best.pt'), num_classes = 1)
# Save updated module
torch.save(model.state_dict(), 'yolov5s_updated.pt')

This time got error message

model = update_module_state_from_ultralytics(arch='yolov5s', version='v4.0', custom_path_or_model = torch.load('./bn.pt'), num_classes = 1)
 File "C:\Users\yyyy\programs\Qt\3rdLibs\pytorch_projects\yolov5-rt-stack\yolort\utils\update_module_state.py", line 61, in update_module_state_from_ultralytics
   model = torch.hub.load(f'ultralytics/yolov5:{version}', 'custom', path_or_model=custom_path_or_model)
 File "C:\Users\yyyy\Anaconda3\envs\ppyolo\lib\site-packages\torch\hub.py", line 370, in load
   model = _load_local(repo_or_dir, model, *args, **kwargs)
 File "C:\Users\yyyy\Anaconda3\envs\ppyolo\lib\site-packages\torch\hub.py", line 399, in _load_local
   model = entry(*args, **kwargs)
 File "C:\Users\yyyy/.cache\torch\hub\ultralytics_yolov5_v4.0\hubconf.py", line 123, in custom
   model = model['model']  # load model
KeyError: 'model'

Any clues how to solve this?Thanks

[zhiqwang]

Hi @stereomatchingkiss , I guess this is a restriction of loading Ultralytics's trained model. The way loading custom model have encountered a problem now.

Maybe you could move the conversion scripts to root path of "ultralytics/yolov5" to make sure the trained model is loaded correctly. Now yolort is a independent package, you can install yolort first following the instructions here.

[stereomatchingkiss]

Maybe you could move the conversion scripts to root path of "ultralytics/yolov5" to make sure the trained model is loaded correctly.

Thanks, done already, I can load the custom model by following codes of yolov5

import torch

from models.experimental import attempt_load

if __name__ == '__main__':
    model = attempt_load('best.pt', map_location='cpu')  # load FP32 model
    torch.save(model.state_dict(), 'bn.pt')

But I cannot convert the model by yolort

import torch
from yolort.utils import update_module_state_from_ultralytics

# Update module state from ultralytics
model = update_module_state_from_ultralytics(arch='yolov5s', version='v4.0', custom_path_or_model = torch.load('./best.pt'), num_classes = 1)
# Save updated module
torch.save(model.state_dict(), 'yolov5s_updated.pt')

[zhiqwang]

Got it! Try the following first, suppose that you've loaded the trained model with ultralytics as model_ultralytics,

from yolort.utils.update_module_state import ModuleStateUpdate

arch = 'yolov5_darknet_pan_s_r40'  # set manually the model structure
num_classes = 1  # set manually the number of classes 
set_fp16 = True

module_state_updater = ModuleStateUpdate(
    arch=arch,
    num_classes=num_classes,
)

module_state_updater.updating(model_ultralytics)

if set_fp16:
    module_state_updater.model.half()

model = module_state_updater.model

# Save updated module
torch.save(model.state_dict(), 'yolov5s_updated.pt')

And I will add more unit-test/instructions for this issue (also welcome for PR if you have interests).

[stereomatchingkiss]

Thanks for your helps, I guess you forgot to load the model

import torch
from yolort.utils import update_module_state_from_ultralytics
from yolort.utils.update_module_state import ModuleStateUpdate

arch = 'yolov5_darknet_pan_s_r40'
set_fp16 = True

module_state_updater = ModuleStateUpdate(
    arch=arch,
    num_classes=1,
)

model = torch.load('bn.pt') #forgot to load model?
module_state_updater.updating(model)

if set_fp16:
    module_state_updater.model.half()

model = module_state_updater.model

# Save updated module
torch.save(model.state_dict(), 'yolov5s_updated.pt')

You can download the models from google drive.

best.pt is the model before save as state_dict
bn.pt is the model save as state_dict

PS : This solution use fp16, but my card(gtx1060) do not support fp16

[zhiqwang]

I guess you forgot to load the model.

Good catch! And I've fixed my the above scripts. Would you mind providing me with an image for testing?

PS : This solution use fp16, but my card(gtx1060) do not support fp16

You can set set_fp16 = False.

[stereomatchingkiss]

Good catch! And I've fixed my the above scripts. Would you mind providing me with an image for testing?

Thanks, image do you mean weights? I put them to google drive already.

[stereomatchingkiss]

Forgot to mention, this script has new error(try set_fp16 as False/True)

Traceback (most recent call last):
  File "convert_v5_models.py", line 22, in <module>
    module_state_updater.updating(model)
  File "C:\Users\yyyy\programs\Qt\3rdLibs\pytorch_projects\yolov5-rt-stack\yolort\utils\update_module_state.py", line 104, in updating
    state_dict = obtain_module_sequential(state_dict)
  File "C:\Users\yyyy\programs\Qt\3rdLibs\pytorch_projects\yolov5-rt-stack\yolort\utils\update_module_state.py", line 167, in obtain_module_sequential
    return obtain_module_sequential(state_dict.model)
AttributeError: 'collections.OrderedDict' object has no attribute 'model'

[zhiqwang]

image do you mean weights? I put them to google drive already.
this script has new error(try set_fp16 as False/True)

Thanks. I've downloaded the trained model weights, and it would be better if you could provide me a test picture, and I will try to debug this issue more carefully later!

[stereomatchingkiss]

image do you mean weights? I put them to google drive already.
this script has new error(try set_fp16 as False/True)

Thanks. I've downloaded the trained model weights, and it would be better if you could provide me a test picture, and I will try to debug this issue more carefully later!

I would like to, but I am using windows10 + anaconda, do requirements.txt works?Export by following

conda list -e > req.txt

link of the file.

[zhiqwang]

I would like to, but I am using windows10 + anaconda, do requirements.txt works? Export by following

It should be works as long as you could load the Ultralytics's trained model successfully.

Currently only the torch and torchvision are restricted, the minimal version are 1.7.1 and 0.8.2 respectively.

EDIT: Sorry I modified the message you sent, and I have reverted it.

[stereomatchingkiss]

I would like to, but I am using windows10 + anaconda, do requirements.txt works? Export by following

It should be works as long as you could load the Ultralytics's trained model successfully.

Thanks, yes, I can load the Ultralytics's trained model successfully(v4.0).

EDIT: Sorry I modified the message you sent, and I have reverted it.

No problem, thanks for your helps, this project save us lots of pain when export the pre-trained model.

[stereomatchingkiss]

Currently only the torch and torchvision are restricted, the minimal version are 1.7.1 and 0.8.2 respectively.

Yes, the version I am using is 1.7.1 and 0.8.2

[zhiqwang]

Hi @stereomatchingkiss , I use the following scripts and it works for me. (And I'm testing on the master branch on ultralytics/yolov5.)

from yolort.utils.update_module_state import ModuleStateUpdate

model = torch.hub.load(
    'ultralytics/yolov5',
    'custom',
    path_or_model='best.pt',  # I download this from Google drive provided by you.
)

# Setup the module state updater
module_state_updater = ModuleStateUpdate(arch='yolov5_darknet_pan_s_r40', num_classes=1)
# Updating module state
module_state_updater.updating(model)
# This is the updated model we want
module_updated = module_state_updater.model

# Save updated module state
torch.save(module_updated.state_dict(), 'yolov5s_updated.pt')

And I can load the converted model weights as below

from yolort.models import yolov5s

model = yolov5s(pretrained=False, score_thresh=0.25, num_classes=1)
ckpt = torch.load('yolov5s_updated.pt', map_location='cpu')
model.model.load_state_dict(ckpt)

model.eval()
model = model.to(device)

But now I didn't know which datasets you are using, if you can provided a picture from the datasets, it would be more helpful.

[stereomatchingkiss]

But now I didn't know which datasets you are using, if you can provided a picture from the datasets, it would be more helpful.

Thanks, it works, it is a model for smoke/fire detection, only train with 1 epoch, may not help much.
You can try with this dataset--kaggle

Do it able to support yolov5m, yolov5l, yolov5x?

[stereomatchingkiss]

if you can provided a picture from the datasets, it would be more helpful.

I though the image mean "create-image-of-currently-installed-ubuntu-on-my-laptop", my bad, thanks for your helps again.

[zhiqwang]

Do it able to support yolov5m, yolov5l, yolov5x?

It should work for yolov5m and yolov5l, but it doesn't support for yolov5x now, but I think that it could work with minor fix (I think yolov5x is somewhat big). PRs are welcome here!

I though the image mean "create-image-of-currently-installed-ubuntu-on-my-laptop", my bad, thanks for your helps again.

Haha, Never mind here. And thanks for your feedback!

[stereomatchingkiss]

It should work for yolov5m and yolov5l, but it doesn't support for yolov5x now, but I think that it could work with minor fix (I think yolov5x is somewhat big). PRs are welcome here!

Thanks, fine with s,m,l.
Will open pull request if I found anything I could help.

[stereomatchingkiss]

The outputs are great, although not the same as yolov5, maybe some pre-processing/post-processing steps are different.
Looks like you forgot to convert color space from bgr to rgb in your "inference-pytorch-export-libtorch.ipynb"?
After convert it to rgb I think the results become better, can detect more smoke/fire, almost no false alarm.

Edit : Export to torch script, results seems good, model(img) do not work, change to model.predict(img) work

[zhiqwang]

The outputs are great, although not the same as yolov5, maybe some pre-processing/post-processing steps are different.

That's a great catch! I think it is caused by the different pre-processing operations. I've verified the the post-processing stages before, it can get the same results as ultralytics/yolov5 (when w/o TTA predict). And I've uploaded a notebook in order to verify the model inference and post-processing stages, this one is a bit outdated now, I haven’t had time to update it. And I am plan to add a dataloader in the predict pipeline to make yolort detect the same results as ultralytics.

Looks like you forgot to convert color space from bgr to rgb in your "inference-pytorch-export-libtorch.ipynb"?

In my impression, ultralytics uses the BGR channel as default, but I am not very sure and need a double check. And it seems that the default image dataloader are using the RGB channel, if you input the image path to model, and use model.predict('image_path') to detect a image, it will be wrong, here also needs further verification.

BTW, all PRs are welcome here.

EDIT: Hi @stereomatchingkiss I post a new issue for this problem, let's move discussion there.