Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

load_old_state_dict of DiffusionModelUNet is wrong #8029

Closed
yiheng-wang-nv opened this issue Aug 20, 2024 · 2 comments · Fixed by #8031
Closed

load_old_state_dict of DiffusionModelUNet is wrong #8029

yiheng-wang-nv opened this issue Aug 20, 2024 · 2 comments · Fixed by #8031
Assignees

Comments

@yiheng-wang-nv
Copy link
Contributor

yiheng-wang-nv commented Aug 20, 2024

Describe the bug
some weights are not handled.

To Reproduce
Using monai-generative==0.2.2 and dev branch of MONAI to compare:

from generative.networks.nets import DiffusionModelUNet
from monai.networks.nets.diffusion_model_unet import DiffusionModelUNet as MonaiDiffusionModelUNet
import torch

input_params = {
    "spatial_dims": 2,
    "in_channels": 1,
    "out_channels": 1,
    "num_channels": [32, 64, 128, 256],
    "attention_levels": [False, True, True, True],
    "num_head_channels": [0, 32, 32, 32],
    "num_res_blocks": 2,
}

old_network = DiffusionModelUNet(**input_params)

new_params = input_params.copy()
new_params.pop("num_channels")
new_params["channels"] = input_params["num_channels"]
new_params["include_fc"] = False
new_params["use_combined_linear"] = False

new_network = MonaiDiffusionModelUNet(**new_params)

for k in new_network.state_dict().keys():
    if "to_q.weight" in k:
        print(k)

for k in old_network.state_dict().keys():
    if "to_q.weight" in k:
        print(k)

new network has layers like:

down_blocks.1.attentions.0.attn.to_q.weight
down_blocks.1.attentions.1.attn.to_q.weight

old network has layers like:

down_blocks.1.attentions.0.to_q.weight
down_blocks.1.attentions.1.to_q.weight

However, the load_old_state_dict function does not handle these layers

@yiheng-wang-nv
Copy link
Contributor Author

this bug is also related to bundle update: Project-MONAI/model-zoo#621

@yiheng-wang-nv yiheng-wang-nv self-assigned this Aug 20, 2024
@yiheng-wang-nv
Copy link
Contributor Author

I can handle this issue (already solved locally) cc @KumoLiu @ericspod

rcremese pushed a commit to rcremese/MONAI that referenced this issue Sep 2, 2024
…oject-MONAI#8031)

Fixes Project-MONAI#8029  .

### Description

A few sentences describing the changes proposed in this pull request.

### Types of changes
<!--- Put an `x` in all the boxes that apply, and remove the not
applicable items -->
- [x] Non-breaking change (fix or new feature that would not break
existing functionality).
- [ ] Breaking change (fix or new feature that would cause existing
functionality to change).
- [ ] New tests added to cover the changes.
- [ ] Integration tests passed locally by running `./runtests.sh -f -u
--net --coverage`.
- [ ] Quick tests passed locally by running `./runtests.sh --quick
--unittests --disttests`.
- [ ] In-line docstrings updated.
- [ ] Documentation updated, tested `make html` command in the `docs/`
folder.

---------

Signed-off-by: Yiheng Wang <[email protected]>
Signed-off-by: Yiheng Wang <[email protected]>
Co-authored-by: YunLiu <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant