The convert.py may not fit the rwkv-4-world-169m.

[kip234]

I have tried to convert the weight file which i download from https://huggingface.co/RWKV/rwkv-4-world-169m with convert.py. I found the following problems.

• All of the keys of weight file begin with 'rwkv.' compared to convert.py.
• The .att. in convert.py should be replaced with .attention..
• The value of rwkv.blocks.0.attention.time_decay is a 1d tensor. The headsnume and headsize in convert.py can't unpacke from w[f"rwkv.blocks.0.attention.time_decay"].shape

I don't know how to fix the third problem.

[harrisonvanderbyl]

Hi @kip234 I am glad to tell you that the reason you cannot convert the v4 world model is due to the fact it has been deprecated in favor of the v5 world model (Also known as the Eagle models)!
You can find the checkpoints here: https://huggingface.co/BlinkDL/rwkv-5-world/tree/main