How can we do fine Tuning / Transfer Learning for a new language

[joazoa]

Hello,

I'm new to k2, but have prior experience with flashlight / Espnet.

Can somebody point me in the right direction to do transfer learning with k2 ?
What is the best way to delete the last layer (I want to change the number of tokens). and reset the learning rate ? I searched the forums and the source code but without luck.

Thank you!

[mohsen-goodarzi]

Hi,
IMO, you have to change the load_checkpoint() function from icefall.checkpoint in a way that when there is a size mismatch between model weights and checkpoint weights, then it ignores the related key while loading checkpoint.
Something like this:

        dst_state_dict = model.state_dict()
        src_state_dict = checkpoint["model"]
        for key in src_state_dict:
            if src_state_dict[key].shape != dst_state_dict[key].shape:
                src_state_dict[key] = dst_state_dict[key]
        model.load_state_dict(src_state_dict, strict=strict)

[yfyeung]

For fine-tuning, please follow this PR: #944
For adaption, please follow this PR: #1059