Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use padding_idx=None for nn.Embedding() in the decoder model #1206

Open
csukuangfj opened this issue Aug 8, 2023 · 2 comments
Open

Use padding_idx=None for nn.Embedding() in the decoder model #1206

csukuangfj opened this issue Aug 8, 2023 · 2 comments

Comments

@csukuangfj
Copy link
Collaborator

We need to change sherpa/sherpa-onnx/sherpa-ncnn to use [-1, 0] as the initial tokens during decoding instead of [0, 0].
The consequence is that if we need to re-export the model, otherwise it will cause runtime error saying that

IndexError: index out of range in self

If we don't change sherpa/sherpa-onnx/sherpa-ncnn, a user has reported that the WER becomes worse.

@kamirdin
Copy link
Contributor

I don't understand that we train the model use [0,0], why we decode by [-1,0]?

        blank_id = self.decoder.blank_id
        sos_y = add_sos(y, sos_id=blank_id)

        # sos_y_padded: [B, S + 1], start with SOS.
        sos_y_padded = sos_y.pad(mode="constant", padding_value=blank_id)

sos_y_padded = sos_y.pad(mode="constant", padding_value=blank_id)

@csukuangfj
Copy link
Collaborator Author

I don't understand that we train the model use [0,0], why we decode by [-1,0]?

        blank_id = self.decoder.blank_id
        sos_y = add_sos(y, sos_id=blank_id)

        # sos_y_padded: [B, S + 1], start with SOS.
        sos_y_padded = sos_y.pad(mode="constant", padding_value=blank_id)

sos_y_padded = sos_y.pad(mode="constant", padding_value=blank_id)

please think about the input of the conv module in the decoder model.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants