Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Constrained decoding fix #248

Open
mjpost opened this issue Mar 10, 2016 · 0 comments
Open

Constrained decoding fix #248

mjpost opened this issue Mar 10, 2016 · 0 comments
Assignees

Comments

@mjpost
Copy link
Member

mjpost commented Mar 10, 2016

Constrained decoding works by ensuring that LM state tuples match against the specified target side. This fails when using KenLM, since it maintains its own private state and doesn't expose the n-gram boundary words to the decoder (instead using the generic DP state mechanism). This could be fixed by always creating a dummy LM feature whenever constrained decoding is requested, which will do the state maintenance that is needed. This could maybe be done at the sentence level or could be done via a decoder switch (meaning that constrained decoding would only work when you've explicitly requested it, which might be a good thing).

Another approach that might be easier would be to check edges against the input and block there, instead of checking states. However the state constraint itself already has holes (there are cases where it can permit a hypothesis that doesn't exactly match the output), and I think this would be worse.

@mjpost mjpost self-assigned this Mar 10, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant