Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not detecting language in large text #18

Open
creker opened this issue Jan 15, 2020 · 1 comment
Open

Not detecting language in large text #18

creker opened this issue Jan 15, 2020 · 1 comment

Comments

@creker
Copy link

creker commented Jan 15, 2020

Example
https://play.golang.org/p/qupLXwVQc4m

First example is a large text in English. The library can't produce confident result - confidence is negative. Second example is a couple of sentences from the same text. The result is correct. It doesn't matter in which language the text is. After certain threshold it will always break.

I checked https://github.com/kapsteur/franco that seems to be using the same model and trigrams. It works.

@abadojack
Copy link
Owner

Thanks @creker. I'm looking into it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants