Premium API leads to a HTTP status code 413 for longer texts #49
Assignees
Comments
lamBOOO
added
1-bug 🐛
|
This issue from the original ltex extension is related (valentjn#812). A possible solution might be to split the large document into a sequence of smaller text (or sentence) chunks, since smaller documents do not result in a 413 error. I could help to implement, if that is a good solution. |
|
Splitting the text into smaller chunks is also the recommendation by the LanguageTool support forum: https://forum.languagetool.org/t/premium-cloud-features-in-vscode-extension/7935/4. |
|
Splitting the text into chunks has some downsides. LanguageTool checks for example, if you start your sentences with the same word again and shows a hint. If we check the sentences separately, this won't work anymore, of course. Your sample document has approx. 24000 characters. Can you please check what the actual limit of LanguageTool Premium is? Or in other words: How small the chunks must be? |
|
Yes, these "global checks" might have a quality decrease when splitting the text into chunks. Maybe, we can introduce a setting to control this character limit, s.t. the text splitting is turned off by default for local language servers? 🤔 I personally came from the Grammarly VSCode extension (https://github.com/znck/grammarly), but Grammarly shut down their API... Before that, the extension was running perfectly and Premium subscribers could link their account. But I don't know how they handled the API limits (if any). Regarding the character count, the following file barely works: Details
When enabling the logging to finest, I see that the following was sent to the API:
Does it help? |
|
For the Obsidian Plugin (https://languagetool.org/insights/post/product-obsidian/), the API is limited to 60k chars and exceeding that limit, I get the following clear error: It's strange that the VSCode extensions fails for way below 60k characters 🤔 (we are way closer to the 20k limit for the free API usage according to https://languagetool.org/http-api/). I could live with 60k characters probably when splitting larger LaTeX files into multiple smaller files. |
|
The website editor also seems to have the 60k limit. |
spitzerd
removed
the
2-unconfirmed
|
LTeX+ does not send plain text but a JSON document in order to distinguish between text and markup. If you check https://languagetool.org/http-api/, you can find following example: LTeX+ splits after each character, so the JSON file size is increased tremendously.
This is not an issue if you run LanguageTool locally, but it seems to be an issue when using LT Premium. I will work on a solution, but I cannot promise the delivery date, of course. Feel free to submit a pull request to https://github.com/ltex-plus/ltex-ls-plus Until a fix is available, you can
|
|
Yes, I saw that in the log and checked the character size of the ASCI JSON but it was not around 60k 😄 What’s the advantage of sending the markdown to the server also? To have correct line numbers etc? Because I noticed, that the official LT web editor strips that away before checking. |
|
From which file is the screenshot taken? 🤔 |
|
The screenshot is from following file: You cannot just the markup, as that would mess up the position information that LanguageTool returns. See https://dev.languagetool.org/java-api#checking-text-with-markup |
|
Ahhh okat thanks, I wasn't looking into that other repo. |
|
I just found this pull request valentjn/ltex-ls#228 . After a review, I merged it into https://github.com/ltex-plus/ltex-ls-plus Can you test with the next nightly release of ltex-ls-plus, please? |
|
I tested it with the nightly release and it works! The text and the markdown is "grouped" and the initial non working MWE works now. I'm even able to check my 15 page LaTeX paper (with some Math), although taking 10 secs to receive the request response: |
|
I also noticed in the log that sometimes, fragmentation happens. It's not so clear to me, how it works. See e.g. https://github.com/ltex-plus/ltex-ls-plus/blob/0f5d5ca101ec95052b1e6814548fb5b512dc21c1/src/main/kotlin/org/bsplines/ltexls/server/DocumentChecker.kt#L306 What is the purpose of this fragmentation? |
|
For the following: \documentclass{article}
\begin{document}
Firstt
% LTeX:
Secondd
% LTeX:
Thirdd
\end{document}I get this log: This suggests, that |
|
Nice catch! Inserting the magic command triggers a re-initialization of ltex-ls-plus, which makes totally sense. I will extend the documentation at https://ltex-plus.github.io/ltex-plus/advanced-usage.html in the upcoming days/weeks with this trick. If you want to, you can help to extend the documentation by submitting a pull request to https://github.com/ltex-plus/ltex-plus |
|
Those are the two points for the documentation, nothing else in my opinion. |
spitzerd
added
the
3-fixed
|
This issue is now fixed on If you don't want to wait, you can try out the nightly pre-release tomorrow. Nightly pre-releases are published every morning at around 4am UTC. |
|
@lamBOOO |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the bug
When using the premium API key, i.e. setting
leads to an
413 Content Too Largeresponse.Steps to reproduce
Steps to reproduce the behavior:
Expected behavior
Also larger LaTeX files, should return the spell checking results when using the premium API.
Sample document
Failing document (too larger):
Working document (small enough):
LTeX configuration
Please paste all configuration settings starting with
ltex.from yoursettings.json. You can help us by temporarily removing some irrelevant settings from yoursettings.jsonand see if the bug still occurs."LTeX Language Server" log file
First, reproduce the bug. Then, go to
View→Outputand selectLTeX Language Serverin the drop-down list. Paste this log here:"LTeX Language Client" log file
First, set the
ltex.trace.serversetting in yoursettings.jsonto"verbose". Then, reload the VS Code window and reproduce the bug. Go toView→Outputand selectLTeX Language Clientin the drop-down list. Paste this log here (note: it will contain your checked document):Version information
List here the version information of the relevant software.
Additional context/information
You can add any other context or information about the problem here.
The text was updated successfully, but these errors were encountered: