Replies: 35 comments 11 replies
-
|
When creating this issue, did the following text appear in front of you? |
Beta Was this translation helpful? Give feedback.
-
And I am guessing too, because I don't have an MRE. Maybe Eglot could be taught by the user not to watch certain folder with very large contents, or maybe you need to explain what the structure of your project and use |
Beta Was this translation helpful? Give feedback.
-
|
Is this MRE?
The LSP mode has this solution for the same problem. |
Beta Was this translation helpful? Give feedback.
-
|
By golly man, did you read that link yourself? Yeah I know I did not give all that was asked (because I was confused myself) at first but what from that link in the red box is missing from the screenshot I responded with? mkdir mre
cd mre
python3 -m venv .venv
source .venv/bin/activate # maybe different depending upon the OS
pip3 install pyright numpy numpydoc
emacs test.py -Q -f toggle-debug-on-error -f eglotDoes this not replicate the issue for you?
Does not look like it. I thought Emacs community was friendlier to the first time bug reporters, maybe the Node under the Pyright's hood is rubbing off. |
Beta Was this translation helpful? Give feedback.
-
Not only did I read it, I wrote it. Because I don't want to write it every time I am faced with a similar difficulties.
I'm sorry you find this unfriendly. All the times it takes to you to read the information I wrote is negligible to the effort I put in writing it again and again and guessing your setup. Many other projects take a similar stance. See for example this link where I modelled some of Eglot's bug reporting practice. If this seem unfriendly to you, you should try put yourself in the shoes of a a single unpaid software maintainer of a popular package dealing with many bug reports (and still trying to have a dayjob and a non-programming-related life).
Sigh. I won't even lose time trying it until you specify what |
Beta Was this translation helpful? Give feedback.
-
When you run The issue, as I tried to guess before, is that there are too many files in the directory I created the
I think the description does not prompt for a change of shoes as mine already fit fine ;). I appreciate what you do. I get it, but do you get why,
You would not have lost time if you had not automatically made the assumption that trying would cause you to lose time?
In the words of a great friend, did the following text appear in front of you? :) 
Perhaps sometimes we are all guilty of expecting pedagogical kindness over reading. |
Beta Was this translation helpful? Give feedback.
-
Yes, it usually does. Emacs and Eglot exhibit different behaviour depending on the contents of the file.
Sorry, that was in the beginning of the issue. I'd rather have a compact MRE where I don't have to scroll up and forth. You could also use
Please put in the work for following the isntructions to supply a good bug report. If you don't want to put this time, you can informally report a problem in the discussions instead. This is also stated in the text that you skipped. |
Beta Was this translation helpful? Give feedback.
-
|
I don't see a form that I can fill out and I think being handed a shell script that literally produces the backtrace is a good bug report. I have read your instructions like 3 times and IDK what else do you want. IDK how that clears things up any further.
mkdir mre
cd mre
python3 -m venv .venv
source .venv/bin/activate # maybe different depending upon the OS
pip3 install pyright numpy numpydoc
emacs test.py -Q -f toggle-debug-on-error -f eglotWhat else do you want!? Honestly, forget about it if you still don't have everything you need. |
Beta Was this translation helpful? Give feedback.
-
I think I have everything, thanks.
I'm on Linux and installing all this stuff will have to be done in a container possibly, but I should find time to look at it this week. |
Beta Was this translation helpful? Give feedback.
-
The If you are on Linux, likely Python 3 is installed, if it is, then Pip is installed as well. You don't need anything else. The command, |
Beta Was this translation helpful? Give feedback.
-
|
Can't reproduce on linux. Googling brought me here: https://www.reddit.com/r/emacs/comments/x4p7mg/fix_annoying_max_open_files_for_emacs/ You could also try explaining to |
Beta Was this translation helpful? Give feedback.
-
|
@simurgh9 when this LSP provider was added to elgot it supposed to use https://github.com/microsoft/pyright, please try to install it via npm or yarn: |
Beta Was this translation helpful? Give feedback.
-
|
@illia-danko thanks, but I don't know if that will make a difference. I think the main problem here is that Pyright tries to watch too many files. Specifically, the files under This isn't a real problem on Linux because watching a directory with many files doesn't require many file descriptors. On MacOSX, it does and there aren't usually enough of them. What I meant by "explaining to pyright that we don't want the contents of the full hidden .venv directory watched" is that maybe there is a Pyright command-line option to achieve that. Or maybe a ticket should be raised with the server to request this. Eglot could also add code to stop watching |
Beta Was this translation helpful? Give feedback.
-
|
BTW just to avoid any miscommunication, I don't think I am on MacOS X. If I understand correctly, that is MacOS 10, (X being a roman numeral). I am on MacOS Venture 13.3.1. |
Beta Was this translation helpful? Give feedback.
-
Force of habit., sorry. They dropped the X some years ago. I last used used MacOS in those versions (when it was a half-decent OS). Doesn't make a difference I think |
Beta Was this translation helpful? Give feedback.
-
|
After analysing this and seeing there's little intention to change things over at Pyright, I can recommend the following fixes:
The latter solution seems to be much easier You'll lose some file-watching capabilities in |
Beta Was this translation helpful? Give feedback.
-
|
Also converting this to a discussion which is what it should have been in the first place. |
Beta Was this translation helpful? Give feedback.
-
|
I don't usually use project.el, e.g., if I really just wanna get completions in a single Python script within a virtual environment, saying that it needs to be within a project sounds like a reason I pick Emacs over any IDEs.
What side effects would that have? Does that mean I no longer get features that require watches, e.g., completions? Also, why do we not have the same issue of excessive file watches when the project structure is like this, Instead of, |
Beta Was this translation helpful? Give feedback.
-
If you use Eglot, you use project.el automatically. LSP requires the idea of a project, or rather a workspace. There's no escaping that.
I don't think so but that's something you should test yourself and/or ask the author of the language server.
Possibly because in the absence of a But just guessing. more categorical statements about specific cases need their own MREs (Your current MRE didn't specify Git usage) |
Beta Was this translation helpful? Give feedback.
-
I suppose but it does not need a git repo. So somehow Eglot figures out the workspace root.
Can we not make sure the mechanism that finds the project root without a |
Beta Was this translation helpful? Give feedback.
-
Yes, Unfortunately, to get the You should read |
Beta Was this translation helpful? Give feedback.
-
|
I also ran into this issue with large Python Pandas libraries and Pyright last week. I 100% agree with entire discussion above about bumping OS file limits, excluding files with project configs, etc. However, in troubleshooting I did find a few other things. I noticed that Pyright (maybe happens with other LSP's) watcher flow is:
There is a good discussion in this section of LSP spec talking about some of efficiencies that clients can do. One such thing is to share the same file descriptors across these two watchers. This should make more efficient than taxing system with (thousands?) of duplicate file notifiers. Another thing I noticed, once file limit was hit, the server just failed - making difficult to see what is happening. It would be great to have Lastly, I noticed that when notified of a new directory created, it should also have a file notifier added. I submitted a PR #1228 for these changes. I understand since Eglot is now part of Emacs, this is not the right way. But I wanted to have it handy for this discussion, and if you feel good about it I can submit paperwork, etc. Lastly, not related to PR, but to the discussion about file limits. This was still failing even though my file limits are large. The problem is related to Emacs on Mac (maybe others) that puts limit at |
Beta Was this translation helpful? Give feedback.
-
Nonwithstanding that your optimization proposal for Eglot is interesting, this would seem completely misguided from Pyright. Do you have any idea why Pyright does it like this? Pyright already sends a |
Beta Was this translation helpful? Give feedback.
-
|
I was going to research that more, I don't know specifically yet. But if I were to guess, Pyright (maybe others) probably does this because client/server are initializing, which maybe running builds, etc. that could impact what is being watched. Based on monitoring changes in vscode for pyright, pyright is pretty aggressive on monitoring files. Seems all of the cross-platform IDE's struggle with the file watchers. Like: https://github.com/microsoft/vscode/wiki/File-Watcher-Issues or https://emacs-lsp.github.io/lsp-mode/page/file-watchers/ . The benefit about the patch I submitted, there is a lot less resources spent adding and removing watches - since only adds/removes the changes. One thing I did just come across, if create a directory via It may make sense to exclude things like |
Beta Was this translation helpful? Give feedback.
-
OK, I understand this is just a guess/conjecture. But it doesn't make sense to me. Why not just start watching when all those hypothesized builds are done? As far as I understand it is this hitherto unjustified behaviour which is at the heart of the motivation for your optimization in PR #1228, which -- interesting and presumably correct as it is -- does complicate what is already complicated code considerably. And I'm not sure I like that. Or at least I would like to know what Eglot is getting in return. So it is important to understand why the server behaves the way it does. My more pessimistic/cynical guess is that the server behaves like this because it can. Because no other client that the M$ folks care about uses kqueue in Mac OSX and they think VSCode is the only client and VSCode has options to ignore server-request file watches. |
Beta Was this translation helpful? Give feedback.
-
|
Yes, I don't disagree with anything you say. [smile]
I noticed testing And that could be the other simpler option, to just ignore the id sent by server for Regardless, of direction you decide... I do think there were a couple other trival changes in the patch that should be added in order to make debugging easier by average users (and less noise for you), such as:
I was also going to send notes to some of the maintainers of MacOS builds of emacs that they should adjust |
Beta Was this translation helpful? Give feedback.
-
OK, so it remains true that the only known Language server whose behaviour would motivate your change is still Pyright.
As far as I understand, this would hurt servers who legitimately want to set groups of watchers and turn them on and off selectively.
Why do you want this? This seems to stem from a misunderstanding of what
Catch-and-throw is a usually bad idea because it discards information about the original locus of the error.
This makes sense.
I can't speak to that, but it's indeed a good idea to send this idea to [email protected] (with a reference to this discussion, if you want) |
Beta Was this translation helpful? Give feedback.
-
It could make sense in some situations where learning about this locus is completely useless, but I'm not convinced it is in this case. |
Beta Was this translation helpful? Give feedback.
-
Nope, no misunderstanding; and yes familiar with emacs debugging. Let me start with disclaimer on why I would do this, and say I'm old school and just the way I was raised. [smile]... There several reasons to go ahead and log to messages, in addition to throwing and signaling. 1) For logging purposes, to be able to capture and refer back or share overall message / event logic context. Even think about if automated testing, logging is important - debugging during the automation is not. 2) Not everything needs to be debugged, vs. just seeing the message. For example, take another "end-user facing" emacs package like dired that's been around since 1980's. Open dired (not with fancy ivy, helm, etc. bubble wrap), and navigate to the bottom of some folder with
It's not always 1-size-fits all. Depends on the "Severity of the Error"; "Knowledge of the Error"; "User Experience"; and "Program Logic". In the case of a file notify error, the severity may be getting elevated too high. Because, right now the "out of file descriptors" error happens which stops eglot completely, giving a poor user experience. Vs. having knowledge of the error and handling with program logic: 1) a simple message could be displayed "Hey you're out of file descriptors, see FAQ 123 of the Eglot documentation. File watchers will only partially work until..."; 2) The LSP server notified the capability did not get registered; 3) and keep working. I do agree catching and throwing is usually a bad idea, especially for a user facing app which should handle known errors. The only reason to do this for a user facing functionality would be for debugging with backtrace (if enabled). Anyway, I think we've beat this dead horse enough. |
Beta Was this translation helpful? Give feedback.
-
But the Are you sure you don't just want this? diff --git a/lisp/jsonrpc.el b/lisp/jsonrpc.el
index ccf0f966574..617aeadb977 100644
--- a/lisp/jsonrpc.el
+++ b/lisp/jsonrpc.el
@@ -176,18 +176,22 @@ jsonrpc-connection-receive
(let* ((debug-on-error (and debug-on-error
(not jsonrpc-inhibit-debug-on-error)))
(reply
- (condition-case-unless-debug _ignore
- (condition-case oops
+ (condition-case-unless-debug oops
+ (condition-case inner-oops
`(:result ,(funcall (jsonrpc--request-dispatcher connection)
connection (intern method) params))
(jsonrpc-error
`(:error
(:code
- ,(or (alist-get 'jsonrpc-error-code (cdr oops)) -32603)
+ ,(or (alist-get 'jsonrpc-error-code (cdr inner-oops)) -32603)
:message ,(or (alist-get 'jsonrpc-error-message
- (cdr oops))
+ (cdr inner-oops))
"Internal error")))))
(error
+ (message (concat
+ "I'm going to reply to the server with -32603 "
+ "because debug-on-error is off and got %s")
+ oops)
'(:error (:code -32603 :message "Internal error"))))))
(apply #'jsonrpc--reply connection id reply)))
(;; A remote notification
It doesn't stop Eglot completely, as far as I can tell from the log. What happens is that Eglot catches the error, replies with "Internal Server Error" via JSONRPC (code -32603) and the server over-reacts with a hara-kiri and kills itself. Again, quite extraordinary behaviour from the Pyright server.
I don't know: in Elisp (and this is even more true in Common Lisp), a backtrace is not just a backtrace, it's an interactive debugger, which allows you to inspect frames and select restarts. Elisp is quite more limited than CL (it doesn't have proper restarts and no equivalent Personally, I'd go so far as saying that catch-and-throw is almost always a bad idea. Even in other non-Lisp languages. I seem to remember in JavaScript it's not very harmful because has such a poor error model anyway and the the full backtrace isn't lost. |
Beta Was this translation helpful? Give feedback.
-
OH, that solves what I was(n't) seeing. Yes, that would be best adjustment. Thanks for working through that! |
Beta Was this translation helpful? Give feedback.
-
|
Did we ever decide about what is the best way forward? |
Beta Was this translation helpful? Give feedback.
-
|
I pushed a variation of @jeff-phil 's patch that should at least make pyright's insane "watch-everything twice then unwatch half of it" become just mildly bizarre and reuse watcher resources. So a 50% cut, could lower the high watermark and allow pyright to work for you. But there's still the fact that pyright simply watches too much stuff. And the fact that it kills itself -- presumably needlessly -- when Emacs tells it it can't honor a particularly greedy request. Can't do anything about that. |
Beta Was this translation helpful? Give feedback.
-
|
Hi, does this patch work in emacs 28.2 ? It works on my mac, but does not on my linux install :( |
Beta Was this translation helpful? Give feedback.
-
Yes. You can verify that the Note: The "patch" mentioned here does not fix any underlying OS configuration issues, such as file descriptors being set too low; nor if too many files are included in a project directory structure. It only prevents file watchers from being duplicated. |
Beta Was this translation helpful? Give feedback.
-
|
I understand that adding excludes similar to those seen in lsp-mode goes against the division of responsibilities envisioned for eglot. However, It doesn't take a huge repo to blow out the file-watcher limit. Even after bumping the limits and swapping It's unlikely that we will be able to get pyright to stop being so abusive with the way it watches files, and for many of us moving away from osx is impossible since we don't always control the environments we work in. Maybe it's worth adding the excludes at the expense of some kludge? |
Beta Was this translation helpful? Give feedback.
-
|
Hi @mattmonja- If you are looking for excludes/ignores, pyright offers this in configuration: https://github.com/Microsoft/pyright/blob/main/docs/configuration.md Using that should not cause excluded and ignored files to be registered by pyright to eglot. The original of this was a bit broader, in that 1) MacOS has a way too low setting for DFD_SIZE, which can/should be adjusted when building emacs by setting |
Beta Was this translation helpful? Give feedback.
-
|
Hello @jeff-phil I'm utilizing your patch and bumped |
Beta Was this translation helpful? Give feedback.
-
You're right, and while this is mostly due Pyright's author stubborness of asking the client to watch each and every file in the repo (see discussion here microsoft/pyright#2108 (comment)), Eglot could/shoud be more robust to this. But what kind of excluding mechanism do you envision? An Elisp variable? With what semantics? And how should Eglot respond to the absurd server's request? Should it do a "let's not and say we did" kind of thing. These questions need to be first answered and then a patch proposed. |
Beta Was this translation helpful? Give feedback.
-
I am on MacOS Venture 13.3.1, GNU Emacs 30.0.50 (
brew install emacs-nightly), Eglot 1.14 (found by looking ateglot.el) and finally Pyright 1.1.307 (pip3 install pyright).My issue is similar to this: #768
As soon as I run
eglot, I get,Here is the backtrace:
And these are the event logs:
I tried googling the no file descriptors left error and found this,
Well, but I am not using
lsp-modebut maybe something similar is happening with eglot?Any hint to a resolution is appreciated.
EDIT
Okay I've figured something out, this only happens when I run
eglotfrom within a file that is in the same directory as a virtual environment folder, e.g.,.venv. I am guessing that happens because emacs tries to create a watch for all the files in there. I am not sure what a good fix is though other than moving the.venvaway.Beta Was this translation helpful? Give feedback.
All reactions