Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[question] There's something wrong with jedi:install-server #343

Closed
kirk86 opened this issue Oct 11, 2019 · 27 comments
Closed

[question] There's something wrong with jedi:install-server #343

kirk86 opened this issue Oct 11, 2019 · 27 comments

Comments

@kirk86
Copy link

kirk86 commented Oct 11, 2019

I'm using company and company-jedi. What I've noticed is that whenever I run jedi:install-server for some reason it brings a deprecated version of jediepcserver.py file where the deferred error of function signatures is still apparent. I've checked the master branch here and the jediepcserver.py has been corrected not to have those errors but still for some reason which I cannot explain the command jedi:install-server brings an old deprecated version of jediepcserver.py can someone have a look at?

I've also made emacs-company-jedi aware of this issue but I think the problem might lie here since in my understanding jedi:install-server brings or at least should bring the files of this repo but for some reason jediepcserver.py is outdated or deprecated with the deferred function signature errors still there in the file.

Here's my config info:

(use-package comapany
  :ensure t
  :init (global-company-mode t))

(use-package company-jedi
  :ensure t
  :init (add-to-list 'company-backends ' company-jedi))

(use-package python
  :hook ((python-mode . jedi:setup)))

jedi:show-version-info
image

Here's the proof, this is jediepcserver.py just after executing jedi:install-server
image

As you can see the jedi.api.defined_names should have been jedi.api.names instead.

Why is this happening, any ideas?

@immerrr
Copy link
Collaborator

immerrr commented Oct 11, 2019

That is interesting, let me try to reproduce it and get back to you.

In the meanwhile, obligatory question: could you please check that you are using the latest version of jedi-core?

@kirk86
Copy link
Author

kirk86 commented Oct 11, 2019

image

Some other info

pip packages
jedi: 0.15.1
epc: 0.05
parso: 0.5.1

@kirk86
Copy link
Author

kirk86 commented Oct 11, 2019

@immerrr I thinks I know what's happening jediepcserver.py seems to be different between melpa-stable and melpa. Although the one coming from melpa doesn't seem to have the deprecated calls to jedi.api.defined_names but rather to the correct ones jedi.api.names it still gives me deferred error whenever I call jedi:get-in-function-call. On the other hand jediepcserver.py coming from melpa stable has the deprecated calls to jedi.api.defined_names but after changing lines 105, 107, 179 it works fine whenever I call jedi:get-in-function-call. Could you please have a look and fix that in both melpa and melpa-stable?

@immerrr
Copy link
Collaborator

immerrr commented Oct 11, 2019

Yep, that's true, the stable distribution hasn't been updated in a while, so that could be the reason.

@kirk86
Copy link
Author

kirk86 commented Oct 11, 2019

Sill though the melpa one gives me deferred error, the only way I could make ti work was to modify the melpa-stable one in the lines I mentioned above

@immerrr
Copy link
Collaborator

immerrr commented Oct 11, 2019

That is interesting... jedi-core package should contain a version of jediepcserver.py, and the one in melpa should be much fresher

@immerrr
Copy link
Collaborator

immerrr commented Oct 11, 2019

20191007.653 version on melpa should have this fixed, let me check...

@kirk86
Copy link
Author

kirk86 commented Oct 11, 2019

I confirm that 20191007.653 does solve the problem but the whole melpa-stabe thing is becoming like a joke. I don't understand the point of calling it stable and confusing the users thinking they'll have a more stable experience there than in melpa when actually in contains outdated buggy code.

@kirk86 kirk86 closed this as completed Oct 11, 2019
@kirk86
Copy link
Author

kirk86 commented Oct 11, 2019

@immerrr Any ideas if this could be caused due to jedi backend.

@immerrr
Copy link
Collaborator

immerrr commented Oct 11, 2019

Not sure.. FWIW jedi (the python package) has been slow for me for a couple releases now, it is very noticeable for example in IPython input prompt.

@immerrr
Copy link
Collaborator

immerrr commented Oct 11, 2019

I confirm that 20191007.653 does solve the problem but the whole melpa-stabe thing is becoming like a joke

I understand your frustration, "bit rot" has not been gentle to jedi-core's stable release, and neither have I. I'll see if I can bump the stable release shortly. (#336 is a related issue)

@kirk86
Copy link
Author

kirk86 commented Oct 11, 2019

jedi (the python package) has been slow for me for a couple releases now

Maybe worth reporting as bug to jedi?
What other alternatives are there especially for python?

@immerrr
Copy link
Collaborator

immerrr commented Oct 11, 2019

You could try installing typeshed, which is a set of stubs for popular libraries out there. Type stubs reduce the amount of work Jedi has to do when deducing the correct types.

But in general, I don't see an open-source alternative in Python world. rope couple of years ago was already quite dated and not enjoying too much love from the community. PyCharm has a decent completion engine, but it comes with the PyCharm itself.

Oh, for Emacs itself, you could try youcompleteme, there's a ycmd server that provides completions for multiple languages, including Python via Jedi. It has a completely different binding, so could help if the problem is in emacs-jedi itself.

@immerrr
Copy link
Collaborator

immerrr commented Oct 11, 2019

Maybe worth reporting as bug to jedi?

Could work. It would help if you had a clear-cut completion example that did not require emacs or emacs-jedi, there's an example in Jedi docs on how to invoke completion from Python code, that would help narrowing down the scope of the sluggishness:

https://jedi.readthedocs.io/en/latest/docs/api.html

>>> import jedi
>>> source = '''
... import json
... json.lo'''
>>> script = jedi.Script(source, 3, len('json.lo'), 'example.py')
>>> script
<Script: 'example.py' ...>
>>> completions = script.completions()

@kirk86
Copy link
Author

kirk86 commented Oct 11, 2019

Thanks for the ycmd pointer! Have you tried it yet?
Regarding the jedi example still not sure what is that supposed to do other just showcasing jedi. There's no (as far as I understand) realistic benchmarking scenario to show how fast or slow jedi is.

PyCharm has a decent completion engine, but it comes with the PyCharm itself.

What are they using? Did they build their own or using something open-source?

@immerrr
Copy link
Collaborator

immerrr commented Oct 11, 2019

Thanks for the ycmd pointer! Have you tried it yet?

Nop, I'm on a "dogfooding" diet :)

Regarding the jedi example still not sure what is that supposed to do other just showcasing jedi. There's no (as far as I understand) realistic benchmarking scenario to show how fast or slow jedi is.

I mean, if you have a specific example that takes very long to complete, you could prepare a script for it: take source = open(path_to_source_file).read(), figure out the line and column numbers, and then run it in the terminal measuring the time spent. If it is still too slow, that's a Jedi issue.

If not, the issue could be with the jediepcserver. You could try enabling logging for the server, btw, if you haven't done it already. I have the following config:

jedi:server-args `("--log-level=DEBUG"
                      ,(format "--log=%s" (expand-file-name "~/jedi.log"))
                      "--log-rotate-max-size=1000000"
                      "--log-rotate-max-count=3"
                      "--log-traceback")

The log, although quite verbose, contains timestamps and could lead to some insights as to what is causing the slowness.

@kirk86
Copy link
Author

kirk86 commented Oct 11, 2019

Thanks for this tip, it's actually very useful.

Now even after running jedi:install-server whenever I close the file and reopen it I get the following message

image

@kirk86
Copy link
Author

kirk86 commented Oct 11, 2019

@immerrr Is this helpful?

Debugger entered--Lisp error: (wrong-type-argument epc:manager nil)
  signal(wrong-type-argument (epc:manager nil))
  jedi:epc--start-epc("/Users/jm/.emacs.d/.python-environments/default/bin/jediepcserver" ("--virtual-env" "/Users/jm/miniconda3/envs/tf" "--log-level=DEBUG" (\, (format "--log=%s" (expand-file-name "~/.emacs.d/jedi.log"))) "--log-rotate-max-size=1000000" "--log-rotate-max-count=3" "--log-traceback"))
  jedi:server-pool--start(("/Users/jm/.emacs.d/.python-environments/default/bin/jediepcserver" "--virtual-env" "/Users/jm/miniconda3/envs/tf" "--log-level=DEBUG" (\, (format "--log=%s" (expand-file-name "~/.emacs.d/jedi.log"))) "--log-rotate-max-size=1000000" "--log-rotate-max-count=3" "--log-traceback"))
  jedi:start-server()
  jedi:get-epc()
  jedi:call-deferred(complete)
  company-jedi-candidates(#f(compiled-function (candidates) #<bytecode 0x405934f1>))
  company--fetch-candidates("")
  company-calculate-candidates("" nil)
  company--begin-new()
  company--perform()
  company-auto-begin()
  company-idle-begin(#<buffer testpyth.py> #<window 1 on testpyth.py> 24 69)
  apply(company-idle-begin (#<buffer testpyth.py> #<window 1 on testpyth.py> 24 69))
  timer-event-handler([t 23968 47558 100681 nil company-idle-begin (#<buffer testpyth.py> #<window 1 on testpyth.py> 24 69) nil 0])

@immerrr
Copy link
Collaborator

immerrr commented Oct 11, 2019

Looks like a problem in the jedi:server-args variable, it has to be a flat list of strings and nothing else, for example mine is C-h v jedi:server-args:

("--log-level=DEBUG" "--log=/home/immerrr/jedi.log" "--log-rotate-max-size=1000000" "--log-rotate-max-count=3" "--log-traceback")

The string I sent you contained a comma which is a special syntax for evaluation inside backquotes and looks like it did not expand well.

@kirk86
Copy link
Author

kirk86 commented Oct 11, 2019

Can I set it as quoted (setq jedi:server-args '(whatever))?

@immerrr
Copy link
Collaborator

immerrr commented Oct 11, 2019

Yes, a quoted list of strings '("foo" "bar" "baz") should work.

@kirk86
Copy link
Author

kirk86 commented Oct 11, 2019

@immerrr This is what I have in my config
image

I remove elpa, .python_environments and all *.elc

fire up emacs, installs packages

I use pyvenv and activate the environment which contains python jedi, epc, parso then M-x jedi:i-s

The C-c C-f testpython.py and this is what I get

Error (jedi): 
================================
Failed to start Jedi EPC server.
================================

*** EPC Error ***
Server may raise an error. Use "M-x epc:pop-to-last-server-process-buffer RET" to see full traceback:
usage: jediepcserver [-h] [--address ADDRESS] [--port PORT]
                     [--port-file PORT_FILE] [--sys-path SYS_PATH]
                     [--sys-path-append SYS_PATH_APPEND]
                     [--virtual-env VIRTUAL_ENV] [--log LOG]
                     [--log-level {CRITICAL,ERROR,WARN,INFO,DEBUG}]
                     [--log-rotate-max-size LOG_ROTATE_MAX_SIZE]
                     [--log-rotate-max-count LOG_ROTATE_MAX_COUNT]
                     [--log-traceback] [--pdb] [--ipdb]
jediepcserver: error: unrecognized arguments: /Users/jm/.emacs.d/jedi.log


*** EPC Server Output (last 10 lines) ***
usage: jediepcserver [-h] [--address ADDRESS] [--port PORT]
                     [--port-file PORT_FILE] [--sys-path SYS_PATH]
                     [--sys-path-append SYS_PATH_APPEND]
                     [--virtual-env VIRTUAL_ENV] [--log LOG]
                     [--log-level {CRITICAL,ERROR,WARN,INFO,DEBUG}]
                     [--log-rotate-max-size LOG_ROTATE_MAX_SIZE]
                     [--log-rotate-max-count LOG_ROTATE_MAX_COUNT]
                     [--log-traceback] [--pdb] [--ipdb]
jediepcserver: error: unrecognized arguments: /Users/jm/.emacs.d/jedi.log


*** EPC Server Config ***
Server arguments: ("/Users/jm/.emacs.d/.python-environments/default/bin/jediepcserver" "--virtual-env" "/Users/jm/miniconda3/envs/tf" "--log-level=DEBUG" "/Users/jm/.emacs.d/jedi.log" "--log-traceback")
Actual command: /Users/jm/.emacs.d/.python-environments/default/bin/jediepcserver
VIRTUAL_ENV envvar: "/Users/jm/miniconda3/envs/tf"

*** jedi-mode is disabled in #<buffer testpyth.py> ***
Fix the problem and re-enable it.

*** You may need to run "M-x jedi:install-server". ***
This could solve the problem especially if you haven't run the command yet
since Jedi.el installation or update and if the server complains about
Python module imports.

Now what's wrong with this, I don't know?

@immerrr
Copy link
Collaborator

immerrr commented Oct 11, 2019

In your server arguments

("/Users/jm/.emacs.d/.python-environments/default/bin/jediepcserver" "--virtual-env" "/Users/jm/miniconda3/envs/tf" "--log-level=DEBUG" "/Users/jm/.emacs.d/jedi.log" "--log-traceback")

instead of

"/Users/jm/.emacs.d/jedi.log"

it should be

"--log=/Users/jm/.emacs.d/jedi.log"

which is what jediepcserver is trying to tell by saying

error: unrecognized arguments: /Users/jm/.emacs.d/jedi.log

@kirk86
Copy link
Author

kirk86 commented Oct 11, 2019

Oh crap I missed that, I was reading on half of the buffer on my 13' screen.

@kirk86
Copy link
Author

kirk86 commented Oct 11, 2019

@immerrr I see a bunch of errors in the logs. I'm attaching the file here for closer inspection. There must be something going wrong here.

jedi.log

@immerrr
Copy link
Collaborator

immerrr commented Oct 11, 2019

Those are scary stacktraces, but they are mostly related to the known assertion errors raised while fetching docstrings:

2019-10-11 19:03:25,780:WARNING:jediepcserver:Cannot get docstring for completion <Completion: root_factors>

Note that they are marked as WARNING, not ERROR, which indicates that the expected function was performed at least to some extent (without docstrings).

@immerrr
Copy link
Collaborator

immerrr commented Oct 13, 2019

In the meantime you can get rid of the especially annoying timeouts by increasing company-async-timeout (the default is 2 seconds which is barely enough to generate the list of completions for numpy library, and that is with an SSD containing all the necessary jedi cache). I have the following in my python-mode hook:

(setq-local company-async-timeout 10)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants