Jedi freezes ST and causes very high RAM usage. #271

deathaxe · 2018-08-25T08:56:04Z

Description

An python file of a Sublime Text package is open.
Enter from collections import S into the first line.

ST suddenly freezes. After a while the console shows some output error. The CPU usage of the plugin_host.exe increases to 100% for one core. The RAM usage increases up to 1.2GB.

I was faced to that issue several times. Sometimes RAM usage even increases and stays at 2.5GB!

Console Output

Traceback (most recent call last):
  File "C:\Apps\Sublime Text 3\sublime_plugin.py", line 685, in on_query_completions
    res = vel.on_query_completions(prefix, locations)
  File "C:\Apps\Sublime Text 3\Data\Packages\Jedi - Python autocompletion\sublime_jedi\completion.py", line 154, in on_query_completions
    location=locations[0]
  File "C:\Apps\Sublime Text 3\Data\Packages\Jedi - Python autocompletion\sublime_jedi\daemon.py", line 83, in ask_daemon_with_timeout
    return request.result(timeout=timeout)
  File "./python3.3/concurrent/futures/_base.py", line 403, in result
concurrent.futures._base.TimeoutError

TaskManager

Infos

Win10 Pro 1803 x64
ST 3176 x64
Jedi v0.13.4
Running python 3.7
jedi-0.12.1
parso-0.3.1

'PYTHONPATH': ';C:\\Apps\\Sublime Text 3\\Data\\Lib\\python3.3;C:\\Apps\\Sublime Text 3\\Data\\Packages\\wcwidth\\all;C:\\Apps\\Sublime Text 3\\Data\\Packages\\sublime_lib\\st3;C:\\Apps\\Sublime Text 3\\Data\\Packages\\tabulate\\all;C:\\Apps\\Sublime Text 3\\Data\\Packages\\PYTHON~1\\st3;C:\\Apps\\Sublime Text 3\\Data\\Packages\\pyyaml\\st3;C:\\Apps\\Sublime Text 3\\Data\\Packages\\dateutil\\all;C:\\Apps\\Sublime Text 3;C:\\Apps\\Sublime Text 3\\Data\\Packages\\pygments\\all;C:\\Apps\\Sublime Text 3\\Data\\Packages\\jsonschema\\all;C:\\Apps\\Sublime Text 3\\python3.3.zip;C:\\Apps\\Sublime Text 3\\Data\\Packages\\pyte\\all;C:\\Apps\\Sublime Text 3\\Data\\Packages\\python-jinja2\\all;C:\\Apps\\Sublime Text 3\\Data\\Packages\\MARKUP~1\\all;C:\\Apps\\Sublime Text 3\\Data\\Packages\\backrefs\\st3;C:\\Apps\\Sublime Text 3\\Data\\Packages\\mdpopups\\st3;C:\\Apps\\Sublime Text 3\\Data\\Packages\\lxml\\st3_windows_x64;C:\\Apps\\Sublime Text 3\\Data\\Packages\\Jedi - Python autocompletion\\dependencies;C:\\Apps\\Sublime Text 3\\Data\\Packages;C:\\Apps\\Sublime Text 3\\Data\\Packages\\pymdownx\\st3;C:\\Apps\\Sublime Text 3\\Data\\Packages\\ruamel-yaml\\st3;C:\\Apps\\Sublime Text 3\\Data\\Packages\\coverage\\st3_windows_x64;C:\\Apps\\Sublime Text 3\\Data\\Packages\\pathtools\\all;C:\\Apps\\Sublime Text 3\\Data\\Packages\\pywinpty\\st3_windows_x64;C:\\Apps\\Sublime Text 3\\Data\\Packages\\regex\\st3_windows_x64;C:\\Apps\\Sublime Text 3\\Data\\Packages\\yaml_macros_engine\\st3;C:\\Apps\\Sublime Text 3\\Data\\Packages\\watchdog\\all',

The text was updated successfully, but these errors were encountered:

kylebebak · 2018-08-28T02:59:03Z

I can confirm that some recent release of Jedi caused a big hit to performance. ST now hangs when trying to populate the autocomplete list, often for as long as a second.

I've been using this plugin for about 5 years and this has never been an issue before.

OSX, ST 3176, Jedi 0.13.4.

srusskih · 2018-08-28T13:22:21Z

@deathaxe Sorry, but I can reproduce the issue with this steps.

@deathaxe @kylebebak Sorry for asking, but, could You please make a "fresh installation" of ST3 (no plugins) and install ONLY Jedi to make sure this is Jedi issue ?

Thanks a lot!

deathaxe · 2018-08-29T21:15:00Z

The issue is 100% reproducible with my setup.

Guess it's somehow related with the provided PYTHONPATH and the dependencies being passed to Jedi.
I use the following little script, which adds the ST's sys.path to the environment. This enables Jedi to see all dependencies automatically. I don't need to care about Package Control installing new ones.

import os
import sys


def update_path_list(var_name, path_list):
    paths = os.environ.get(var_name, "").split(os.pathsep)
    paths += path_list
    os.environ[var_name] = os.pathsep.join(paths)


# add ST's python search paths to the local environment
update_path_list("PYTHONPATH", sys.path)

Reading the console error message carefully makes me wonder about the line containing the python3.3. Jedi runs a python deamon, which runs on the external interpreter (3.7) here. Maybe one of the sys.path entries conflicts with the standalone python 3.7?

Maybe I even got the culprit - not sure. ST's sys.path includes python3.3.zip. Looks like the python 3.7 tries to use it rather than its own stdlib and crashes in the concurrent package.

I disabled my script and put all sys.path entries except the python3.3.zip to jedi's "python_package_paths". Seems to work.

kylebebak · 2018-08-30T00:32:58Z

@srusskih

I haven't tried with a fresh installation, but let me explain what I did.

I removed SublimeJEDI, started typing some characters into a function call, and the autocomplete list appeared instantaneously.

I then reinstalled SublimeJEDI, started typing the same characters into the same function call, and Sublime Text hangs for nearly a second before showing the autocomplete list.

In my opinion, this is convincing evidence that SublimeJEDI is causing the slowdown. I've isolated the variable that causes the slowdown. The variable is whether or not SublimeJEDI is installed. This didn't start happening until a couple of weeks ago, maybe less. What do you think?

srusskih · 2018-08-30T07:58:16Z

@kylebebak first completion is slow - yeap, that's ok. next completion should be much faster.

But, I still can't reproduce editor slowdown to fix the issue. That's why I'm asking to experiment.

srusskih · 2018-08-30T07:59:39Z

@deathaxe thanks. seems like parso (or not) tried to parse to much.

kylebebak · 2018-09-06T15:01:44Z

@srusskih

I made a small video to show the slowdown, where I type "url" as an argument into a function 4 separate times.

Each time I'm typing "u-r-l" as quickly as I can, but ST hangs visibly for more than half a second after "u" appears but before "r" appears.

I've been using Jedi for years and I've never seen slowdown like this. I think this demonstrates that this is not just a problem with slowdown on the first completion. Each time it's just as slow. This project has only 2 modules, with a total of <300 lines of code.

@deathaxe
Could you show me your "python_package_paths" setting for Jedi, so I can see if this gets rid of the slowdown on my machine?

deathaxe · 2018-09-06T16:39:58Z

Just added a webbrowser.get(...).open() line somewhere in my code. The first time I enter url, it takes quite a long time - 1 to 3 seconds. But after that the delays seem constant at something like 0.3 to 0.6s.

The culprit for high RAM was the second line in my case.

// These settings override Default settings for the package
{
    "python_package_paths": [
        "C:\\Apps\\Sublime Text 3",
        // "C:\\Apps\\Sublime Text 3\\python3.3.zip",
        "C:\\Apps\\Sublime Text 3\\Data\\Lib\\python3.3",
        "C:\\Apps\\Sublime Text 3\\Data\\Packages",
        "C:\\Apps\\Sublime Text 3\\Data\\Packages\\pygments\\all",
        "C:\\Apps\\Sublime Text 3\\Data\\Packages\\ruamel-yaml\\st3",
        "C:\\Apps\\Sublime Text 3\\Data\\Packages\\sublime_lib\\st3",
        "C:\\Apps\\Sublime Text 3\\Data\\Packages\\tabulate\\all",
        "C:\\Apps\\Sublime Text 3\\Data\\Packages\\lxml\\st3_windows_x64",
        "C:\\Apps\\Sublime Text 3\\Data\\Packages\\pathtools\\all",
        "C:\\Apps\\Sublime Text 3\\Data\\Packages\\wcwidth\\all",
        "C:\\Apps\\Sublime Text 3\\Data\\Packages\\coverage\\st3_windows_x64",
        "C:\\Apps\\Sublime Text 3\\Data\\Packages\\watchdog\\all",
        "C:\\Apps\\Sublime Text 3\\Data\\Packages\\pywinpty\\st3_windows_x64",
        "C:\\Apps\\Sublime Text 3\\Data\\Packages\\backrefs\\st3",
        "C:\\Apps\\Sublime Text 3\\Data\\Packages\\dateutil\\all",
        "C:\\Apps\\Sublime Text 3\\Data\\Packages\\MARKUP~1\\all",
        "C:\\Apps\\Sublime Text 3\\Data\\Packages\\pymdownx\\st3",
        "C:\\Apps\\Sublime Text 3\\Data\\Packages\\pyte\\all",
        "C:\\Apps\\Sublime Text 3\\Data\\Packages\\PYTHON~1\\st3",
        "C:\\Apps\\Sublime Text 3\\Data\\Packages\\pyyaml\\st3",
        "C:\\Apps\\Sublime Text 3\\Data\\Packages\\regex\\st3_windows_x64",
        "C:\\Apps\\Sublime Text 3\\Data\\Packages\\yaml_macros_engine\\st3",
        "C:\\Apps\\Sublime Text 3\\Data\\Packages\\python-jinja2\\all",
        "C:\\Apps\\Sublime Text 3\\Data\\Packages\\jsonschema\\all",
        "C:\\Apps\\Sublime Text 3\\Data\\Packages\\mdpopups\\st3",
        "C:\\Apps\\Sublime Text 3\\Data\\Packages\\Jedi - Python autocompletion\\dependencies",
    ],
}

kylebebak · 2018-09-06T17:48:36Z

Hmm...

My settings for Jedi are just the default settings. In particular, python_package_paths is an empty list. So something else is causing the slowdown for me. At least now we have a reproducible case.

Just typing and retyping various parts of the following causes slowdown that doesn't get better with repeated attempts. But maybe this is to be expected. @srusskih Any ideas?

import webbrowser
webbrowser.get('abc').open(url)

mew1033 · 2018-09-20T18:03:09Z

I'm having the same issue with the same error:

Traceback (most recent call last):
  File "C:\Program Files\Sublime Text 3\sublime_plugin.py", line 685, in on_query_completions
    res = vel.on_query_completions(prefix, locations)
  File "C:\Users\Chandler\AppData\Roaming\Sublime Text 3\Packages\Jedi - Python autocompletion\sublime_jedi\completion.py", line 154, in on_query_completions
    location=locations[0]
  File "C:\Users\Chandler\AppData\Roaming\Sublime Text 3\Packages\Jedi - Python autocompletion\sublime_jedi\daemon.py", line 83, in ask_daemon_with_timeout
    return request.result(timeout=timeout)
  File "./python3.3/concurrent/futures/_base.py", line 403, in result
concurrent.futures._base.TimeoutError

Anything I can do to help figure out what's going on? And is there a temporary workaround?

ac130kz · 2018-10-03T06:59:49Z

Same issue (Ubuntu 18.04, fresh installation of Sublime), even though my "python_interpreter": "/usr/bin/python3":

Traceback (most recent call last):
  File "/opt/sublime_text/sublime_plugin.py", line 685, in on_query_completions
    res = vel.on_query_completions(prefix, locations)
  File "/home/ac130kz/.config/sublime-text-3/Packages/Jedi - Python autocompletion/sublime_jedi/completion.py", line 154, in on_query_completions
    location=locations[0]
  File "/home/ac130kz/.config/sublime-text-3/Packages/Jedi - Python autocompletion/sublime_jedi/daemon.py", line 83, in ask_daemon_with_timeout
    return request.result(timeout=timeout)
  File "./python3.3/concurrent/futures/_base.py", line 403, in result
concurrent.futures._base.TimeoutError

mew1033 · 2018-10-29T22:50:06Z

@srusskih Is there anything I can do to help locate this issue and reproduce it? I've disabled Sublime Jedi because of the slowdowns and I'm missing it.

srusskih · 2018-10-30T11:10:21Z

@mew1033 do you have a way how the issue can be reproduced?

srusskih · 2018-11-12T17:25:11Z

reproducible case is in #276 👍 but didn't measure the RAM... only slowdown

mew1033 · 2018-11-14T05:49:28Z

@srusskih Do you still want me to get a reproducible configuration set up, or did you figure it out from @onebits?

barathbheeman · 2018-12-08T14:37:34Z

any update on the issue? with jedi st is literally unusable with up to 5 seconds of delay every time I try type anything.

ninja542 · 2018-12-08T20:40:09Z

yeah, I recently noticed this, and I had to remove the package, but this package was pretty useful and I hope I can continue using it :/

srusskih · 2018-12-10T13:32:32Z

any update on the issue? with jedi st is literally unusable with up to 5 seconds of delay every time I try type anything.

@barathbheeman sounds really bad, it looks like Jedi (jedi lib) by it self are really slow on your env.

srusskih · 2018-12-10T13:34:03Z

as workaround feel free install one of previous versions :(

srusskih · 2018-12-10T15:06:49Z

Have found this davidhalter/jedi#1242

davidhalter · 2018-12-15T19:22:08Z

I only just realized how bad this issue was. Big sorry for that. I therefore did a release immediately. (0.13.2)

srusskih · 2018-12-27T11:10:03Z

please check a new release

mew1033 · 2019-03-06T20:32:05Z

I'm still having slowdowns on the latest release. Windows 10, Sublime 3, Python 2.

kylebebak · 2019-03-28T15:02:11Z

I know this may not be helpful, but I can also still confirm that auto-complete for Python files using Sublime Jedi is much slower than auto-complete for any other language I use in ST.

Entering if var is None: into the editor causes noticeable lag as I type if, var, and None.

If I retype if var is None: again, right below the previous if var is None, I run into the same lag. In other words, it doesn't get faster after fetching completions for a given piece of text.

Who knows if this is an issue with Sublime Jedi or with Jedi, but this is obviously not how this is supposed to work. And again, in the past (before August 2018 for me) lag was never a problem with this plugin.

kylebebak · 2019-03-28T15:30:06Z

I added the following to User/Python.sublime-settings to disable auto-complete except on ., as suggested in readme.

"auto_complete_triggers": [{"selector": "source.python", "characters": "."}],
"auto_complete_selector": "-",

This is nice, because ST no longer lags like crazy trying to fetch the Jedi auto-completions, and you still get the info popup if you hover over text, and you get auto-completions for methods and properties and anything that's preceded by ..

But, it kills the rest of Sublime Text's auto-complete functionality for your Python files, which is totally unacceptable. The readme even suggests doing this not just for your Python files, but in all your ST files, by placing the above text in User/Preferences.sublime-settings!

The workaround to me seems to be creating a setting that only disables auto-complete for Sublime Jedi, while keeping the rest of its functionality. I will look into doing this.

srusskih · 2019-03-28T16:25:55Z

@kylebebak thanks for a feedback.
If you would have a time and inspiration could you please collect some logs to investigate the issue?
how to set up logging you can find here https://github.com/srusskih/SublimeJEDI#logging

thanks. will try to look what I can find.

kylebebak · 2019-03-28T19:22:47Z

Hey @srusskih

Thanks (again) for the super fast response!

I opened a PR, #286, that addresses this slowdown in a different way. Please let me know what you think!

If you'd still like me to look into logging, I can do that this weekend.

deathaxe closed this as completed Aug 6, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Jedi freezes ST and causes very high RAM usage. #271

Jedi freezes ST and causes very high RAM usage. #271

deathaxe commented Aug 25, 2018 •

edited

Loading

kylebebak commented Aug 28, 2018

srusskih commented Aug 28, 2018

deathaxe commented Aug 29, 2018

kylebebak commented Aug 30, 2018

srusskih commented Aug 30, 2018

srusskih commented Aug 30, 2018

kylebebak commented Sep 6, 2018 •

edited

Loading

deathaxe commented Sep 6, 2018

kylebebak commented Sep 6, 2018

mew1033 commented Sep 20, 2018 •

edited

Loading

ac130kz commented Oct 3, 2018 •

edited

Loading

mew1033 commented Oct 29, 2018

srusskih commented Oct 30, 2018 •

edited

Loading

srusskih commented Nov 12, 2018 •

edited

Loading

mew1033 commented Nov 14, 2018

barathbheeman commented Dec 8, 2018

ninja542 commented Dec 8, 2018

srusskih commented Dec 10, 2018 •

edited

Loading

srusskih commented Dec 10, 2018

srusskih commented Dec 10, 2018

davidhalter commented Dec 15, 2018

srusskih commented Dec 27, 2018

mew1033 commented Mar 6, 2019

kylebebak commented Mar 28, 2019 •

edited

Loading

kylebebak commented Mar 28, 2019 •

edited

Loading

srusskih commented Mar 28, 2019

kylebebak commented Mar 28, 2019

Jedi freezes ST and causes very high RAM usage. #271

Jedi freezes ST and causes very high RAM usage. #271

Comments

deathaxe commented Aug 25, 2018 • edited Loading

Description

Console Output

TaskManager

Infos

kylebebak commented Aug 28, 2018

srusskih commented Aug 28, 2018

deathaxe commented Aug 29, 2018

kylebebak commented Aug 30, 2018

srusskih commented Aug 30, 2018

srusskih commented Aug 30, 2018

kylebebak commented Sep 6, 2018 • edited Loading

deathaxe commented Sep 6, 2018

kylebebak commented Sep 6, 2018

mew1033 commented Sep 20, 2018 • edited Loading

ac130kz commented Oct 3, 2018 • edited Loading

mew1033 commented Oct 29, 2018

srusskih commented Oct 30, 2018 • edited Loading

srusskih commented Nov 12, 2018 • edited Loading

mew1033 commented Nov 14, 2018

barathbheeman commented Dec 8, 2018

ninja542 commented Dec 8, 2018

srusskih commented Dec 10, 2018 • edited Loading

srusskih commented Dec 10, 2018

srusskih commented Dec 10, 2018

davidhalter commented Dec 15, 2018

srusskih commented Dec 27, 2018

mew1033 commented Mar 6, 2019

kylebebak commented Mar 28, 2019 • edited Loading

kylebebak commented Mar 28, 2019 • edited Loading

srusskih commented Mar 28, 2019

kylebebak commented Mar 28, 2019

deathaxe commented Aug 25, 2018 •

edited

Loading

kylebebak commented Sep 6, 2018 •

edited

Loading

mew1033 commented Sep 20, 2018 •

edited

Loading

ac130kz commented Oct 3, 2018 •

edited

Loading

srusskih commented Oct 30, 2018 •

edited

Loading

srusskih commented Nov 12, 2018 •

edited

Loading

srusskih commented Dec 10, 2018 •

edited

Loading

kylebebak commented Mar 28, 2019 •

edited

Loading

kylebebak commented Mar 28, 2019 •

edited

Loading