autocompletion for python is very slow

[vv111y]

• I am using in spacemacs, let me know what config info you need
• company-lsp-async t
• two python processes are hogging CPU

profiler (excerpt) below:

- timer-event-handler                                            8064  58%
 - apply                                                         8060  58%
  - company-idle-begin                                           4302  31%
   - company-auto-begin                                          4107  29%
    - company--perform                                           4107  29%
     - company--begin-new                                        4107  29%
      - company-calculate-candidates                             4105  29%
       - company--fetch-candidates                               4045  29%
        - sit-for                                                4000  29%
         - read-event                                            3824  27%
          - timer-event-handler                                  3782  27%
           - apply                                               3781  27%
            - #<compiled 0x476069e1>                             2067  15%
             + lsp-ui-sideline--run                              2067  15%
            - #<compiled 0x400e1a3f>                             1703  12%
             + eldoc-print-current-symbol-info                   1703  12%
            + sp-show--pair-function                                6   0%
            + flycheck--handle-idle-change-in-buffer                  1   0%
          + #<compiled 0x44502a6d>                                 39   0%
        + #<compiled 0x5e493301>                                   42   0%
        + company-call-backend-raw                                  3   0%
       + company--postprocess-candidates                           59   0%
         company--preprocess-candidates                             1   0%
      + company--multi-backend-adapter                              2   0%
   + company-post-command                                         193   1%
  - #<compiled 0x5e4663f5>                                       2806  20%
   - lsp-ui-sideline--run                                        2804  20%
    - lsp--send-request-async                                    2709  19%
     - lsp--send-no-wait                                         2550  18%
      - process-send-string                                      2332  17%
       - timer-event-handler                                     2326  16%
        + apply                                                  2325  16%
          timer-activate-when-idle                                  1   0%
       + #<compiled 0x44502a6d>                                     6   0%

[vv111y]

Reported here palantir/python-language-server#397 in the case of being in a virtualenv.

I have this problem while not in a virtualenv though.

[tigersoldier]

In general the slowness comes from the language server if you are using async completion. It's hard to tell if it's the case without knowing your python code base. On the emacs side, the value of company-idle-delay may affect how often a request is sent to the server.

You may also try setting company-lsp-cache-candidates to 'auto (this is the default value) or t. This will cache the first completion results and reuse them as much as possible. Won't help for the initial completion but can improve for later filtering.

[vv111y]

It looks like that is the case. I watched htop while typing, a pyls process and a jedi process hogged cpu while autocomplete hung.

Interesting, not all modules responded the same. Tensorflow worst offender so far; os, sys, matplotlib, didn't take as much time.

So far company-lsp-cache-candidates not helping.

I'll open issue at python-language-server

[vv111y]

The autocompletes uses Jedi and he has fixed the issue with large libraries such as tensorflow davidhalter/jedi#1116 in releases 13.0 and 13.1

Other relevant lsp packages have been updated as well. Appears problem is solved.

Thank you.