Extract method codeaction broken

[balazser]

• pylsp-rope version: 0.1.11
• Text editor/IDE/LSP Client: pylsp v1.7.1
• Python version: 3.11.2
• Operating System: macOS 13.0
  • INFO: Python version: 3.11.2
  • INFO: pynvim version: 0.4.3

Description

I have installed pylsp-rope and experiencing issues with the extract (global as well) method functionality.

Describe what you were trying to get done.

The examples from https://github.com/python-rope/rope/blob/master/docs/overview.rst#extract-method end up with the errors in the following section.

def a_func():
    a = 1
    b = 2 * a
    c = a * 2 + b * 3

Tell us what happened, what went wrong, and what you expected to happen.

Without nvim region/visual-mode:

LSP[pylsp] pylsp-rope: Should extract complete statements.                                                                                                                                   
LSP[pylsp] pylsp-rope: string index out of range  

With nvim region/visual-mode:

LSP[pylsp] pylsp-rope: string index out of range
LSP[pylsp] pylsp-rope: Syntax error in file <string> line <1>: unexpected EOF while parsing

Details

If there was a crash, please include the traceback here.

[ERROR][2023-02-26 21:56:53] .../vim/lsp/rpc.lua:733	"rpc"	"pylsp"	"stderr"	'2023-02-26 21:56:53,018 CET - ERROR - pylsp_rope.plugin - Exception when doing workspace/executeCommand: string index out of range
Traceback (most recent call last):
  File "/Users/balazser/.pyenv/versions/3.8.16/envs/my-project/lib/python3.8/site-packages/pylsp_rope/plugin.py", line 143, in pylsp_execute_command
    return commands[command](workspace, **arguments[0])()
  File "/Users/balazser/.pyenv/versions/3.8.16/envs/my-project/lib/python3.8/site-packages/pylsp_rope/refactoring.py", line 36, in __call__
    rope_changeset = self.get_changes()
  File "/Users/balazser/.pyenv/versions/3.8.16/envs/my-project/lib/python3.8/site-packages/pylsp_rope/refactoring.py", line 100, in get_changes
    rope_changeset = refactoring.get_changes(
  File "/Users/balazser/.pyenv/versions/3.8.16/envs/my-project/lib/python3.8/site-packages/rope/refactor/extract.py", line 82, in get_changes
    new_contents = _ExtractPerformer(info).extract()
  File "/Users/balazser/.pyenv/versions/3.8.16/envs/my-project/lib/python3.8/site-packages/rope/refactor/extract.py", line 287, in extract
    extract_info = self._collect_info()
  File "/Users/balazser/.pyenv/versions/3.8.16/envs/my-project/lib/python3.8/site-packages/rope/refactor/extract.py", line 313, in _collect_info
    self._find_definition(extract_collector)
  File "/Users/balazser/.pyenv/versions/3.8.16/envs/my-project/lib/python3.8/site-packages/rope/refactor/extract.py", line 386, in _find_definition
    collector.definition = parts.get_definition()
  File "/Users/balazser/.pyenv/versions/3.8.16/envs/my-project/lib/python3.8/site-packages/rope/refactor/extract.py", line 550, in get_definition
    return "\
%s" % self._get_function_definition()
  File "/Users/balazser/.pyenv/versions/3.8.16/envs/my-project/lib/python3.8/site-packages/rope/refactor/extract.py", line 602, in _get_function_definition
    unindented_body = self._get_unindented_function_body(returns)
  File "/Users/balazser/.pyenv/versions/3.8.16/envs/my-project/lib/python3.8/site-packages/rope/refactor/extract.py", line 726, in _get_unindented_function_body
    return self._get_single_expression_function_body()
  File "/Users/balazser/.pyenv/versions/3.8.16/envs/my-project/lib/python3.8/site-packages/rope/refactor/extract.py", line 737, in _get_single_expression_function_body
    extracted = _get_single_expression_body(self.info.extracted, info=self.info)
  File "/Users/balazser/.pyenv/versions/3.8.16/envs/my-project/lib/python3.8/site-packages/rope/refactor/extract.py", line 1111, in _get_single_expression_body
    extracted.lstrip()[0] in "({[" and extracted.rstrip()[-1] in ")}]"
IndexError: string index out of range
'

Better exception:

[ERROR][2023-02-26 22:15:56] .../vim/lsp/rpc.lua:733	"rpc"	"pylsp"	"stderr"	"2023-02-26 22:15:56,343 CET - ERROR - pylsp_rope.plugin - Exception when doing workspace/executeCommand: string index out of range
Traceback (most recent call last):
  File \"/Users/balazser/.pyenv/versions/3.8.16/envs/my-project/lib/python3.8/site-packages/pylsp_rope/plugin.py\", line 143, in pylsp_execute_command
    return commands[command](workspace, **arguments[0])()
           │        │        │            └ [{'document_uri': 'file:///Users/balazser/test.py', 'global_': False, 'similar': False, 'range...
           │        │        └ <pylsp.workspace.Workspace object at 0x105cf51c0>
           │        └ 'pylsp_rope.refactor.extract.method'
           └ {'pylsp_rope.refactor.extract.method': <class 'pylsp_rope.refactoring.CommandRefactorExtractMethod'>, 'pylsp_rope.refactor.extra...
  File \"/Users/balazser/.pyenv/versions/3.8.16/envs/my-project/lib/python3.8/site-packages/pylsp_rope/refactoring.py\", line 36, in __call__
    rope_changeset = self.get_changes()
                     └ <pylsp_rope.refactoring.CommandRefactorExtractMethod object at 0x10676cd30>
  File \"/Users/balazser/.pyenv/versions/3.8.16/envs/my-project/lib/python3.8/site-packages/pylsp_rope/refactoring.py\", line 100, in get_changes
    rope_changeset = refactoring.get_changes(
  File \"/Users/balazser/.pyenv/versions/3.8.16/envs/my-project/lib/python3.8/site-packages/rope/refactor/extract.py\", line 82, in get_changes
    new_contents = _ExtractPerformer(info).extract()
                   │                 └ <rope.refactor.extract._ExtractInfo object at 0x10676c6d0>
                   └ <class 'rope.refactor.extract._ExtractPerformer'>
  File \"/Users/balazser/.pyenv/versions/3.8.16/envs/my-project/lib/python3.8/site-packages/rope/refactor/extract.py\", line 291, in extract
    extract_info = self._collect_info()
                   └ <rope.refactor.extract._ExtractPerformer object at 0x10676ccd0>
  File \"/Users/balazser/.pyenv/versions/3.8.16/envs/my-project/lib/python3.8/site-packages/rope/refactor/extract.py\", line 317, in _collect_info
    self._find_definition(extract_collector)
    │                     └ <rope.refactor.extract._ExtractCollector object at 0x10676c310>
    └ <rope.refactor.extract._ExtractPerformer object at 0x10676ccd0>
  File \"/Users/balazser/.pyenv/versions/3.8.16/envs/my-project/lib/python3.8/site-packages/rope/refactor/extract.py\", line 390, in _find_definition
    collector.definition = parts.get_definition()
    │                      └ <rope.refactor.extract._ExtractMethodParts object at 0x10676ce20>
    └ <rope.refactor.extract._ExtractCollector object at 0x10676c310>
  File \"/Users/balazser/.pyenv/versions/3.8.16/envs/my-project/lib/python3.8/site-packages/rope/refactor/extract.py\", line 554, in get_definition
    return \"\
%s\" % self._get_function_definition()
                    └ <rope.refactor.extract._ExtractMethodParts object at 0x10676ce20>
  File \"/Users/balazser/.pyenv/versions/3.8.16/envs/my-project/lib/python3.8/site-packages/rope/refactor/extract.py\", line 606, in _get_function_definition
    unindented_body = self._get_unindented_function_body(returns)
                      │                                  └ []
                      └ <rope.refactor.extract._ExtractMethodParts object at 0x10676ce20>
  File \"/Users/balazser/.pyenv/versions/3.8.16/envs/my-project/lib/python3.8/site-packages/rope/refactor/extract.py\", line 730, in _get_unindented_function_body
    return self._get_single_expression_function_body()
           └ <rope.refactor.extract._ExtractMethodParts object at 0x10676ce20>
  File \"/Users/balazser/.pyenv/versions/3.8.16/envs/my-project/lib/python3.8/site-packages/rope/refactor/extract.py\", line 741, in _get_single_expression_function_body
    extracted = _get_single_expression_body(self.info.extracted, info=self.info)
                │                           │                         └ <rope.refactor.extract._ExtractMethodParts object at 0x10676ce20>
                │                           └ <rope.refactor.extract._ExtractMethodParts object at 0x10676ce20>
                └ <function _get_single_expression_body at 0x1064f2670>
  File \"/Users/balazser/.pyenv/versions/3.8.16/envs/my-project/lib/python3.8/site-packages/rope/refactor/extract.py\", line 1120, in _get_single_expression_body
    extracted.lstrip()[0] in \"({[\" and extracted.rstrip()[-1] in \")}]\"
    │                                  └ ''
    └ ''
IndexError: string index out of range

"

[lieryan]

Hi @balazser, thanks for writing this issue.

A few questions, what lsp client are you using? Are you using the native Neovim lsp, CoC, vim-lsp, or something else?

[balazser]

Hi @lieryan,
I'm using native nvim lsp with neovim/nvim-lspconfig. Is there anything else I can do to help you with debugging?

[lieryan]

Hi @balazser, thanks for clarifying that you're using Neovim's native LSP client.

The codeaction support in Neovim's native LSP is very, very buggy. I had already documented some of the issues of neovim native client here, but one of the most visible and annoying bug is that neovim sends garbage line/col numbers to the LSP server on the first code action triggered in a file in an editing session. I had just retested this again just now on NVIM v0.7.2 (the latest Neovim version available on Ubuntu), and the bug is still there; I'm surprised Neovim hadn't fixed this glaring bug for so long.

There's not much we can really do to workaround this bug from within pylsp-rope, we could have caught the garbage value to prevent the pylsp-rope from returning a traceback, but that's really only hiding a broken client behavior. None of the other LSP clients in Vim that I tested on (vim-lsp, ALE, and Coc) exhibits this behavior.

To workaround this bug, you need to select a text, trigger a code action, unselect the text, re-select again, and then re-trigger the code action. Obviously, that is an unusably bad user experience, which is why our docs recommends against using the Neovim native LSP. Any of the other LSP plugins works well in Neovim. Click here for a video demonstrating the issue and the workaround.

If you want to help with this issue, you can try reporting the bug in Neovim's bug tracker.