Code completion sometimes overwrites existing text

[krassowski]

Happens only when completing a code symbol which is in the middle of a line (not at the end). Needs unit tests to prevent this in the future.

Example:
DataFrame(da<tab>) leads to DataFrame(dta=), should DataFrame(data=)

R examples:
data.frame(row.nam<tab> → data.frame(row.nrow.names
data.f<tab> → data.fdata.frame

[krassowski]

Minimal Python example without the need to import anything:

CamelCase = 1
mean(cam<tab>

[krassowski]

Another test case: from jupyter_helpers.tabl<tab>

[krassowski]

def display_table():
    pass

disp<tab>data

leads to
ddisplay_table<cursor>
should be:
display_table<cursor>data

[yunair]

I don't think this is a valid test case

def display_table():
    pass

disp<tab>data

Python doesn't support to write code like this

[krassowski]

Some explanations: I use <tab> to indicate invocation of the completer with tab key and <cursor> to indicate the expected position of cursor (thus the code is valid).

[krassowski]

The from jupyter_helpers.tabl<tab> can be reproduced in bare Python with:

from os.pat<tab> → choose path or os.path; the result expected for both is from os.path, but the latter produces os.os.path. This is because the kernel returns prefixed tokens (with os.), while LSP (pyls) does not (and we can handle either prefixed responses or non-prefixed responses, but not both mixed together, due to how the completer in JupyterLab is written).

• we cannot detect the prefix from kernel response alone
• we have to take a look at the token preceding the requested token to infer the prefix, but then the completions with repeats may be affected..
• or, we could strip the prefix from kernel responses - and this is what is already present in the merge step - apparently does not always work...

[krassowski]

Even more nicer test case from os.pathse<tab> → from os.pathsep. This one had false negatives. Need to have one with pyls completion; from os.pat<tab> → select pathsep → from os.pathsep will work.

I now got fixes for every case reported so far, will PR soon.