Partial freeze of jupyterlab when typing "pd.read_pickle"

[stephenkraemer]

First of all thanks for your great work! This is really cool stuff!

Description

In notebooks beyond moderate size (a few hundred lines of code and minimal output), when I type "pd.read_pickle('path')", the function call is underlined in red, and jupyterlab partially freezes: i) the kernel does not respond anymore ii) the document cannot be saved anymore iii) also other kernels do not respond anymore.

Reproduce

Unfortunately, this happens so randomly that I have no useful reproducible example. If this makes this issue less than helpful, please close it right away! I just wanted to report my current troubles in case it's interesting.

However, I have found that the error can be prevented if one types slowly enough, so that the autocompletion kicks in. Ie: if I type "pd." and then wait until the autocompletion pop-up appears, the error will not happen.

Expected behavior

"pd.read_pickle" is correctly recognized as legitimate code and jupyterlab does not freeze

Context

• Operating System and version: CentOS
• I am connecting to a worker node on a computing cluster via ssh tunneling - maybe that leads to a lag causing the problem?
• Browser and version: Firefox 72 and Chrome 79
• JupyterLab version: 1.1.4
• jupyter-lsp version: 0.7.0b0
• @krassowski/jupyterlab-lsp version: 0.7.1

Troubleshoot Output

Paste the output from running `jupyter troubleshoot` from the command line here.
You may want to sanitize the paths in the output.

If you think this is worth following up upon, I'd be happy to try and copy-paste log messages when the error appears next!

[krassowski]

Thank you for the report and very useful insights. It sounds like a race condition...

I experienced similar hiccups with the pyls server in the past. My workaround is:

• killall pyls from console, which makes the JL work again
• save documents that need saving
• continue work without lsp features, or restart JupyterLab

I hope the workaround helps - until a better solution is available. Please let me know if it works for you.

[stephenkraemer]

Thank you very much for the quick response! A workaround that allows me to continue my ongoing work when things get stuck is perfect for now, great! I'll report back if this workaround should ever fail. Thanks again!

[bollwyvl]

We're hoping #199 addresses this case by moving as much language server i/o to separate threads which shouldn't block other things happening: stay tuned for frontend/backend releases that include those changes. In addition, we likely need a more fine-grained way to restart a particular server: unfortunately, the most "Lab-native" way of doing this, by hooking into the "Running" sidebar area, isn't available yet on a stable release, but Lab 2.0 is well into its second release candidate, and we'll be tracking progress towards that in #186, but we might need a stop-gap approach for those that can't upgrade for a while even when it does come out.

Let's keep this open in the meantime.

[asteppke]

For reference: I am experiencing a very similar issue in a different context:

Context

• OS: Windows 10
• jupyter-lsp 0.7.0
• jupyterlab 1.2.6
• @krassowski/jupyterlab-lsp v0.7.1

So it seems that this is not tied to a specific JupyterLab version or the OS.

It is not always reproducible though, I could get the Lab interface to stop reacting completely with a simple input like this sometimes:

import pandas as pd
import numpy as np

np.zer <TAB key>

In a longer notebook I can reproduce the error now and on the command line get the following error that originates from jedi:

[I 11:04:33.683 LabApp] Build is up to date                                                                                         
[I 11:04:34.928 LabApp] Kernel started: ea3d1906-5af0-4476-ac1a-0cd9a2b78085                                                        
-2020-02-28 11:04:41,113 UTC - ERROR - pyls_jsonrpc.endpoint - Failed to handle request 1                                           
Traceback (most recent call last):                                                                                                  
  File "C:\Users\alexander\Anaconda3\lib\site-packages\jedi\cache.py", line 141, in wrapper                                         
    return dct[key]                                                                                                                 
KeyError: ((), frozenset({('resolve_stars', True)}))                                                                                
                                                                                                                                    
During handling of the above exception, another exception occurred:                                                                 
                                                                                                                                    
Traceback (most recent call last):                                                                                                  
  File "C:\Users\alexander\Anaconda3\lib\site-packages\pyls_jsonrpc\endpoint.py", line 113, in consume                              
    self._handle_request(message['id'], message['method'], message.get('params'))                                                   
  File "C:\Users\alexander\Anaconda3\lib\site-packages\pyls_jsonrpc\endpoint.py", line 182, in _handle_request                      
    handler_result = handler(params)                                                                                                
  File "C:\Users\alexander\Anaconda3\lib\site-packages\pyls_jsonrpc\dispatchers.py", line 23, in handler                            
    return method(**(params or {}))                                                                                                 
  File "C:\Users\alexander\Anaconda3\lib\site-packages\pyls\python_ls.py", line 330, in m_text_document__hover                      
    return self.hover(textDocument['uri'], position)                                                                                
  File "C:\Users\alexander\Anaconda3\lib\site-packages\pyls\python_ls.py", line 264, in hover                                       
    return self._hook('pyls_hover', doc_uri, position=position) or {'contents': ''}                                                 
  File "C:\Users\alexander\Anaconda3\lib\site-packages\pyls\python_ls.py", line 155, in _hook                                       
    return hook_handlers(config=self.config, workspace=workspace, document=doc, **kwargs)                                           
  File "C:\Users\alexander\Anaconda3\lib\site-packages\pluggy\hooks.py", line 286, in __call__                                      
    return self._hookexec(self, self.get_hookimpls(), kwargs)                                                                       
  File "C:\Users\alexander\Anaconda3\lib\site-packages\pluggy\manager.py", line 93, in _hookexec                                    
    return self._inner_hookexec(hook, methods, kwargs)                                                                              
  File "C:\Users\alexander\Anaconda3\lib\site-packages\pluggy\manager.py", line 337, in traced_hookexec                             
    return outcome.get_result()                                                                                                     
  File "C:\Users\alexander\Anaconda3\lib\site-packages\pluggy\callers.py", line 80, in get_result                                   
    raise ex[1].with_traceback(ex[2])                                                                                               
  File "C:\Users\alexander\Anaconda3\lib\site-packages\pluggy\callers.py", line 52, in from_call                                    
    result = func()                                                                                                                 
  File "C:\Users\alexander\Anaconda3\lib\site-packages\pluggy\manager.py", line 335, in <lambda>                                    
    outcome = _Result.from_call(lambda: oldcall(hook, hook_impls, kwargs))                                                          
  File "C:\Users\alexander\Anaconda3\lib\site-packages\pluggy\manager.py", line 87, in <lambda>                                     
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,                                                          
  File "C:\Users\alexander\Anaconda3\lib\site-packages\pluggy\callers.py", line 208, in _multicall                                  
    return outcome.get_result()                                                                                                     
  File "C:\Users\alexander\Anaconda3\lib\site-packages\pluggy\callers.py", line 80, in get_result                                   
    raise ex[1].with_traceback(ex[2])                                                                                               
  File "C:\Users\alexander\Anaconda3\lib\site-packages\pluggy\callers.py", line 187, in _multicall                                  
    res = hook_impl.function(*args)                                                                                                 
  File "C:\Users\alexander\Anaconda3\lib\site-packages\pyls\plugins\hover.py", line 26, in pyls_hover                               
    signature = next((x.to_string() for x in definition.get_signatures() if x.name == word), '')                                    
  File "C:\Users\alexander\Anaconda3\lib\site-packages\pyls\plugins\hover.py", line 26, in <genexpr>                                
    signature = next((x.to_string() for x in definition.get_signatures() if x.name == word), '')                                    
  File "C:\Users\alexander\Anaconda3\lib\site-packages\jedi\api\classes.py", line 642, in to_string                                 
    return self._signature.to_string()                                                                                              
  File "C:\Users\alexander\Anaconda3\lib\site-packages\jedi\inference\signature.py", line 30, in to_string                          
    s = self.name.string_name + '(' + ', '.join(param_strings()) + ')'                                                              
  File "C:\Users\alexander\Anaconda3\lib\site-packages\jedi\inference\signature.py", line 12, in param_strings                      
    for n in self.get_param_names(resolve_stars=True):                                                                              
  File "C:\Users\alexander\Anaconda3\lib\site-packages\jedi\cache.py", line 143, in wrapper                                         
    result = method(self, *args, **kwargs)                                                                                          
  File "C:\Users\alexander\Anaconda3\lib\site-packages\jedi\inference\signature.py", line 96, in get_param_names                    
    params = process_params(params)                                                                                                 
  File "C:\Users\alexander\Anaconda3\lib\site-packages\jedi\inference\utils.py", line 20, in wrapper                                
    return list(func(*args, **kwargs))                                                                                              
  File "C:\Users\alexander\Anaconda3\lib\site-packages\jedi\inference\star_args.py", line 149, in process_params                    
    signature.get_param_names(resolve_stars=False)                                                                                  
  File "C:\Users\alexander\Anaconda3\lib\site-packages\jedi\inference\star_args.py", line 79, in _remove_given_params               
    for key, _ in arguments.unpack():                                                                                               
  File "C:\Users\alexander\Anaconda3\lib\site-packages\jedi\inference\arguments.py", line 212, in unpack                            
    arrays = self.context.infer_node(el)                                                                                            
  File "C:\Users\alexander\Anaconda3\lib\site-packages\jedi\inference\context.py", line 210, in infer_node                          
    return infer_node(self, node)                                                                                                   
  File "C:\Users\alexander\Anaconda3\lib\site-packages\jedi\inference\syntax_tree.py", line 148, in infer_node                      
    return _infer_node_if_inferred(context, element)                                                                                
  File "C:\Users\alexander\Anaconda3\lib\site-packages\jedi\inference\syntax_tree.py", line 161, in _infer_node_if_inferred         
    return _infer_node_cached(context, element)                                                                                     
  File "C:\Users\alexander\Anaconda3\lib\site-packages\jedi\inference\cache.py", line 42, in wrapper                                
    rv = function(obj, *args, **kwargs)                                                                                             
  File "C:\Users\alexander\Anaconda3\lib\site-packages\jedi\inference\syntax_tree.py", line 166, in _infer_node_cached              
    return _infer_node(context, element)                                                                                            
  File "C:\Users\alexander\Anaconda3\lib\site-packages\jedi\debug.py", line 82, in wrapper                                          
    return func(*args, **kwargs)                                                                                                    
  File "C:\Users\alexander\Anaconda3\lib\site-packages\jedi\inference\syntax_tree.py", line 62, in wrapper                          
    return func(context, *args, **kwargs)                                                                                           
  File "C:\Users\alexander\Anaconda3\lib\site-packages\jedi\inference\syntax_tree.py", line 176, in _infer_node                     
    return infer_atom(context, element)                                                                                             
  File "C:\Users\alexander\Anaconda3\lib\site-packages\jedi\inference\syntax_tree.py", line 301, in infer_atom                      
    return context.py__getattribute__(atom, position=position)                                                                      
  File "C:\Users\alexander\Anaconda3\lib\site-packages\jedi\inference\context.py", line 75, in py__getattribute__                   
    values = ValueSet.from_sets(name.infer() for name in names)                                                                     
  File "C:\Users\alexander\Anaconda3\lib\site-packages\jedi\common\value.py", line 32, in from_sets                                 
    for set_ in sets:                                                                                                               
  File "C:\Users\alexander\Anaconda3\lib\site-packages\jedi\inference\context.py", line 75, in <genexpr>                            
    values = ValueSet.from_sets(name.infer() for name in names)                                                                     
  File "C:\Users\alexander\Anaconda3\lib\site-packages\jedi\inference\names.py", line 237, in infer                                 
    self.tree_name                                                                                                                  
  File "C:\Users\alexander\Anaconda3\lib\site-packages\jedi\plugins\__init__.py", line 21, in wrapper                               
    return built_functions[name](*args, **kwargs)                                                                                   
  File "C:\Users\alexander\Anaconda3\lib\site-packages\jedi\plugins\stdlib.py", line 827, in wrapper                                
    return func(inference_state, context, tree_name)                                                                                
  File "C:\Users\alexander\Anaconda3\lib\site-packages\jedi\inference\syntax_tree.py", line 744, in tree_name_to_values             
    raise ValueError("Should not happen. type: %s" % typ)                                                                           
ValueError: Should not happen. type: del_stmt                                                                                       

The notebook is then frozen and I cannot recover it from this point. Only a manual abort helps where the extension is then complaining about open files:

forrtl: error (200): program aborting due to control-C event                                               
Image              PC                Routine            Line        Source                                 
libifcoremd.dll    00007FFA9B6D3B58  Unknown               Unknown  Unknown                                
KERNELBASE.dll     00007FFAEBF75FD3  Unknown               Unknown  Unknown                                
KERNEL32.DLL       00007FFAED8D7BD4  Unknown               Unknown  Unknown                                
ntdll.dll          00007FFAEE20CED1  Unknown               Unknown  Unknown                                
[I 11:08:23.446 LabApp] Interrupted...                                                                     
[I 11:08:23.447 LabApp] Shutting down 1 kernel                                                             
[I 11:08:23.647 LabApp] Kernel shutdown: ea3d1906-5af0-4476-ac1a-0cd9a2b78085                              
Error in atexit._run_exitfuncs:                                                                            
Traceback (most recent call last):                                                                         
  File "C:\Users\alexander\Anaconda3\lib\site-packages\jupyter_lsp\session.py", line 111, in stop          
    self.writer.close()                                                                                    
  File "C:\Users\alexander\Anaconda3\lib\site-packages\jupyter_lsp\stdio.py", line 39, in close            
    self.stream.close()                                                                                    
OSError: [Errno 22] Invalid argument                                                                       

[krassowski]

@asteppke This seems to be a known problem with pyls dependencies:

• Internal Python Language Server error spyder-ide/spyder#11756
• Completion errors with jedi/lsp/emacs davidhalter/jedi#1484

you should downgrade the Jedi to '0.15.2' and the Parso to '0.5.2'

@stephenkraemer could you pip freeze and check what versions of Jedi/Parso did you use?

[stephenkraemer]

Thanks for following up!

I have jedi == 0.15.2 and parso==0.5.2. My environment was created with conda, then I installed jupyter-lsp with pip. Note: I am a bit behind with the jupyterlab-lsp releases: jupyter-lsp==0.7.0b0 and jupyterlab-lsp==v0.7.1.

I will install jupyterlab 2.0 and the newest release of jupyterlab-lsp soon, and report back if something new happens.

Maybe interesting: for me, the freeze can (as probably expected) happen with different code fragments. But it does seem that for me it exclusively occurs with pandas top level functions (pd.concat, pd.read etc.). Maybe that's just because I use pandas more often than most other packages though...

Thanks a lot!

[krassowski]

Thank you! Yes, I think that the freezes are generally related to auto-completion requests triggered by typing a dot after a module or object name, and not specific to any particular piece of code - but I might be wrong, and the more we know the easier it will be to pin this down, so please continue providing more details if you observe this again.

One hypothesis on the origin: if the pyls server is not handling concurrent requests well, i.e. we start autocompletion and in the meantime the user types something which modifies the virtual document the autocompletion starts to operate on a modified document, something goes wrong and brings an error.