Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PR: Migrate introspection services to use the Language Server Protocol (LSP) #4751

Merged
merged 127 commits into from
Aug 20, 2018
Merged
Changes from 1 commit
Commits
Show all changes
127 commits
Select commit Hold shift + click to select a range
4ee91c9
Add proof-of-concept server
andfoy Jul 15, 2017
4d0c22e
Improve editor and workspace capabilites documentation
andfoy Jul 18, 2017
6e2139f
Handle SIGINT and SIGTERM signals to close all connections
andfoy Jul 19, 2017
db63b91
Close TCP socket and terminate language-server process on SIGTERM/SIGINT
andfoy Jul 21, 2017
c399889
Handle ConnectionAbortedError on Windows
andfoy Jul 21, 2017
8b0f176
Handle SIGBREAK on Windows
andfoy Jul 21, 2017
536fef9
Handle SIGBREAK on Widows
andfoy Jul 21, 2017
ea464c6
Add connection timeout instead of waiting a fixed time interval
andfoy Jul 21, 2017
f7fcdf4
Module extraction and code refactor
andfoy Jul 21, 2017
e25d285
Process server responses
andfoy Jul 21, 2017
49ec0f6
Add basic expected server capabilities settings
andfoy Jul 22, 2017
9afeb75
Add LSPClient base implementation skeleton
andfoy Jul 24, 2017
b1ec308
Add class and function decorators to differentiate LSP method handlers
andfoy Jul 24, 2017
2184681
Extract LSP method signature to decorator
andfoy Jul 25, 2017
96db142
Merge remote-tracking branch 'upstream/master' into language_server_a…
andfoy Jul 25, 2017
2e583dd
Send initialize request when transport layer is ready
andfoy Jul 25, 2017
20f65d6
Minor settings configuration
andfoy Jul 26, 2017
388cdd2
Merge remote-tracking branch 'upstream/master' into language_server_a…
andfoy Aug 10, 2017
9a54c69
Start LSP client from Spyder main window
andfoy Aug 10, 2017
09637f5
Start LSP client after creating editor plugin
andfoy Aug 10, 2017
ae707a0
LSP Plugin Manager Config Page WIP
andfoy Aug 11, 2017
7959110
Plugin configuration page is now complete
andfoy Aug 12, 2017
a46774e
Minor command/host validation issues corrected
andfoy Aug 12, 2017
20d8ee1
Start LSP client on file open action
andfoy Aug 15, 2017
a98b8ce
Fix PEP8 errors
andfoy Aug 15, 2017
78d4276
Merge remote-tracking branch 'upstream/master' into language_server_a…
andfoy Aug 15, 2017
23e0ead
Close language server when is not required
andfoy Aug 16, 2017
387bf35
Correct PEP8 errors
andfoy Aug 16, 2017
9e16e7a
Select port using select_port
andfoy Aug 17, 2017
884d95e
Reload clients if settings are updated
andfoy Aug 24, 2017
280f725
Merge remote-tracking branch 'upstream/master' into language_server_a…
andfoy Aug 27, 2017
bc384e7
Implement textDocument/didOpen request
andfoy Aug 28, 2017
2db33c3
Add textDocument/publishDiagnostics handler
andfoy Aug 29, 2017
94fd388
Back to work
andfoy Sep 18, 2017
48d5536
Send textDocument/DidOpen request each time a file is opened
andfoy Sep 19, 2017
b0e6979
Remove tabs
andfoy Sep 19, 2017
d7c0f21
Transport layer should read Content-Length bytes
andfoy Sep 20, 2017
4b95a4b
Improve textDocument/didOpen notifications behaviour at startup
andfoy Sep 20, 2017
acfed93
Add textDocument/publishDiagnostics handler on codeeditor
andfoy Sep 20, 2017
7ba5ff1
Display linter hints/warnings/errors on Line Number area
andfoy Sep 20, 2017
f177bda
Hightlight linter conflicting selections by modifying bg color
andfoy Sep 21, 2017
e2beec1
Improve warning highlighting visualization
andfoy Sep 21, 2017
1e125d5
textDocument/didChange first experiments
andfoy Sep 22, 2017
54187c4
Send all document instead of sending incremental updates
andfoy Sep 22, 2017
11c5d5a
Remove linter selections after 10s
andfoy Sep 25, 2017
8a574aa
Merge changes from upstream
andfoy Sep 25, 2017
6909f5b
Hid linter selection after 5s
andfoy Sep 25, 2017
a5fd36a
Solve ZMQ communication and blocking issues
andfoy Sep 26, 2017
2191f6c
Send textDocument/completion requests
andfoy Sep 26, 2017
20183a8
textDocument/completion request responses should arrive to the codeed…
andfoy Sep 26, 2017
63b0888
Auto completion should be working now
andfoy Sep 27, 2017
d627e21
Close completion widget on Escape Key press
andfoy Sep 27, 2017
45b607f
Display completion documentation alongside dialog (If available)
andfoy Sep 27, 2017
330ba83
Show documentation tooltip only if it's available
andfoy Sep 27, 2017
e5d9d92
Handle and process textDocument/signatureHelp requests
andfoy Sep 28, 2017
dda4660
Handle textDocument/hover requests
andfoy Sep 29, 2017
3afde13
Merge with upstream
andfoy Oct 2, 2017
b3c7bbd
Disable some features
andfoy Oct 2, 2017
1db53b2
Improve socket reading procedure
andfoy Oct 4, 2017
ccc39d8
Merge remote-tracking branch 'upstream/master' into language_server_a…
andfoy Oct 20, 2017
db0c317
Perform Go-to-definition requests
andfoy Oct 20, 2017
12db7db
Go To Definition: Same file
andfoy Oct 31, 2017
bf4fcbe
Go To Definition: Disable end column highlighting
andfoy Oct 31, 2017
a1c960b
Go to definition complete
andfoy Nov 5, 2017
a0398c4
Implement document/WillSave notification
andfoy Nov 10, 2017
4ae4ee0
Implement Document/DidClose notification
andfoy Nov 10, 2017
85253fb
Disable LSP configuration page
andfoy Nov 10, 2017
354392e
Cleanup begins
andfoy Nov 16, 2017
b9f9e29
Close all servers when Spyder closes down
andfoy Nov 16, 2017
7330308
Merge files with upstream
andfoy Nov 16, 2017
7c4bcb0
Correct lsp_client.py __main__ test
andfoy Nov 16, 2017
1e37637
Correct initialization problems
andfoy Nov 17, 2017
dc07070
Merge with upstream
andfoy Jun 22, 2018
02062d0
Minor transport consumer issues fixed
andfoy Jun 22, 2018
a029c2f
Go-to-definition fixed
andfoy Jun 22, 2018
a9dadde
Disable function hinting when documentation is None
andfoy Jun 22, 2018
8490670
Test migration begins
andfoy Jun 22, 2018
b750784
Prevent LSP transport client module testing
andfoy Jun 22, 2018
72778e0
Add depdencies to Travis
andfoy Jun 22, 2018
46d9fed
Add depdencies to Appveyor
andfoy Jun 23, 2018
69711a8
Prevent LSP transport client module testing
andfoy Jun 23, 2018
e21f0d3
Enable RDP debugging
andfoy Jun 23, 2018
79bd920
Add pexpect to dependencies
andfoy Jun 23, 2018
8ae9fac
Warning diagnostic tests fixed
andfoy Jun 23, 2018
3940e54
Fix introspection test
andfoy Jun 23, 2018
d9d07ee
Integrate documentation inspection calls to LSP methods
andfoy Jun 24, 2018
c2f777f
Fix get_help test
andfoy Jun 24, 2018
9ab3c25
Fix calltip test
andfoy Jun 24, 2018
c499108
Prevent test_get_help from hanging on
andfoy Jun 24, 2018
229f30a
Kill LSP processes before get_help testing
andfoy Jun 24, 2018
9ee5ee9
Killtest_calltip: Send LSP request manually
andfoy Jun 24, 2018
cc719ad
test_calltip: Remove document_did_change call
andfoy Jun 24, 2018
3cb2853
test_calltip: Fixed document_did_open call
andfoy Jun 24, 2018
5fdd128
test_calltip: Fixed
andfoy Jun 24, 2018
3e00469
test_adding_warnings: Fixed
andfoy Jun 24, 2018
a6a5cae
text_get_help: Fixed
andfoy Jun 24, 2018
b5969ff
Skip test_calltip on PY2
andfoy Jun 24, 2018
3c89543
Merge with upstream
andfoy Jun 25, 2018
f3aa040
Disable installation on Travis
andfoy Jun 25, 2018
16cee99
Improve server spawning and teardown
andfoy Jun 26, 2018
88fcb65
Update test_get_help
andfoy Jun 26, 2018
58b3340
Disable Spyder installation on Appveyor
andfoy Jun 26, 2018
cc0ce6f
Prevent textDocument/didClose call if LSP is down
andfoy Jun 26, 2018
02d90a5
Reimplement TCP consumer on Windows
andfoy Jun 26, 2018
3b1203d
Prevent TypeError while parsing JSON
andfoy Jun 26, 2018
329a03c
Daemonize consumer thread, create blocking socket
andfoy Jun 26, 2018
47ecb9d
Daemonize thread via class attribute
andfoy Jun 26, 2018
15cd3f5
Disable Ctrl+Break event on Windows
andfoy Jun 26, 2018
ad88cf0
test_flag_painting: Fixed
andfoy Jun 26, 2018
7c70fd6
test_flag_painting: Minor error correction
andfoy Jun 26, 2018
5c529be
Merge with upstream
andfoy Jun 27, 2018
c66aa54
Fix byte conversion issues on LSP transport consumer
andfoy Jun 27, 2018
63085cc
Add pyls dependency to NO_CONDA CI machine on Travis
andfoy Jun 27, 2018
5d30978
Codeeditor: Perform LSP request only if LSP server is available
andfoy Jun 27, 2018
2053cc9
Use SPY_TEST_USE_INTROSPECTION to launch pyls on CIs
andfoy Jun 27, 2018
e459fa2
Add test introspection env var on test_warnings
andfoy Jun 27, 2018
70b6c7f
Skip pyls tests on Appveyor/Py3
andfoy Jun 27, 2018
bc781cb
Add pyls JSON configuration sources
andfoy Jun 30, 2018
9b0b29f
Update test_adding_warnings to reflect pydocstyle inclusion
andfoy Jun 30, 2018
c78087d
Update test_move_warnings to reflect pydocstyle inclusion
andfoy Jun 30, 2018
a1222e7
Redirect LSP log file to config folder
andfoy Aug 11, 2018
7e6618c
Merge with master
andfoy Aug 11, 2018
63c628f
Remove additional arg to file open on lsp_client spawn
andfoy Aug 11, 2018
41b5563
Create lsp_logs folder under .config
andfoy Aug 11, 2018
40a3016
Disable Rope, Pydocstyle and McCabe
andfoy Aug 16, 2018
ac66380
Blacklist spyder/utils/code_analysis on Circle
andfoy Aug 16, 2018
0471faa
Disable pydocstyle tests
andfoy Aug 16, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 53 additions & 16 deletions spyder/utils/code_analysis/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,10 @@
"""Spyder MS Language Server v3.0 client implementation."""

import os
import sys
import zmq
import json
import time
import signal
import socket
import logging
import argparse
Expand Down Expand Up @@ -58,6 +58,9 @@
LOG_FORMAT = ('%(levelname) -10s %(asctime)s %(name) -30s %(funcName) '
'-35s %(lineno) -5d: %(message)s')

# LOG_FORMAT = ('%(asctime)s %(hostname)s %(name)s[%(process)d] '
# '(%(funcName)s: %(lineno)d) %(levelname)s %(message)s')

logging.basicConfig(level=logging.INFO, format=LOG_FORMAT)
logging.basicConfig(level=logging.DEBUG, format=LOG_FORMAT)
logging.basicConfig(level=logging.ERROR, format=LOG_FORMAT)
Expand Down Expand Up @@ -85,13 +88,15 @@ def run(self):
while True:
with self.mutex:
if self.stopped:
LOGGER.debug('Stopping Thread...')
break
try:
recv = self.socket.recv(4096)
LOGGER.debug(recv)
self.zmq_sock.send_pyobj(recv)
except socket.error:
pass
LOGGER.debug('Thread stopped.')

def stop(self):
with self.mutex:
Expand Down Expand Up @@ -160,7 +165,6 @@ def __initialize(self):
def start(self):
LOGGER.info('Ready to recieve/attend requests and responses!')
self.reading_thread.start()
self.__listen()

def stop(self):
LOGGER.info('Sending shutdown instruction to server')
Expand All @@ -170,8 +174,13 @@ def stop(self):
if self.is_local_server_running:
LOGGER.info('Closing language server process...')
self.server.terminate()
LOGGER.info('Closing TCP socket...')
self.socket.close()
LOGGER.info('Closing consumer thread...')
self.reading_thread.stop()
LOGGER.debug('Joining thread...')
self.reading_thread.join()
LOGGER.debug('Exit routine should be complete')

def shutdown(self):
method = 'shutdown'
Expand All @@ -185,16 +194,15 @@ def exit(self):
request = self.__compose_request(method, params)
self.__send_request(request)

def __listen(self):
while True:
events = self.zmq_socket.poll(TIMEOUT)
requests = []
while events > 0:
client_request = self.zmq_socket.recv_pyobj()
LOGGER.debug("Client Event: {0}".format(client_request))
requests.append(client_request)
server_request = self.__compose_request('None', {})
self.__send_request(server_request)
def listen(self):
events = self.zmq_socket.poll(TIMEOUT)
requests = []
while events > 0:
client_request = self.zmq_socket.recv_pyobj()
LOGGER.debug("Client Event: {0}".format(client_request))
requests.append(client_request)
server_request = self.__compose_request('None', {})
self.__send_request(server_request)

def __compose_request(self, method, params):
request = {
Expand All @@ -218,16 +226,45 @@ def __send_request(self, request):
self.request_seq += 1


class TerminateSignal(Exception):
"""Terminal exception descriptor."""
pass


class SignalManager:
"""Manage and intercept SIGTERM and SIGKILL signals."""

def __init__(self):
self.original_sigint = signal.getsignal(signal.SIGINT)
self.original_sigterm = signal.getsignal(signal.SIGTERM)
signal.signal(signal.SIGINT, self.exit_gracefully)
signal.signal(signal.SIGTERM, self.exit_gracefully)

def exit_gracefully(self, signum, frame):
LOGGER.info('Termination signal ({}) captured, '
'initiating exit sequence'.format(signum))
raise TerminateSignal("Exit process!")

def restore(self):
signal.signal(signal.SIGINT, self.original_sigint)
signal.signal(signal.SIGTERM, self.original_sigterm)


if __name__ == '__main__':
sig_manager = SignalManager()
client = LanguageServerClient(host=args.server_host,
port=args.server_port,
workspace=args.folder,
zmq_port=args.zmq_port,
use_external_server=args.external_server,
server=args.server,
server_args=unknownargs)
client.start()
try:
client.start()
except KeyboardInterrupt:
client.stop()
sys.exit(0)
while True:
client.listen()
except TerminateSignal:
pass
client.stop()
sig_manager.restore()
os.kill(os.getpid(), signal.SIGTERM)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't work on Windows, i.e. processes can't be killed this way there. If you want a cross-platform way of doing this, please take a look at psutil.kill.