Build against JupyterLab 4.1, fix tests

[krassowski]

References

Builds upon #1052

Code changes

• uses new LSP extensionFactory API (JupyterLab 4.1) to remove workarounds which we used to make it kind of work in Lab 4.0.
• adds a workaround for upstream issue LSP extensionFactory is never called for file editors jupyterlab/jupyterlab#15868
• reduces some warnings in places known to have false positives to debug statements
• removes no-longer needed patches for VirtualDocument as these were merged upstream and released in 4.1

User-facing changes

• requires JupyterLab 4.1
• the use of new new LSP extensionFactory API means that:
  • LSP features work more reliably as the correct virtual document and editor is known rather than found from heuristics
  • LSP CodeMirror extensions are not added to editors which do not have an LSP-enabled editor reducing spurious warnings and improving performance
  • LSP CodeMirror extensions are only added once the LSP server was connected
• fixed a bug in new highlights getting instantly removed when navigating to a beginning of cell from another cell due to focus events being handled incorrectly
• fixed a regression introduced in v5.0.2 which meant that overrides for native kernel and context completion providers were never active

Backwards-incompatible changes

None

Chores

• linted
• tested
• documented
• changelog entry

[github-actions]

👈 Launch a binder notebook on branch krassowski/jupyterlab-lsp/lab-41-cont

[krassowski]

I guess this should not be the same as cache folder.

[krassowski]

All Windows tests are failing with After 10.0s, D:\a\jupyterlab-lsp\jupyterlab-lsp\build\reports\windows_38\atest\2\logs\lab.log.txt did not contain: ('The following Language Servers will be available',)

Failed to read D:\a\jupyterlab-lsp\jupyterlab-lsp\build\reports\windows_311\atest\1\logs\lab.log.txt forcing unicode...
 'utf-8' codec can't decode byte 0xf6 in position 138499: invalid start byte

[krassowski]

So we are back at mostly green except for Windows + Python 3.8:

I will merge this and create follow-up issues to tackle things to address. Thanks @bollwyvl for starting this!