Possibility to collaborate and boost the whole ecosystem

[lextm]

Thanks for creating this project. It looks a lot interesting as you built a custom language server that,

• Has nice autocompletion support.
• Seems to keep HTML pages up-to-date, so that live preview becomes much simpler and more performant.

I'd like to suggest some changes that can make this extension more successful,

1. Merge your syntax highlighting rules with https://github.com/trond-snekvik/vscode-rst so that Trond can fully take care on syntax highlighting feature for all related extensions. I can create a pull request to help install vscode-rst as a dependency of your extension.

   I saw Use semantic tokens for syntax highlighting #238 but that is going to take some time to finish.

2. Once 1 is done, I can then revise my extension vscode-restructuredtext to install yours as a dependency.

Then all reStructuredText users can consume services from the three extensions as below,

• vscode-rst, syntax highlighting
• esbonio, language server and live preview
• vscode-restructuredtext, linter and other editor features.

There is no need to duplicate features in multiple extensions as we each have limited resource to spend.

The collaboration between vscode-restructuredtext and vscode-rst has been a big success, as the latter receives 63k new users from the 353k active users of the former.

Let me know what's your opinion on above.

[alcarney]

Thanks for taking an interest in this! 😄

I think what you suggest makes a lot of sense, if you want to make a pull request for 1. I'd be happy to take a look.

However, the HTML preview hasn't received as much attention as the completions and to be honest I'm not entirely happy with how the VSCode portion of previews is currently implemented - In fact, it is partially broken for me at the moment, I need to put some time in to get it into a better state.

But yes, happy to collaborate 😄

[lextm]

While I am preparing to port more live preview related code to this repo, I noticed that the effort is far more than integrating directly to esbonio language server in vscode-restructuredtext repo.

Therefore, I proposed a new plan as below,

• vscode-rst, syntax highlighting (option 1)
• esbonio, language server alone without the VSCode extension (option 2)
• vscode-restructuredtext, linter and other editor features

I can port more of your extension code over in the coming weeks, and will also report a few gaps that esbonio language server currently have.

[alcarney]

Is there anything in particular that is causing an issue?

I have thought about changing the way the preview works… currently the server tells the extension where the build files live and the extension reads the html from disk and displays it in a webview. The problem with this is that all the urls for images, css, js etc need to be rewritten to work within the isolated context the webview runs in - otherwise none of them will load correctly.

Now the extension does attempt to do this, but I keep finding situations where urls are missed and the preview doesn’t load the page correctly.

Instead I’ve played around with having the server spin up a http.server in the build directory and having the webview pull in the html with an <iframe>. This seems to work better though there are some details to figure out when it comes to reloading the page and keeping the preview in sync with the current document.

Do you have any thoughts on this? I imagine any changes here would have an impact on the work you’re trying to do.

[alcarney]

Also I’m thinking about pushing a release out in the not too distant future (maybe towards the end of March?) as there’s quite a few changes now that would be good to get out there - would that have any impact on the work you’re doing?

[lextm]

So, talk about some history here.

The live preview in vscode-restructuredtext was primarily a mix from VSCode's own Markdown extension and https://marketplace.visualstudio.com/items?itemName=tht13.rst-vscode. Throughout the years the main problems are,

1. It executes sphinx-build behind the scene (each round of modification triggers a new build), which is terribly slow.
2. Its line counting mechanism is elementary, so synchronized scrolling works badly.

Your approach today is much cleaner, and the plan to add a local HTTP server looks even cooler, but forcing vscode-restructuredtext users (369k) today to use esbonio's live preview deprives them from many of the useful features. That's why I chose to integrate the esbonio language server directly with vscode-restructuredtext yesterday and shipped the new releases. It keeps all current feature set (single file preview, synchronized scrolling, show source, etc.) there, and leaving enough time for esbonio's live preview to grow up.

No rush here for a new release, as I can see you have many in the plate. I am monitoring your changes, so will incorporate whatever necessary to vscode-restructuredtext.

Note that the snooty-lextudio package was downloaded 15k last month, https://pypistats.org/packages/snooty-lextudio, so you should expect a similar number for esbonio package in the coming months. Will forward bug reports from users here, so that you can analyze and resolve them.

[lextm]

A table is shared below to compare the features of the two VSCode extensions right now,

	vscode-restructuredtext	esbonio
language server	Y, from esbonio	Y
link editing	Y, from esbonio	Y
section navigation	Y, from vscode-rst	Y, from vscode-rst
syntax highlighting	Y, from vscode-rst	Y, from vscode-rst
basic live preview	Y, partly from esbonio	Y
conf.py selection	Y
no conf.py mode	Y
sync scrolling	Y
show source	Y
linter integration	Y (doc8/rstcheck/rst-lint)
list editing	Y
table editing	Y, from TatsuyaNakamori
section header editing	Y

Nice progress so far.

[ssbarnea]

I cannot state this more: please join efforts and ensure we have only one community maintained RST extension, not 10 of them, all broken in different places and lacking active maintenance. As a python developer that needs to edit RST files, I find the state of Restructured text support in vscode very frustrating with too many broken extensions around, to the level of throwing exceptions.

I happen to also be a vscode extension/lsp developer (Ansible) with an interesting background. Back in 2020, the status of editing Ansible files with vscode was a similar PITA, with more than dozen extensions published and most of them unmaintained and even broken, including an official one published by Microsoft. It took me about one year to work on consolidating the usable bits from these and getting one Ansible vscode extension to rule them all. I had to contact other extension developers and convince them that we are better together and ensure we can just one extension in the marketplace, but one that is very good. Today situation is very different, searching https://marketplace.visualstudio.com/search?term=ansible&target=VSCode&category=All%20categories&sortBy=Relevance you only see 2-3 results and the recommended one is obvious. In January 2021 the same page was returning 15+ Ansible extensions, and the first one was the Microsoft one which did not receive any update for more than 3 years, it was practically abandonware. We contacted Microsoft and they also agreed to take down their extension once the "winner" was showing signs of being an active project.

I mention all of these here because I would like to see something similar happening about RST. Just join efforts and show openness regarding development, avoiding trying to use the extension for personal or corporate publicity.

It will just not work in the long run, especially because maintaining an extension takes a LOT of efforts, something a single person or even company cannot afford to ensure in long term (years!).

vscode marketplace team is already aware of more of less natural spamming of the marketplace and probably they will start to do things to reduce the number of extensions published, or at least boosting the actively maintained ones.

I hope these words help you!

[alcarney]

Thank you for sharing your thoughts, it's definitely useful to have an outside perspective on this

I think the collaboration you are hoping for is already happening - though I admit we can do better.

The primary focus of this repo is the language server, which has (quite recently!) been integrated into the reStructuredText extension and it's highlighting all the scenarios it doesn't cover yet. Unfortunately, this is resulting in a broken mess right now but it should get better once we get past this initial stage.

While the reStructuredText extension is most likely the "winner" in this scenario, I do think there is some value in keeping the Esbonio extension around. Currently it serves as a "reference implementation" for the language server that projects for other editors (like coc-esbonio) can use as a starting point, as well as a safe space to try out new ideas like the new http.server based preview mechanism without breaking too many people's workflows.

Of course, neither of those arguments are anything a normal user actually cares about, so at the very least it needs to be made clearer that people should be going for the reStructuredText extension by default.

I hope that makes sense 🙂