Editable TextDocumentContentProvider for extensions

[anorborg]

I'm just getting started with extension development so please forgive me if this is a inappropriate feature request. I'm looking to create an extension that with allow me to select a block of code in a source language (i.e. html) and have the option to edit the block in a different language (i.e. jade/pug). I would imagine it would behave similarly to the TextDocumentContentProvider, but have the ability to edit in the secondary window, then transpile back to the source when completed. I didn't see anything in the API documents that would allow me to do this now. Thanks.

[jrieken]

You can do that by adding the onDidChange event to your provider. A sample is this: https://github.com/Microsoft/vscode-extension-samples/blob/master/previewhtml-sample/src/extension.ts#L13

[anorborg]

Apologies if I'm misunderstanding @jrieken, but doesn't that refer to changes in the source document? I'm looking to allow editing in secondary (in the example you gave me, preview window). The TextDocumentContentProvider documentation seems to explicitly state "readonly documents".

[jrieken]

but doesn't that refer to changes in the source document?

Yes, but if you make the document 'render' using the previewHtml command it will also update

[anorborg]

I don't think I'm being clear, so again I apologize. Let me try to explain the use case I'm trying to accomplish in detail. If I have a source document in html, source.html, I want to highlight a code block, for example:

<div class="form-group">
    <input class="form-control" id="oldId" value="test" />
</div>

I then want to be able to execute a command like "Edit As Pug/Jade" which will open a secondary editor window that will automatically transpile to something like the following:

.form-group
    input.form-control#oldId(value="test")

In that secondary editor window I would update the pug/jade code to something like the following (updating the id field):

.form-group
    input.form-control#newId(value="test")

I then would execute another command indicating I'm done editing that would close the secondary editor window and the source document, source.html, would be update based off the changes made in the Pug/Jade to the following HTML:

<div class="form-group">
    <input class="form-control" id="newId" value="test" />
</div>

Notice that the id field was changed based on the changes made in the secondary editor window.

Again I do apologize if I'm missing your explanation, but based on the code and the example you've pointed to, it seems I hadn't quite explained what I was trying to accomplish correctly.

[jrieken]

Ah, thanks now I get it. We don't support that users make changes in virtual documents and I am also unsure what that would mean... You could work around this by making a 'untitled' to 'temp' document somewhere

[eamodio]

@jrieken can this be reopened as a feature request? Supporting editable virtual documents could be pretty useful in certain scenarios.

For example: https://github.com/pprice/vscode-better-merge/issues/25

Quick summary -- using editable virtual documents to compare changes and (manually) sync the edits back to the main document.

An alternative request could be supporting virtual documents that are actually "windows" (a slice) of a real document. But that sounds much more complicated and less broadly applicable.

[jrieken]

Ok, valid request after all. Unsure yet how the API will look like, maybe just offering a save method on your provider is enough. That would make it editable and we will call that method when going through our 'normal' save.

[eamodio]

Yeah -- maybe something like the following:

onDidEdit(document: TextDocument, contentChanges: TextDocumentContentChangeEvent[]): void;
onWillSave(document: TextDocument, reason: TextDocumentSaveReason): Thenable<TextEdit[] | void>;

While having document: TextDocument on these methods is a little strange it could provide nice access to the document object to look at edits, etc.

Utilizing the onDid|onWill structure here probably isn't right as these are "callbacks" not events.

Maybe:

edited(document: TextDocument, contentChanges: TextDocumentContentChangeEvent[]): void;
saving(document: TextDocument, reason: TextDocumentSaveReason): Thenable<TextEdit[] | void>;

Or just:

edit(document: TextDocument, contentChanges: TextDocumentContentChangeEvent[]): void;
save(document: TextDocument, reason: TextDocumentSaveReason): Thenable<TextEdit[] | void>;

Anyway my 2c :)

[jrieken]

Well, the edit and save event already exist globally and will also fire for virtual documents. It more that some needs to store the underlying string. So, it'll be more something like

// ...
saveTextDocumentContent(uri: Uri, content: string): Thenable

[eamodio]

That makes sense, but maybe it is worth something like:

willSaveTextDocumentContent(uri: Uri, reason: TextDocumentSaveReason): Thenable<TextEdit[] | void>;

That would happen before any global events, so that the provider could get the first crack at making any edits. While you can kind of do this with the global event, there is no guaranteed order -- and it seems like it would be nice to give the provider the first say in the edits (so that the global events could reflect them).

And imo that works well with saveTextDocumentContent giving the provider the final say as well.

Also the only other thing that needs to be added is a way to express that the virtual document should not be read-only.

Maybe

provideTextDocumentContent(uri: Uri): Promise<string>

Becomes

provideTextDocumentContent(uri: Uri): Promise<string | { content: string, readonly: boolean }>

?

[jrieken]

That would happen before any global events, so that the provider could get the first crack at making any edits.

Not sure I have understood. The edits have already happened at this point and it's just about persisting them now. Are you suggesting to have special pre-save hook API for further modifications?

[eamodio]

Kind of -- basically just a way to make sure the provider can get at something like the global onWillSaveTextDocument: Event<TextDocumentWillSaveEvent> without using the global event to guarantee order. Though I may just be over complicating it ;)

[jrieken]

@bpasero Is there a generic ways for this: options.readOnly = !(this.input instanceof UntitledEditorInput);? I now need to make some resource editor input not readonly but I don't wanna hard-code a list of schemas etc

[eamodio]

Not totally related -- but it would be sweet if there was a way to even open a real file opened as read-only. So I don't know if that would influence things.

Also not really related, but since this likely requires decent changes to TextDocumentContentProvider -- it would be great if the encoding detection worked for it just like files (and be able to switch the encoding like a file)

[bpasero]

@jrieken currently not.

There is a lot more to making any resource participate in dirty/save lifecycle of the workbench properly beyond making the editor actually writeable by the user. The reason for that is that so far only file:// and untitled:// supported saving and this is not implemented in a generic way based on the resource scheme but rather there are 2 registries of models (ITextFileEditorModelManager and IUntitledEditorService) around that are being used in various places to:

• show dirty indicators throughout the workbench outside the editor (the only dirty indicator that comes for free is the one for the editor tab based on the EditorInput.isDirty() implementation)
• backup dirty models when they change for hot-exit
• implement the Save, Save All and Save As, Revert actions
• implement encoding switcher

Having another service where we can provide these features for any resource (contributed by extensions) would probably be needed. This service could then also be used to to opt-in for an editable editor based on the resource that is opened.

I can look into adopting this service once it is there to hook it into the various places of workbench land. Some methods needed for the features mentioned (motivated by IUntitledEditorService):

• events when model gets dirty (for dirty state indicators)
• events for when model changes content (for hot exit backup)
• events for encoding (if we ever want to expose that)
• total dirty count for all models or a specific model

[jrieken]

@bpasero fyi I have made first steps in joh/save with this commit: 65a59ae. Let me know if that will also work for things like untitled-resources. I think it would be nice to first migrate that onto the new infrastructure and have it validated and then do the new API

[bpasero]

@jrieken that is a good idea, I will have a look to see what is missing.

[bpasero]

@jrieken I ended up not adopting this for untitled because that would have ended up in a debt-minefield and we are not in debt week. Instead I pushed some changes to joh/save to enable a proof of concept that enables:

• editor is editable when opening a supported resource
• editor shows dirty when content gets dirty
• save and revert work
• closing the editor asks for confirm

To try it out, I pushed a text model content provider (F1 > Open Extensions Resource) that also registers a text model saver.

This is far from being done and there is lots of work still left in the workbench to make this usable in the same way it is working for files. I discovered some areas while working on the proof:

• make our dirty indicators outside the editor fit for this
• support auto save (or not?)
• support hot exit
• support save participants
• support to set the encoding for saving
• support undo properly (undo enough to get back to saved state)
• support to search through resources (today we only search file & untitled)
• revisit our hardcoded assumptions in places like actions that only think file & untitled resources are editable
• debt: make the confirmation dialog reusable
• debt: reuse more code between file saving and resource saving

I am a bit in a conflict here with multi root priorities, will talk to @egamma. We have to decide if we want to ship a MVP that does not provide the same functionality as files/untitled (e.g. no hot exit for now) or actually invest in identical experiences.

[jrieken]

There is too many open ends and issues for making this the light variant for supporting remove dev scenarios. I move this issue off the plan and let's continue the discussion about save et in #29194.

@anorborg I feel like we hijacked this issue and kept talking about saving where your request is just about having virtual documents editable. Am I correct assuming there is difference? E.g you just wanna open an writeable (scratch) editor in which can by typed but which not necessarily must be persisted/saved onto some storage?

[letdowncrush]

@jrieken I'm looking for something along those lines.

My scenario would work as follows:

• Launch virtual document from active text editor
• Edit content in the virtual document
• Sync content back to the document in the text editor

The only purpose of the content held in the virtual document would be to sync back to the real document - persistence in my case would be undesirable.

[ejizba]

We would like this as well. As discussed in this issue here's our scenario:

1. Download an Azure resource (storage blob, Cosmos DB document, etc.)
2. Open resource in virtual document
3. Edit virtual document
4. Upload changes back to Azure

We don't want to clutter the user's file system with these resources, and that's why virtual documents would work great for us.

[leonidaspir]

We would very much like to have a way to push back changes from a virtual document to the real document.

Persistence isn't necessary, just one to one reflection.

[markvincze]

I would need this too for providing an edit functionality in my Code Fragments extension (https://github.com/markvincze/vscode-codeFragments).

Is there any alternative to do something like this today?

[jrieken]

Is there any alternative to do something like this today?

Snippets?

[markvincze]

@jrieken I'm not sure yet how snippets can be used for this purpose.
Do you have an example where this was done?

[jrieken]

@markvincze That might need some more API but your extensions reminds me of snippets, e.g. the F1 > Insert Snippet command plus #26706. Kudos. We can think of making snippets more discoverable (so that you can populate a tree from them) and to make them more dynamic (preserving their static nature, e.g. find something between snippets from disk and snippets computed by completion item providers).

Then, and that's some completely different, and potentially interesting for everyone here is the FileSystemProvider-API that we will ship in April: #47475. It will allow you do to whatever a file system does and you can integration into the editor that way. Like the MemFS-sample: https://marketplace.visualstudio.com/items?itemName=jrieken.vscode-memfs

[iminside]

Hello! I also need a similar functionality, but I see a slightly different solution - callback

workspace.onWillOpenTextDocument((document: TextDocument) => document : TextDocument)

callback with the possibility of substituting the original document with a new document.

I could do it like this

workspace.onDidOpenTextDocument(document => {
    if (document.languageId === 'mylang') {
        const textEdit = new TextEdit(new Range(0, 0, document.lineCount, document.eol), 'new text')
        const edit = new WorkspaceEdit()
        edit.set(document.uri, [textEdit])
        workspace.applyEdit(edit)
    }
})

but in this case the document is immediately marked as "changed" (which, however, is logical) and closing it leads to a call to the save dialog, which I do not need.

In general, I want to replace the original text of the document when opening the file, so that it can be edited as a regular file, and when saving, convert it back to the format I need.

I watched the FileSystemProvider but I did not understand how it can help me solve my problem.

[jrieken]

In general, I want to replace the original text of the document when opening the file, so that it can be edited as a regular file, and when saving, convert it back to the format I need.

Please check-out the support for file system providers that we have added for 1.23. They will allow you to do exactly that: https://code.visualstudio.com/updates/v1_23#_filesystem-providers

[brainz80]

@jrieken & @bpasero Could you perhaps while working on this see if you also could also grapple this request #824 ? I feel they're partly related.

[student020341]

Hello from 2019! Google takes me to this issue or other issues that reference this one when searching for a way to edit the results from a TextDocumentContentProvider / edit a virtual document. I thought I'd provide my use case for it here instead of opening a new issue.

I'm implementing a TreeDataProvider for an abstract remote/api based system. I use ssh2 to fetch a directory listing and create a collection of nodes for the tree data made with vscode.window.createTreeView. That's working so my next step is to open, edit, and save these remote documents (which may or may not actually exist on the remote end).

I implemented a very simple TextDocumentContentProvider that throws some "test!" text back from provideTextDocumentContent and create a remoteNonsense URI scheme to serve it with. Looks great! Except I can't edit it.

So, my desire to do this in vs code is that I would like to utilize other existing extensions, autocomplete for when certain file extensions are involved (like json formatting or js autocomplete), and the relatively light weight editor with lots of useful keybinds and commands.

So to answer the question posed in 2016/2017 regarding what should happen when a user saves a virtual document, I would appreciate an onSave hook that allows me to make the necessary api call and return a promise to determine whether or not the update was successful. So my provider reaches out to an api or other abstract back end that gives me a virtual document, I can make change to it, and submit the modified document back.

There is an extension I am aware of that uses workspaces and creates an "sftp filesystem" for remote editing, but my use case is a little different. In fact, it would probably be best suited to its own application, but I managed to get all of the pieces I need for this to work together except this final crucial step.

So, is this still being considered for a feature?

[jrieken]

@student020341 Please read #10547 (comment). That's exactly what you want, using a custom tree for ssh isn't what you should do.

[student020341]

@jrieken Ah, at a glance, that does seem to be what I'm looking for, thanks! I reached this point by searching first for how to make a tree since I laid out what pieces I needed for this to work. But it seems making a custom file system provider does all of the pieces I need together!

Thanks for the fast response. So, is this issue still open because editable virtual documents are being considered? I do have 1 other idea for their use case:

I think it would be useful to be able to select a section of a document and change the language mode for just that selection. Editing that in its own temporary window would be useful so the original document didn't have to freak out at other parts of the file about the invalid syntax going on (or other potential multiple language mode bugs) for that other language mode. The simplest case that comes to mind would be react where the boiler-plate repo has global styles defined in javascript within a template string. And in vanilla js, a lot of examples for templating without a framework have html inside template strings as well. I think it would be super useful to be able to highlight that content and set HTML as the language mode to utilize extensions and vscode's highlighting.

I took a look at the language highlight/grammar files and saw the difference between something like a js file and embedded js, so I also suspect having an isolated virtual document of just the 1 language would make this easier to achieve.

[hediet]

Please check-out the support for file system providers that we have added for 1.23. They will allow you to do exactly that: https://code.visualstudio.com/updates/v1_23#_filesystem-providers

For the scenario described in this issue, would it be sufficient to throw for all the non supported operations in the file system provider (like createDirectory, delete, rename - they are all not required for the original issue)?

[jrieken]

Yeah, implementing stat, read, and write should be enough (tho I didn't try this)

[xpe]

Hello, I'm late to the party. I read over the comments above. I'm here because I would like to write an extension that uses a TextDocumentContentProvider to edit a (virtual) concatenation of files in a directory. In case that doesn't make sense, here would the virtual document I'd like to edit:

=====[ file_1.markdown ]=====
# File 1
This is the first file.
=====[ file_2.markdown ]=====
# File 2
This is the second file.
=====[ file_3.markdown ]=====
# File 3
This is the third file.

Going way back in time, IIRC, the first time I saw this kind of behavior might have been with Scrivener, a macOS program for writers. It was a game changer.

Would some people in the know summarize 'where we are' (i.e. what may have changed in VS Code related to this thread) to help me understand the feasibility of working on such an extension.

[gjsjohnmurray]

@xpe I think the thread you just opened at microsoft/vscode-discussions#855 is a better place to continue this.