Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Embedded YarnEditor webview custom editor #1

Conversation

TranquilMarmot
Copy link

@TranquilMarmot TranquilMarmot commented May 25, 2020

What is this?

Hello friends! 👋 Thank you for Yarn Spinner; this tool is awesome! 🧶 🐈

A while back I saw this draw.io extension for VSCode and was blown away by the idea of embedding a full JS app inside of an editor... so naturally when I came across the Yarn Editor, the first thing I thought was... "how long would it take to embed this editor in VSCode?!" The answer is... about the length of a long weekend.

This uses a git submodule to include the YarnEditor repo. Currently, this PR has the submodule pointed to my fork/branch of YarnEditor. There is a PR against YarnEditor that is needed for this to work: https://github.com/YarnSpinnerTool/YarnEditor/pull/186

Once that PR is merged, this one can be updated to have the proper submodule that points to the real YarnEditor.

This is in TypeScript. If you've never encountered TypeScript before don't fret! This PR doesn't do anything too crazy with it.

This has been tested on both Windows and Linux.

Also, don't freak out at the +2,000 lines added! Most of that is package-lock.json.

For reference on how a lot of this was done, check out:
https://code.visualstudio.com/api/extension-guides/webview
https://code.visualstudio.com/api/extension-guides/custom-editors
https://github.com/microsoft/vscode-extension-samples/tree/master/custom-editor-sample

Features

  • Open, edit, and save .yarn and .yarn.txt files in your workspace in the Yarn Editor without leaving VSCode (this is killer for Unity projects!!!)
  • Saving is handled by VSCode and it will mark files as dirty when they're changed (so you don't accidentally close them after making changes!)
  • You can also run the "vanilla" editor with a "Start Yarn Editor" command via the command palette (F1 menu). This is basically the equivalent of the Electon app... but in a VSCode extension!
  • Options are configured in VSCode's settings instead of in the editor. When running the "vanilla" editor, it will still show the "File" menu and let you change settings, but they will not persist.
  • Automatically detect when the user has a "dark" VSCode theme and switch to night view. This can be overridden by a special option in the VSCode extension settings.
  • Easily swap between the editor and the syntax-highlighted text view
  • When viewing a diff, VSCode will open two editors side-by-side with the old and new files

Screenshots

(Note: these were all taken before a bug with arrow rendering position was fixed)

Sally.yarn sample, with automatically detected night mode:
vscode-yarn-sally

Sally.yarn sample, with blueprint theme:
vscode-sally-blueprint

VSCode settings for the extension; these map 1:1 to the settings modal and even have the same dropdown options:
vscode-yarn-settings

Switching between the syntax-highlighted text view and the editor view. This can be done by clicking the ... menu button on the VSCode tab and choosing "Reopen With..."
vscode-yarn-switch

StressTest.yarn open alongside another file; stress test opens surprisingly quickly!
vscode-yarn-stress

How to try this out

Checkout this branch and then run git submodule init && git submodule update. This should pull down the proper branch of YarnEditor.

Then, run npm install && npm run yarneditor:build. This will build YarnEditor and copy it to out/dist (the extension's code is transpiled to JS and copied to out as well when it runs).

You can then hit F5 to run the extension. It will open a new VSCode window with the extension loaded. From there, try opening a .yarn file or hitting F1 to run the "Start Yarn Editor" command.

Notes

The amount of HTML string manipulation in here doesn't feel great, but honestly it's not all that bad and YarnEditor's layout and HTML made it easy.

Icons

Originally, I had this extension include an "icon set" but it seems like VSCode can only have one icon set active at a time right now. If we want the yarn icon to properly appear for .yarn files it seems like a PR would need to be opened against https://github.com/hellopao/vscode-seti-icons

This issue against VSCode tracks the ability to have extension provide just one icon: microsoft/vscode#14662

Undo/redo

I couldn't manage to get native undo/redo working for changes. It seems like the editor would need to be reworked to only track specific lines in the underlying file changing to get this to work properly. Undoing/redoing adding/removing nodes still works within the editor, though.

Lint

Okay, for some people lint is a touchy subject. Currently, this is just using the most basic lint rules that work with TypeScript and Prettier.

If you want, we can make this obey the same lint rules as YarnEditor and we can add a prettierrc.json to make Prettier do the right things.

.gitmodules Outdated
@@ -0,0 +1,4 @@
[submodule "YarnEditor"]
path = YarnEditor
url = https://github.com/TranquilMarmot/YarnEditor.git
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will need to be updated once https://github.com/YarnSpinnerTool/YarnEditor/pull/186 is merged

"theme": "light"
},
"engines": {
"vscode": "^1.44.0"
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This had to be bumped up from 1.38.0 since 1.44.0 was the most recent version to fully support custom editors (this is some cutting edge stuff! 😎 )

"displayName": "Yarn Editor",
"selector": [
{
"filenamePattern": "*.yarn"
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I considered also adding .json, .twee, and all the other supported formats with "priority": "optional" here but I'm not sure how supported those are in the editor.

"path": "./yarn-icon-theme.json"
}
],
"configuration": {
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All of these configuration settings were ripped straight from settings.js. This makes it so that you can change settings in your VSCode preferences!

@daviddq
Copy link

daviddq commented May 25, 2020

Super cool! 👏👏👏👏👏👏👏

@TranquilMarmot
Copy link
Author

Okay, I think this is finished! Got everything working smoothly, including settings and marking files as dirty and letting VSCode handle saving/loading.

Here's a copy of the extension built with vsce:
yarn-spinner-1.2.0-2b4e5bf.zip

GitHub wouldn't let me upload it as a .vsix file, so to install it:

  • Download and change the extension to .vsix (this is just a .zip file, so if you want you can open it and look at what's inside first)
  • Run code --install-extension yarn-spinner-1.2.0-2b4e5bf.vsix
  • Restart VSCode if you have it running
  • Open a .yarn file!

@blurymind
Copy link

Thanks for working on this! It does look very cool 👍👌
I will try to do a review and get it merged asap

@blurymind
Copy link

correct me if I am wrong, but once merged, this will become available via
https://marketplace.visualstudio.com/items?itemName=SecretLab.yarn-spinner

I will review the changes required on the yarnEditor side of things. Somebody else needs to review it on the yarnSpinner side :)

I am pretty excited to see this in vscode. Very very cool!

@blurymind
Copy link

blurymind commented May 27, 2020

Ok I tested this and found some problems on my end. Not sure if they need addressing at this pr or the other. Noted them here
https://github.com/YarnSpinnerTool/YarnEditor/pull/186#issuecomment-634651197

@desplesda if this gets in yarnSpinnerTool, bondagejs is also going to become a part of yarnSpinnerTool - making it even more important to add the new missing syntax features to it. Playtesting inside yarnEditor, inside vscode is done using bondagejs - not yarnSpinner

We are starting to have more than one js technologies requiring yarn syntax parsing to be done with a js api - javascript needs to be a first class citizen in terms of yarn syntax support imo - this is where all the debugging will be

@blurymind
Copy link

blurymind commented May 27, 2020

There are a bunch of recent regressions in yarnEditor that need urgent fixing. Will this extension always use the latest version of yarnEditor? If not, can we update it to use a more recent version - after we fix the recent regressions? Would be good to update it before this pr is approved

@TranquilMarmot
Copy link
Author

TranquilMarmot commented May 30, 2020

Alright, here's what I ended up with RE: Editing nodes in VSCode's text editor!

Here's another .vsix file if you want to try it out:
yarn-spinner-1.2.0-0ed46c5.zip

By default, nodes still open in ace but there's now a button to edit them in VSCode (this doesn't show up in the web app; I checked 😉 )
image

Clicking this button will create a new, temporary file and open it up. At the same time, it adds a node.js file watcher to it that listens for changes; when the file changes, the extension reads it and sends it back to the editor which will then update the node. This lets you edit node titles, tags, and bodies (color and position can still be changed from the yarn editor):
image

This ends up working surprisingly well! If you open up two nodes, you actually get two temporary files; the extension is listening to both of them and changes made to either are sent back to the editor. This means you can do cool things like editing two nodes side-by-side!
image

The extension will automatically delete any temporary files when it "deactivates" (i.e. when the window is closed).

On top of that, I added an "Always Open Nodes In Visual Studio Code Editor" option that will make it so that double-clicking on a node will always open it in the VSCode text editor.

image


Main issue with this is that if the main yarn file changes outside the editor, then these temporary files will go out-of-sync.

This is also an issue if you have a .yarn file open and it changes outside of the editor (i.e. if you revert it with git)... the yarn editor won't pick up on the changes unless you close and re-open it. I tried to have the extension listen for changes to the file, but right now it only deals with updating the whole yarn file so it was refreshing the editor with every update (dragging nodes became impossible).

A future improvement to this (not doing it in these PRs because it would require reworking a significant part of YarnEditor) would be to have more specific messages being sent back and forth between the extension and the editor. For example, when a node changes, rather than sending the whole yarn file, just send something like { type: "NodeUpdated", node: { title: "Whatever", body: "Some text" }} back to the extension.

This would also require having the extension parse out the .yarn file so it can keep track of nodes internally and learning how to write them back out. Basically, re-implementing most of data.js inside the extension and having the extension track the state more tightly.

@TranquilMarmot
Copy link
Author

Okay, fixed a slew of bugs with the "Edit in Visual Studio Code Text Editor" functionality and updated the YarnEditor branch to be in line with master.

Another .vsix: yarn-spinner-1.2.0-e711fbb.zip

And another really cool screenshot of editing two nodes at once 😄
image
As soon as you save one of the temporary files, the updates to the .yarn file are pretty much instant! It feels great.

@desplesda
Copy link
Contributor

What's needed before we can merge this PR?

@blurymind
Copy link

a bit more testing - see if you find things not working as expected :)
Also the other (yarnEditor) pr to get merged

@TranquilMarmot
Copy link
Author

Updated this PR and the other one to have the latest YarnEditor; here's another .vsix: yarn-spinner-1.2.0-aa30f93.zip

@desplesda I'd say this is done. Would definitely love folks to do some more testing! Once https://github.com/YarnSpinnerTool/YarnEditor/pull/186 is merged I can update this PR to have its submodule point to the right repo.

@blurymind
Copy link

The changes required on the yarnEditor side for this to work have now been marged to the master branch
👍

@TranquilMarmot
Copy link
Author

TranquilMarmot commented Jun 11, 2020

Okay! The submodule in this has been updated to point to the real YarnEditor which has the needed changes merged in. This should also be good to go now.

@daviddq
Copy link

daviddq commented Jun 24, 2020

What's missing for an official release of the extension? I'm excited to get the latest features and fixes by @TranquilMarmot 😄

@blurymind
Copy link

I think this can be merged now, but waiting for someone on this repo to review and merge it :)

@desplesda
Copy link
Contributor

I'll review this PR early this coming week! Excited to get this merged.

@TranquilMarmot
Copy link
Author

Hey all, as good as this PR works with Yarn Editor, there were a few things that didn't quite sit right with me:

  • No shared code between the editor and the extension; the extension ends up re-implementing a lot of logic from the editor
  • The whole git submodule thing is quite painful to deal with (this PR is already horribly out of date 😅 )
  • The state management between the editor and the extension isn't great
    • Currently, Yarn Editor has to send all nodes to the extension with every change
    • The main issue with this is that it breaks down VSCode's concept of changes and its undo/redo functionality
  • The hacks that had to be done to get the editor to embed in the VSCode webview are pretty nasty and are brittle. They are likely to break with future changes to Yarn Editor.
  • Ace editor is a large dependency that is not needed in VSCode
  • The Yarn Editor embedded in VSCode has stylistic clashes (colors, icons, overall design, etc.)
  • No tests in the current Yarn Editor made it difficult to know if I broke anything when making changes to it 🙈

All of the above inspired me to spend the past month putting together YarnLoom, a more native and tightly-integrated version of the Yarn Editor that plays nicely with VSCode:

https://github.com/tranquilmarmot/yarnloom
https://marketplace.visualstudio.com/items?itemName=TranquilMarmot.yarn-spinner-loom

Any .yarn files made with Yarn Editor work with YarnLoom and vice versa! It uses the same mechanisms for node colors, tags, and placement.

Key differences:

  • Common library of functions used for parsing/generating yarn files that are shared between the editor and the extension
  • The editor sends much more specific messages back to the extension so that it can track changes for better undo/redo functionality
    • This means node renames, additions, color changes, edits, and everything else support native undo/redo
    • See the YarnLoom CONTRIBUTING.md doc for more info
  • Uses VSCode supplied CSS variables for colors so that it matches the user's VSCode theme
  • Uses codicons for icons so they match VSCode
  • Additional features
    • More tag management tools
    • Renaming nodes also renames links to those nodes
    • Option for case sensitivity and regex when searching
    • Better feedback around renaming/adding nodes with names that already exist
    • Uses D3 for drawing node relationships, resulting in much smoother rendering
  • Missing features (these may be added later)
    • Preview using bondage.js
    • Selecting and manipulating multiple nodes at once
    • BBCode support
    • JSON/XML/twee/twee2 import/export support (only .yarn files are supported)
  • Lots and lots of tests 😄
  • 100% TypeScript 😉

You're welcome to merge this and keep it!

I'll be happy to answer any questions you have but will probably be putting more effort into maintaining YarnLoom.

@blurymind
Copy link

blurymind commented Aug 1, 2020 via email

@desplesda
Copy link
Contributor

This is really nicely done, and I especially love your remix of Cecile’s logo!

Something I’d like to look at, sooner rather than later, is simplifying the Yarn editor ecosystem. Currently, there’s Yarn Editor (both as a web app and as a desktop app), this repo’s VS Code extension, the current VS Code extension, and in Unity there’s the Merino editor. Is consolidating any of these projects something you (and potentially other authors) would be interested in?

@daviddq
Copy link

daviddq commented Aug 5, 2020

Super cool! I love the extension. Thanks a lot for your great work 👏 👏 👏 👏

@TranquilMarmot TranquilMarmot deleted the tranquilmarmot/add-webview branch October 19, 2020 17:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants