Graviton Rewrite 📝

[marc2332]

Graviton Rewrite

The main idea behind this possible rewrite is to have different re-usable modules which can be used to build different types of Graviton Instances. I think that it will highly improve the scalability, maintainability and make it easier for people to contribute to. Also, I need to make an effort to make a better documentation.

There will not be any more NodeJS (except for developing purposes such as using NPM) or ElectronJS.
Instead, there will be a backend written in Rust and then frontends that will communicate to the backend. These frontends could be written in any language that supports HTTP requests and WebSockets connections.

The modules would be something like:

• Graviton Core Library: the backend that would ultimately be used to access the filesystem, run processes, etc...
• Graviton Core CLI: A cli to easily run a Graviton Core as a service
• Graviton Web: A web-based frontend implementation that would connect to a Graviton Core
• Graviton Desktop: Graviton Core Library + Graviton Web, a desktop version that would ship the backend and the web frontend through for example a webview (tauri).
• Graviton Server: A service that would run a Graviton Core and host a Graviton Web in an HTTP server

Graviton Core

Graviton core will be written in Rust for speed and safety. It will have a JSON RPC HTTP server for normal method calls and a WebSockets server to emit events from the core to the connected frontends.

It will also persist all the frontend states, so they could reconnect and still have the previous session as it was.

Example situation

There could be a Graviton Core service running in a server and multiple frontends that could connect to it as if it was in the same machine. This would be like connecting to an external machine but with zero overhead, because that's how it already works.

Piece table

It would be great to have a Piece Table system where changes could be tracked and then when saving a file from a frontend, instead of sending the whole new file to the core, the changes from the piece table would be sent. And then, in the Core, properly apply the received piece table to the original file. This will make it faster when communicating the Core and the frontend with large files.

More about Piece table:

• https://gaetgu.github.io/org/2021-11-03-Text%20Editors:%20Part%20One.html by @gaetgu
• https://darrenburns.net/posts/piece-table/

Graviton Web

This will be written in React ;)

Graviton Extensions

All extensions would be installed in the Graviton Core and loaded in the corresponding frontend state in the backend. Extensions would make use of an API that has the ability to create widgets / components that they are then interpreted into the corresponding frontend. So there is no need to every frontend load the extension itself, and therefore, it only needs to support the API.

Feel free to propose new ideas or improve whatever I said !

[LucasCtrl]

Graviton Web
This will be written in React ;)

Just a quick note for those who are starting to pull their hair out: Graviton Web with React is our vision for Graviton.
Feel free to take the core and build the web part with the framework you like or if you are a cracker, in vanilla JS 😉.

Same with Tauri for the webview

[marc2332]

Yeah! The idea is that the core is frontend-agnostic, meaning that although I would like to maintain a web-based frontend, anybody could create their own frontend version with the tools and languages of preference :D

[ghost]

I wouldn't use HTTP & WS except when working with Graviton Web. IPC is much more optimized and integrated.

[marc2332]

Instead of killing the first graviton core when the first window is closed, why not have the core check if any windows are depending on it and only close it when that number is 0.

Yeah, thats one approach too

[ghost]

Instead of killing the first graviton core when the first window is closed, why not have the core check if any windows are depending on it and only close it when that number is 0.

No, whoever who wants to use the original Graviton will have a branch that he can push code to. At least that's what I think.

Maintaining 2 gravitons would be hard and redundant. We should only maintain the new one.

[ghost]

I am not sure about that. But, the desktop version could maybe use a direct connection to the Core instead of relying on HTTP and WS.

• Equivalent of HTTP -> jsonrpc_core_client::transports::local (https://docs.rs/jsonrpc-core-client/18.0.0/jsonrpc_core_client/transports/local/index.html)
• Equivalent of WS-> std::sync::mpsc (https://doc.rust-lang.org/std/sync/mpsc/)

But using this direct connection will basically prevent an idea I thought about:

That's what IPC is meant to do.

Let's suppose you have 1 Graviton Desktop opened, this will launch a Graviton Core and a Graviton Webview. Now, you open a second Graviton Desktop, my idea is to reuse the already existing Graviton Core, instead of creating a new one. And when the first window get killed, and so the Graviton Core, then the second Graviton Desktop would create it's own Graviton Core.

This was just a random thought I had.

I'd launch two cores on two different processes, as having multiple graviton windows would make the Core slower. Some stuff like updating Graviton could be shared, but e.g. file processing, text highlighting and LSP should be separate. This wouldn't only be easier to maintain but faster.

All extensions would be installed in the Graviton Core and loaded in the corresponding frontend state in the backend. Extensions would make use of an API that has the ability to create widgets / components that they are then interpreted into the corresponding frontend. So there is no need to every frontend load the extension itself, and therefore, it only needs to support the API.

Extensions could also be shared and splitter into processes at the same time, with a special API. What do you think? Some tasks are specific to a window and some are global. It'd also be easier to have them separated, but some would be faster by being shared.

[marc2332]

Instead of killing the first graviton core when the first window is closed, why not have the core check if any windows are depending on it and only close it when that number is 0.

No, whoever who wants to use the original Graviton will have a branch that he can push code to. At least that's what I think.

Maintaining 2 gravitons would be hard and redundant. We should only maintain the new one.

That is not what @gaetgu is talking about. But of course, the old graviton will be left intact in a different branch.

[ghost]

Instead of killing the first graviton core when the first window is closed, why not have the core check if any windows are depending on it and only close it when that number is 0.

No, whoever who wants to use the original Graviton will have a branch that he can push code to. At least that's what I think.
Maintaining 2 gravitons would be hard and redundant. We should only maintain the new one.

That is not what @gaetgu is talking about. But of course, the old graviton will be left intact in a different branch.

Oh. I misread. Yes, his idea is ok.

[gaetgu]

Looks great! I would be happy to help with the graviton core library, especially with the text handling (piece table implementation, file systems, etc.)

[ghost]

I can help with the Build system / CI, packaging and bundling related-stuff!

[marc2332]

Besides the editor itself, it would be also great to update the dependencies in https://github.com/Graviton-Code-Editor/website, such as updating NextJS v10 to v12.
I will probably remove the built-in documentation and replace it with a new version, probably using something like Mdbook, rustdoc and tsdoc.

[LucasCtrl]

Is the website necessary if we have a doc?
We can use the home page of the doc to introduce the application.
If we build a nice looking theme it can be enough (like the introduction page on mdBook doc).

[LucasCtrl]

I will probably remove the built-in documentation and replace it with a new version, probably using something like Mdbook, rustdoc and tsdoc.

If you use Rust for Graviton core, I would go for mdBook

[ghost]

Is the website necessary if we have a doc? We can use the home page of the doc to introduce the application. If we build a nice looking theme it can be enough (like the introduction page on mdBook doc).

Yes of course. The website would have Graviton Web too.

Besides the editor itself, it would be also great to update the dependencies in https://github.com/Graviton-Code-Editor/website, such as updating NextJS v10 to v12.
I will probably remove the built-in documentation and replace it with a new version, probably using something like Mdbook, rustdoc and tsdoc.

These docs already work and are integrated into the website, so why change it?

[marc2332]

Because they are very bad, and rustdoc and tsdoc are automated.

[marc2332]

Just merged the changes from the rewrite into the main branch.

[marc2332]

I am seriously thinking about making the Core crate, transport-agnostic, meaning that instead of opening up a HTTP JSON RPC and a WebSockets server itself, it will provide an interface in which to implement a transport. In that case, I would provide two optional implementations by default. On a "local" one, meaning that the communication is done through the same Rust process, just like https://docs.rs/jsonrpc-core-client/18.0.0/jsonrpc_core_client/transports/local/index.html . And second, something like what it is right now, an HTTP JSON RPC and WebSockets.

And finally, make the desktop app use the "local" transport. This will mean more security and speed, but also not possible to do the idea of reusing the same core for multiple instances.

[marc2332]

I am seriously thinking about making the Core crate, transport-agnostic, meaning that instead of opening up a HTTP JSON RPC and a WebSockets server itself, it will provide an interface in which to implement a transport. In that case, I would provide two optional implementations by default. On a "local" one, meaning that the communication is done through the same Rust process, just like https://docs.rs/jsonrpc-core-client/18.0.0/jsonrpc_core_client/transports/local/index.html . And second, something like what it is right now, an HTTP JSON RPC and WebSockets.

And finally, make the desktop app use the "local" transport. This will mean more security and speed, but also not possible to do the idea of reusing the same core for multiple instances.

The core is now transport-agnostic, and the desktop version uses a Local implementation instead of over HTTP

[franz-fletcher]

How did you get this working with Tauri? Pls share, Tauri has stonewalled the call for a local IPC implementation for passing data between the UI and Native threads. It's such a pain that a lot of energy is spent on their convoluted FFI architecture and every time someone brings up a genuine use case for low latency data transport for their apps it usually ends with a lecture about their philosophy before they close the ticket 😕

[marc2332]

How did you get this working with Tauri? Pls share, Tauri has stonewalled the call for a local IPC implementation for passing data between the UI and Native threads. It's such a pain that a lot of energy is spent on their convoluted FFI architecture and every time someone brings up a genuine use case for low latency data transport for their apps it usually ends with a lecture about their philosophy before they close the ticket 😕

hi, I think I might have not explained it well but I am still relying on Tauri's IPC. Since Graviton also runs on the browser it needs to have some abstraction over the core communication, therefore, the desktop version uses Tauri's IPC and when running in the browser it connects to a HTTP JSON RPC core running remotely. But anyway, as I said, in the desktop version, which runs on Tauri, relies on it's default IPC. Maybe you could try https://github.com/tauri-apps/tauri-invoke-http ?

[ghost]

Great idea!