Improve documentation on passing objects across RPC

doc/Plugin-API.md

@@ -66,12 +66,20 @@ To communicate with each other, the implementation of each side of the API - `Ma
 The proxy is based on the interface of the other side: `Main` implementation has a proxy of the `Ext` interface and vice versa.
 The implementations do not have explicit dependencies to each other.
 
-Communication via RPC only supports transferring plain JSON objects: Only pure DTO objects without any functions can be transmitted.
-Consequently, objects with functions need to be cached and references to such objects need to be transmitted as handles (ids) that are resolved to the original object later on to invoke functions.
+### Encoding and Decoding RPC Messages
 
-For instance, in [LanguagesExtImpl](https://github.com/eclipse-theia/theia/blob/master/packages/plugin-ext/src/plugin/languages.ts)#registerCodeActionsProvider a new code action provider is cached on the `Ext` side and then registered on the `Main` side via its handle.
-When the code action provider’s methods are later invoked on the `Main` side (e.g. in [LanguagesMainImpl](https://github.com/eclipse-theia/theia/blob/master/packages/plugin-ext/src/main/browser/languages-main.ts)#provideCodeActions), it calls the `Ext` side with this handle.
-The `Ext` side then gets the cached object, executes appropriate functions and returns the results back to the `Main` side (e.g. in [LanguagesExtImpl](https://github.com/eclipse-theia/theia/blob/master/packages/plugin-ext/src/plugin/languages.ts)#$provideCodeActions).
+The communication between each side of the API is goverend by proxies that use a `RpcProtocol` on a given channel to transmit RPC messages such as requests and notifications.
+In Theia, the encoding and decoding process of RPC messages can be customized through dedicated `RpcMessageEncoder` and `RpcMessageDecoder` classes that can be provided when a new RpcProtocol is created.
+The `RpcMessageEncoder` writes RPC messages to a buffer whereas the `RpcMessageDecoder` parses a binary message from a buffer into a RPC message.
+
+By default, Theia uses an encoder and decoder based on [msgpackr](https://www.npmjs.com/package/msgpackr) that already properly handles many JavaScript built-in types, such as arrays and maps.
+We can separately extend the encoding and decoding of our own classes by installing extensions using the `MsgPackExtensionManager` singleton, accessible from both ends of the channel.
+Examples of this can be found in the [extension for Errors](https://github.com/eclipse-theia/theia/blob/72421be24d0461f811a39324579913e91056d7c4/packages/core/src/common/message-rpc/rpc-message-encoder.ts#L171) and the [extension for URI, Range and other classes](https://github.com/eclipse-theia/theia/blob/72421be24d0461f811a39324579913e91056d7c4/packages/plugin-ext/src/common/rpc-protocol.ts#L223).
+We call the registration of these extensions in `index.ts` to ensure they are available early on.
+Please note that msgpackr [always registers extensions globally](https://github.com/kriszyp/msgpackr/issues/93) so the extensions leak into all connections within Theia, i.e., also non-API related areas such as the frontend-backend connection.
+And while the number of custom extensions is [limited to 100](https://github.com/kriszyp/msgpackr?tab=readme-ov-file#custom-extensions), checking the extensions for every single message may impact performance if there are many extensions or the conversion is very expensive.
+Another hurdle is that we need to ensure that we always detect the correct type of object purely from the object shape which may prove difficult if there are hundreds of objects and custom instances from user code.
+Consequently, we mainly use the msgpackr extension mechanism for very few common classes but rely on custom data transfer objects (DTOs) for the Theia plugin API, see also [Complex objects and RPC](#complex-objects-and-rpc).
 
 ## Adding new API
 
@@ -147,27 +155,25 @@ They each create the proxy to the other side in their constructors by using the
 
 ### Complex objects and RPC
 
-Only pure DTO objects without any functions or references to other objects can be transmitted via RPC.
-This often makes it impossible to just transfer objects provided by a plugin directly via RPC.
-In this case a DTO interface is necessary.
-These are defined [plugin-ext/src/common/plugin-api-rpc.ts](https://github.com/eclipse-theia/theia/blob/master/packages/plugin-ext/src/common/plugin-api-rpc.ts).
-Utility functions to convert between DTO and API types on the `Ext` side are usually added to [plugin-ext/src/plugin/type-converters.ts](https://github.com/eclipse-theia/theia/blob/master/packages/plugin-ext/src/plugin/type-converters.ts).
-Thus, this is also a good starting point to look for conversion utilities for existing types.
+When [encoding and decoding RPC messages](#encoding-and-decoding-rpc-messages) pure DTO objects that only carry properties can always be transmitted safely.
+However, due to the necessary serialization process, functions and references to other objects can never be transmitted safely.
 
 If functions of objects need to be invoked on the opposite side of their creation, the object needs to be cached on the creation side.
-The other side receives a handle (basically an id) that can be used to invoke the functionality on the creation side.
+The other side receives a handle (usually an id) that can be used to invoke the functionality on the creation side.
 As all cached objects are kept in memory, they should be disposed of when they are no longer needed.
-
-For instance, in [LanguagesExtImpl](https://github.com/eclipse-theia/theia/blob/master/packages/plugin-ext/src/plugin/languages.ts)#registerCodeActionsProvider a new code action provider is created and cached on the `Ext` side and then registered on the `Main` side via its handle.
-When the code action provider’s methods are later invoked on the `Main` side (e.g. in [LanguagesMainImpl](https://github.com/eclipse-theia/theia/blob/master/packages/plugin-ext/src/main/browser/languages-main.ts)#provideCodeActions), it calls the `Ext` side with this handle.
-The `Ext` side then gets the cached object, executes appropriate functions and returns the results back to the `Main` side (e.g. in [LanguageExtImpl](https://github.com/eclipse-theia/theia/blob/master/packages/plugin-ext/src/plugin/languages.ts)#$provideCodeActions).
-
+For instance, in [LanguagesExtImpl#registerCodeActionsProvider](https://github.com/eclipse-theia/theia/blob/master/packages/plugin-ext/src/plugin/languages.ts) a new code action provider is created and cached on the `Ext` side and then registered on the `Main` side via its handle.
+When the code action provider’s methods are later invoked on the `Main` side (e.g. in [LanguagesMainImpl#provideCodeActions](https://github.com/eclipse-theia/theia/blob/master/packages/plugin-ext/src/main/browser/languages-main.ts)), it calls the `Ext` side with this handle.
+The `Ext` side then gets the cached object, executes appropriate functions and returns the results back to the `Main` side (e.g. in [LanguagesExtImpl#$provideCodeActions](https://github.com/eclipse-theia/theia/blob/master/packages/plugin-ext/src/plugin/languages.ts)).
 Another example to browse are the [TaskExtImpl](https://github.com/eclipse-theia/theia/blob/master/packages/plugin-ext/src/plugin/tasks/tasks.ts) and [TaskMainImpl](https://github.com/eclipse-theia/theia/blob/master/packages/plugin-ext/src/main/browser/tasks-main.ts) classes.
 
+To [ensure correct type conversion](#encoding-and-decoding-rpc-messages) between the Theia backend and the plugin host we define an API protocol based on types and DTOs that can be transmitted safely.
+The plugin API and it's types are defined in [plugin-ext/src/common/plugin-api-rpc.ts](https://github.com/eclipse-theia/theia/blob/master/packages/plugin-ext/src/common/plugin-api-rpc.ts) with some additional conversion on the `Ext` side being defined in [plugin-ext/src/plugin/type-converters.ts](https://github.com/eclipse-theia/theia/blob/master/packages/plugin-ext/src/plugin/type-converters.ts).
+Thus, this is also a good starting point to look for conversion utilities for existing types.
+
 ### Adding new types
 
 New classes and other types such as enums are usually implemented in [plugin-ext/src/plugin/types-impl.ts](https://github.com/eclipse-theia/theia/blob/master/packages/plugin-ext/src/plugin/types-impl.ts).
-They can be added here and the added to the API object created in the API factory.
+They can be added there and then we can add them to the API object created in the API factory.
 
 ## Additional Links
 

packages/core/src/common/message-rpc/msg-pack-extension-manager.ts

@@ -21,7 +21,7 @@ import { addExtension } from 'msgpackr';
  * required for the default RPC communication. MsgPackR extensions
  * are installed globally on both ends of the communication channel.
  * (frontend-backend, pluginExt-pluginMain).
- * Is implemented as singleton as it is  also used in plugin child processes which have no access to inversify.
+ * Is implemented as singleton as it is also used in plugin child processes which have no access to inversify.
  */
 export class MsgPackExtensionManager {
     private static readonly INSTANCE = new MsgPackExtensionManager();