[Feature Proposal]: Allow add_custom_type to register new 'engine-like' types

[sturnclaw]

A more concise version of #6067, as that is slightly confusing and long.

The point of the feature is to allow the mechanism for registering custom types in GDScript - EditorPlugin::add_custom_type, (as well as creating a mechanism for PluginScript/GDNative) to create 'engine-like' types. I'll explain what those are in a moment.

Currently add_custom_type is a thin factory for creating a node and assigning a script to it - a functionality already present via instancing of Scenes. The only 'custom type' related feature currently present is that it adds an entry in the Create A Node dialog.

An 'engine-like' type - for the purpose of this issue - is a custom type defined by GDNative / PluginScript / GDScript that extends an Engine class. (Likely either Resource or Node and its subclasses.)

'Engine-like' types may have scripts assigned to them as is normal for an engine or 'built-in' type, and shall transparently execute the functionality of the 'base script', regardless of whether a script is assigned to them or not.

'Engine-like' types will be present in the Create A Node dialog as normal (if they inherit from Node).

An 'engine-like' type is not a C++ type compiled into the engine, nor are they designed to allow C++ types compiled into the engine to inherit from them. They are, however, intended to allow a plugin - written in GDScript, GDNative, Mono, PluginScript, etc. to extend the engine's built-in type list with useful types, without having to code and compile the types directly into the engine.

A 'base script' is the custom behavior, implemented in GDNative, PluginScript, GDScript, or similar, that is bound to the custom type when the type is registered. This script or behavior shall, like C++ classes, always be executed and shall transparently behave like a C++ class to the perspective of the user.

Thus, a custom type with a script attached shall execute both the base script and the 'user' script, regardless of whether the 'user' script inherits directly from the 'base script' or a class higher in the inheritance tree, following the normal conventions for executing parent-class methods.

Creating 'engine-like' types that inherit other 'engine-like' types shall follow the above behavior, including that each 'engine-like' type in the inheritance hierarchy shall have their 'base scripts' executed as is normal for 'built-in' inheritance trees.

'Engine-like' types shall be loaded as early as possible in the engine lifecycle, to allow custom Resource types to be loaded from disk without issue.

TL;DR: You can define custom types and resources in a way that just works. More better, less hassle.

Implementation features:

• Allow loading new globals for type-names into GDScript
• Add mechanism to store 'engine-like' types and their 'base scripts', in a way that is accessible at runtime
• Add generation of new editor documentation pages at runtime

Pinging @willnationsdev, @Zylann. Did I miss anything? If I did, please get it to me ASAP.

[willnationsdev]

@Web-eWorks @reduz The issue at hand here is that reduz doesn't want to have custom type information tracked by the core of the engine, and I think I finally get WHY.

What you and I think of as "custom types" are really just the ability to assign script constraints on an Object. We want the ability to tell an object, "you may only have scripts that derive custom_script". The fact that you can't remove the custom_script from the Object through the traditional set_script method is the main point here.

Now, the ClassDB is meant to only ever store information about the in-engine C++ information, and the reason for that is because each scripting language that becomes associated with Godot will generally speaking be assumed to have its own means of recognizing type names (Python, C++, and C# all do this on their own). GDScript and VisualScript don't support this because Godot has always tracked individual classes based on their file, but he doesn't think this should be solved by storing their information in the ClassDB core, precisely because that information isn't associated with the core, but rather with individual scripting languages, and modules at that. If anything, the information should be managed by the scripting languages themselves just like all the other languages do (so, for example, the GDScriptLanguage may have a GDScriptDB of its own that tracks individual scripts it registers). This makes it crystal clear which classes are C++ classes and which ones are associated with a scripting language (and WHICH language they are associated with at that!).

The big question right now is whether @reduz will permit us to setup the script constraints on the Object class because without it, we are limited to only performing design-time checks of custom types within the Editor, and not run-time checks. Everything else (the GDScript and VisualScript globals, the in-editor documentation, and all of the Editor UI changes that accompany "custom types") can all be handled without making changes to core (and in fact I have made many of those changes in my branch).

All I need to do is create corresponding namespacing / custom type definitions in the GDScriptLanguage and VisualScriptLanguage classes and replace all references to ClassDB with references to the respective type storage data structures I create.

The other big issue is, even if @reduz acquiesces to let Object have script constraints, I'm not even exactly sure HOW to do it properly with the Object class because it still has to check type information for scripts against a particular scripting language. The only way to cleanly do it would be to have Object agnostically hand the verification off to some other class that is exposed to it. You would have to define some means of comparing two scripts AS RefPtrs (because Object is unaware of Script, or Ref<> for that matter) and hand that comparison off to the relevant ScriptDB-whatever that knows what the inheritance hierarchies for the given scripting language are and can tell you a) this RefPtr has been assigned name X, b) RefPtr A is a parent of RefPtr B, and c) the name Y has been assigned to RefPtr C.

[sturnclaw]

@willnationsdev I think I get what you are thinking of - just restricting it to a script that inherits the 'base script'. That's fine, and maybe that's even a better idea.

My idea was to bundle a base class and a Script (GDScript, C#, etc.) together, creating a new 'conceptual type' that, to all outward appearances, functions as if it were defined by the engine itself. I think I've laid out how the 'conceptual type' is supposed to work above.

If the parties involved want to go with your idea, then this issue can be archived for later reference, perhaps to be re-opened when I actually get an implementation for my idea.

[ghost]

IMO, this is getting a bit overwhelming to understand. It needs to be reduced, reduced and reduced again. I believe this is a case of circumlocution and/or information overload

[willnationsdev]

My idea was to bundle a base class and a Script (GDScript, C#, etc.) together, creating a new 'conceptual type' that, to all outward appearances, functions as if it were defined by the engine itself

@Web-eWorks And that might have worked in another setting, but the whole reason I am abandoning that concept (which would requireClassDB access) is because reduz doesn't want to go that route. And he is right when he says that the Editor can handle all of that, i.e. it can create the object with the script already as it is doing. The only difference here is that, in my version, to access those conceptual types, you would use a DB class associated with the scripting language rather than instancing the object from the ClassDB and then adding the script after-the-fact. But if the script constraints can be worked out, you'll effectively get your idea already because instancing the Object will auto-assign to it and lock the desired script anyway.

[willnationsdev]

@girng It is, in the reduced sense, the request for...

• script constraints on objects (Object must have a given script).
• The ability to add global GDScript identifiers for accessing given scripts (and ideally VisualScript too).
• The ability to generate in-editor documentation for scripts (if a plugin provides a handful of scripts with global identifiers, you can bet people will wanna look up documentation on how those scripts work).

On a basic level, that is the main "want".

On a secondary level, building on those ideas, is an additional desire for namespacing the script global identifiers, so that engine types, a game's scripts, and each of the plugins' scripts can all have global identifiers in GDScript without having name collisions.

That's the short version of everything.

[willnationsdev]

@Web-eWorks I think I just figured out the best solution that meets reduz's needs as well as ours. I wrote it in the original Issue, but I'll re-post it here for more easily locating it.

Content:

So, just reviewing some of the information in script_language.h, I feel like I could really just move the new content I added in the ClassDB over to the ScriptLanguage class and create a sort of ScriptDB singleton that only ever gets created if the scripting language intends to use it. And because object.cpp already includes script_language.h, it would be able to do the following in set_script(const RefPtr &p_script)...

1. check whether p_script is in fact a script
2. check which language p_script belongs to
3. hand off the p_script and the Object instance's custom_script to the ScriptDB for the associated ScriptLanguage,
4. get a response from the ScriptDB as to whether the p_script can be assigned to the Object given the constraints implied by the custom_script on the Object instance.

This would allow us to keep both ClassDB clean of any modifications and keep object.h clean (aside from a setter/getter for custom_script). It also prevents unnecessary complication of the scripting languages that have no need for relying on Godot to define identifiers and namespaces (like GDScript and VisualScript would need, unlike C#, C++, Python, etc.).

Then all you'd need to do is define ScriptServer methods that can perform identifier / namespace checks across all applicable scripting languages in order to replace the experimental ClassDB functionality of confirming custom types / custom type inheritance hierarchies. Easy enough.

[sturnclaw]

Alright then. Closing this to be revisited later.