Preprocess scripts on export to take constant feature tags into account

[Mickeon]

Describe the project you are working on

A not-so typical Puzzle Bubble-like that has made general use of OS.has_feature("debug")

Describe the problem or limitation you are having in your project

Thorough my game I have a good chunk of code shielded behind Feature Tags. This is quite useful, as it allows me to filter out debugging code from the final release, as well as apply conditional game logic to overcome a platform limitation (exclusive mobile controls, bug workarounds)

While it normally is not concerning in a simple game, the act of even reading Feature Tags at runtime can be slow.
The same, identical check is bound to happen in different parts of the project, sometimes every frame. Yet, the result will always be consistent, for that specific build.

As far as my understanding goes, there's no way to append or remove Feature Tags at runtime (at least most?).
They're platform-specific and are read-only. It wouldn't make sense for a iOS device to turn into a Android during execution, so why checking if this has even occurred?

Describe the feature / enhancement and how it helps to overcome the problem or limitation

Implement, generally speaking, some sort of optimisations to the way Feature Tags are compiled.

For example:

• When the passed argument of OS.has_feature() is a constant String (e.g. "windows", "web"...), the condition could be evaluated and converted to true or false at compile time;
• With the above requirement, and whenever OS.has_feature() is present in a conditional statement, or is chained inside one (e.g. if OS.has_feature("windows") and event.is_action_pressed("test_action")...), any nested code could be completely discarded at compile time if the feature isn't supported, since there should be no way to even run it on that target platform.

It seems like there's a desire to implement something like C's #if without worrying about overhead, so I believe this would be a pretty satisfactory way to do so, while being fairly accessible and user-friendly, as it would work seamlessly and efficiently without new users even needing to know about it.

Should something similar to this proposal be implemented, the Feature Tags documentation should definitely be updated to make note of it.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

I apologise for the poor quality of the example. I may update it if suggestions arise.
This is rather inaccurate to the inner workings and final bytecode, but suppose we have this vague Script:

extends Node

func _ready():
    if OS.has_feature("demo"):
         show_full_game_store_page()
         if OS.has_feature("mobile"):
              add_exclusive_levels()
    
func _input():
    if OS.has_feature("debug") and event.is_action_pressed("test_action"):
        instantly_win_level()

func _process(_delta):
    if OS.has_feature("debug"):
        display_debug_info()
    
    if OS.has_feature("JavaScript"):
         manage_webpage() # interact with the webpage in some interesting way
    
    var allow_gyro := OS.has_feature("mobile")
    pass # rest of game logic

Depending on what Feature Tags are included, the compiler could process them differently.
When only "debug" is included, if you were perfectly capable of extracting the same exact script from a release build, this is what you may see:

extends Node

func _ready():
    pass
    
func _input():
    if true and event.is_action_pressed("test_action"):
        instantly_win_level()

func _process(_delta):
    if true:
        display_debug_info()
    
    var allow_gyro := false
    pass # rest of game logic

On the other hand, when only "mobile" and "demo", a custom tag, are included in the build:

extends Node

func _ready():
    if true: 
         show_full_game_store_page()
         if true:
              add_exclusive_levels()
    
func _input():
    pass

func _process(_delta):
    var allow_gyro := true
    pass # rest of game logic

Note: ideally, the lone if true statements should be optimised and outright discarded in the final bytecode. They're here for demonstration.

If this enhancement will not be used often, can it be worked around with a few lines of script?

Surely. One way to mitigate the performance issues would be to assign the Feature status to a property as soon as possible.

var in_debug := OS.has_feature("debug")
var in_mobile := OS.has_feature("mobile")
...

But it is odd, to say the least. Feature Tags are, in a sense, constant values on their own, for aforementioned reasons above.

Is there a reason why this should be core and not an add-on in the asset library?

I don't believe there's any way for an add-on to directly do this.

[Calinou]

Related to #944 (which is for the shader language).

Has OS.has_feature() measured to be a bottleneck in real world projects exported in release mode? Adding a preprocessor would add a lot of complexity to GDScript and the export system.

[Mickeon]

I do not have concrete numbers currently, and... it's true. It's fairly complicated to implement such a thing. But, if something like it were to be added someday, Feature Tags would be among the few things I can think of that could benefit from it.

In a way the proposal is more-so about introducing the preprocessor seamlessly, without "polluting" the language with new keywords.

[KoBeWi]

Some old issues:
godotengine/godot#26649
godotengine/godot#6340

It would be extremely useful if was possible to disable parts of code depending on tags. For example, my game has a dependency on Steam module that provides its own Steam singleton. My problem is that if I want to publish the game on non-Steam platform (e.g. GoG), the module should be disabled. But then all code that uses this singleton, even conditionally, will break, because the class won't be available in the namespace and the scripts won't parse. Sure, there are workarounds for that, but being able to just "compile out" these parts of the code would make the life easier.

btw, there's also #373, which would make testing much easier.

[nathanfranke]

I propose this syntax

feature debug:
    print("Doing some debugging!")

feature windows:
    print("Sorry about that.")

I like having the feature names be identifiers, but we could probably re-use the $ logic, so both feature abc and feature "a/b+c" would work.

[Calinou]

Nim uses a when keyword as a compile-time if. Maybe we can do something like this for compile-time checks in general.

when sizeof(int) == 2:
  echo "running on a 16 bit system!"
elif sizeof(int) == 4:
  echo "running on a 32 bit system!"
elif sizeof(int) == 8:
  echo "running on a 64 bit system!"
else:
  echo "cannot happen!"

Quoting from the linked page for posterity:

The when statement is almost identical to the if statement with some exceptions:

• Each condition (expr) has to be a constant expression (of type bool).
• The statements do not open a new scope.
• The statements that belong to the expression that evaluated to true are translated by the compiler, the other statements are not checked for semantics! However, each condition is checked for semantics.

The when statement enables conditional compilation techniques. As a special syntactic extension, the when construct is also available within object definitions.

[dreadpon]

One of the concerns of my current project is to conditionally exclude parts of the app (like a full/trial version, but also with specific Linux builds that exclude, say, a VR/XR component).

This lead me to having:

• Hacky workarounds using strings to load scripts (instead of preload or just using the ClassName)
  • This kills any autocompletion of parse-time checks that I would really like to have
• Having my builds automatically include enabled plugins and singletons
  • So i have to abstract singletons as well using strings and loads
  • Plugins are bundled in the final build no matter what with virtually no way to exclude them from the build unless manually disabling, exporting and enabling again

I realise the last point is a bit off-topic, but I think it illustrates what the final overhaul of the entire system will need to deal with.

Not to mention, most solutions proposed so far exclude code inside functions/methods, but what preprocessor allows to exclude is entire subclasses, properties and methods themselves. This system is a lot more flexible IMO, and I already know that for my project simple method-level optimizations won't be enough.

Having conditional parsing would be a great help, but it alone isn't really enough for a real production environment. I realize some of the things above can be solved with a better app architecture buuuut... As it currently stands, even the good arhitecture will be dragged down by the workarounds required to make it behave just the way it needs to.

[dalexeev]

I created an export plugin that does this: dalexeev/gdscript-preprocessor.

[Xananax]

I would suggest a zig-inspired comptime and inline.

func _ready():
  inline if OS.has_feature("demo"):
    show_full_game_store_page()
    inline if OS.has_feature("mobile"):
      add_exclusive_levels()

func _input():
  inline if OS.has_feature("debug"):
    if event.is_action_pressed("test_action"):
      instantly_win_level()

const var levels := {}

func populate_levels():
  comptime:
    for file in dir:
      if some_condition(file):
        levels.append(file)

No need to follow the syntax; for example, one construct might be sufficient; it could be called const instead of comptime, etc. But the idea of a keyword that can apply to multiple statements, expressions, or blocks is more versatile than a specific word, even if in the beginning it might work only for if.

[CreepyGnome]

I would think any language that is going to be used to ship commercial products would have a preprocessor directives syntax that could be used to remove blocks of code so they do not get shipped with the product. If the code, even if compiled to assembly and stuck in an executable, can be hacked and modified to use the code we disable with if blocks in the code because it still shipped is no good. It has a perf side effect as well and is useful to ensure code that could unlock a demo into a full version does not even ship. There are plenty of other good reasons for a language to have some form of preprocessor directives.

I think Godot and GDScript have matured enough to make it part of the language and not lean on 3rd party tools to hack similar functionality.

[KoBeWi]

is useful to ensure code that could unlock a demo into a full version does not even ship

If you ship full game's content with the demo, players will find a way to unlock it anyway. The best way to prevent "unlocking" demo is to not include all files from the full game.

[CreepyGnome]

is useful to ensure code that could unlock a demo into a full version does not even ship

If you ship full game's content with the demo, players will find a way to unlock it anyway. The best way to prevent "unlocking" demo is to not include all files from the full game.

That is something you should also do when it is possible, as it may not always be possible with all games that you add a demo version of. It's best to provide ALL the standard practices to ensure code and files that are not wanted to be shipped do not ship. Godot has the not shipping files feature, but no ability to not ship code feature.

Its just something I think a language needs to have to move in to a more mature portion of its life especially when it is going to be used by commercial products.

We have thinks like the previously mentioned gdscript-preprocessor add-on which is more of a workaround way to solve the problem, and its nice that add-ons like this exist and shows that there is a demand for this functionality.