Simple3D API Extended functionality

[Brackets-Coder]

This idea is mostly for @Xeltalliv

I think it'd be great if we could expand the functionality of the Simple3D API to include more than just external transforms.

Things like managing objects and their transformations, extra rendering things, etc. would be really cool.

I once expressed interest on Vadik1's profile on Scratch of creating a Simple3D Physics extension based on their 3D Rigid Body Physics Engine, but due to limited API access this is really difficult.

This idea/request is mainly for a possible physics extension, but more things can be done with it as well.

[Xeltalliv]

It is not as simple. It either needs to be very well-designed locked-down in functionality API or I can just add xeltallivSimple3Dapi.i_will_not_ask_for_help_when_this_breaks() that exposes extension's internals, that will keep getting non-backwards compatible changes every few updates. The problem with the second approach is that people tend to load Simple3D from the gallery (auto-updates), but expansion extensions from text (needs to be updated manually), meaning that for everyone who does that, it's not a question of "if", but rather "when" extensions stop being compatibile and their existing projects break.

In it's current state, Simple3D is usable, but isn't complete. So far, I've been designing it in a way where specifics of it's implementation aren't accessible to the projects using it, so I'm free to change them at any time. Like there is nothing stopping me from porting it to WebGPU if at any point I decide to do so.

"Simple3D: experimental optimizations" has been in stuck development for the past few months. Once it's done and merged, it will have a lot of changes, which you can view here. And once that is merged, I will not stop on it.

Also, if I were to make a physics extension, it would just handle the math and data processing, without anything rendering related. Scratchers will then need to create the "glue" code between the physics extension and the rendering extension of their choice themselves.

[Brackets-Coder]

Yes, I think you once discussed the complications of backwards and forward compatibility.
I still think it would be great if we could still make a 3D physics engine, whether it integrates directly into the extension and would have a format like Box2D or if it simply provides a framework for users to build off of. I don't know, it's just a thought.

[Brackets-Coder]

@Xeltalliv A few other features that I think would be helpful in Simple3D. All of these are suggestions, don't take them seriously.

1. Basic collision/intersection detection (not physics). This means one added boolean block called "is [mesh] touching [mesh]?", essentially copying the functionality of its two-dimensional counterpart. This might be very helpful in programming game functionality. For example, a player stepping on a button, or replicating the functionality of Unreal Engine's trigger boxes.

2. Raycast object. This would draw a straight line in a specified direction starting at one point and not ending until it collides with something. For performance, most raycast objects in 3D renderers will have a certain distance at which they stop in the case that they don't collide with anything. I'm aware that this may be very hard to implement, it's just a suggestion. It may integrate well with Augmented Reality if detecting real-world collision is supported. This may also come with the ability to retrieve a ray's length and the object it is colliding with (if available).

3. I swear I had another but it just slipped my mind.

I don't know your perspective on this, but like I said, they're just suggestions. I'd like to hear what you have to say, though. How hard would these be to implement, and what kind of work does this require? Would these suggestions even be practical on the Scratcher's side?

[Xeltalliv]

You seem to have a common misconception about how Simple3D works. There are no 3D objects. Simple3D does not know, does not care and has no way of knowing where anything is. It just sequentially executes the basic drawing commands it gets from the project to plot pixels on the screen and those pixels is the only thing it knows.

"is [mesh] touching [mesh]?" logically makes no sense. Using Simple3D is like making a single-sprite project that draws everything by moving/rotating/scaling the only sprite in the project and stamping it's costumes (same costume is usually stamped many times within a single frame). Meshes are like costumes. You can't have a block for "is [costume] touching [costume]?" because costumes on their own don't exist on the stage. And the moment you stamp them, they aren't "placed on the stage", they just turn into pixels on the pen layer which you can't really work with either. The same applies to meshes in Simple3D, except it's even more complicated because you can specify any transformation matrices for any of the 3 steps of rendering, change them as many times per frame as you want, switch between depth test modes, render targets, etc. The result of Simple3D is a 2D image. I can (only) add 2D per-pixel collision, but that probably wouldn't be that useful.

Simple3D is basically a mix of OpenGL 1.0 fixed function matrix transformations (very closely copied) and OpenGL 1.5 array buffers (but abstracted into a nicer and easier to work with concept called "meshes"). OpenGL is widely understood as just rendering API and people don't ask to add collision to it. Simple3D is something like that.

Also collision needs to be done on the CPU, but one of the primary selling points of Simple3D is that it only stores everything in VRAM, without storing duplicate of everything in RAM and I'd like to keep it that way.

[Xeltalliv]

The suggestions you provided would've made sense if there was an extension "Simple3D Objects" sitting on top of "Simple3D" to handle tracking of objects, scenes and cameras internally and handle sending the corresponding rendering commands to "Simple3D", while letting the project modify the 3D objects on the 3D scene.

[Brackets-Coder]

The suggestions you provided would've made sense if there was an extension "Simple3D Objects" sitting on top of "Simple3D" to handle tracking of objects, scenes and cameras internally and handle sending the corresponding rendering commands to "Simple3D", while letting the project modify the 3D objects on the 3D scene.

Interesting. That's what I would've guessed. Since Simple3D is nothing more than a renderer, wouldn't it be better to have a three.js extension or something similar for more complex object processing, custom shaders, collision, physics, game mechanics, etc?

[Xeltalliv]

There is now c48cb80

[Brackets-Coder]

looks great, but if i ever make a physics extension it will be a while because of #1732

[Brackets-Coder]

this opens so many more possibilities