Scientific simulation in Bevy

[alice-i-cecile]

Bevy has tremendous potential as a framework for scientific simulations: it's fast, modular, easy to read/write and hooks in readily to convenient features like graphics and UIs.

This is a tracking issue to discuss the important features to help us get there (h/t @alec-deason). Many of these are "ecosystem" issues, rather than something that needs to be solved by the Bevy engine itself.

Universal features

1. Improved documentation and API maturity. Templates and guides are a big part of this, as is battle-tested correctness.
2. Charts / graphing integration. See https://github.com/eliotbo/bevy_plot for a community-made option!
3. Statistical integrations.
4. Ergonomic data logging.
5. Headless mode (Headless rendering to images #22, bevy cli #436), with good file-driven configuration.
6. Deterministic RNG. This includes good enough APIs to get to zero-system ambiguities (Ambiguous system ordering #1312) and ordered query iteration (Ordered iteration over Queries #1470), but the story is complex.
7. Fixed point support, for more stable computations (see Ability to set Transform fundamental native type to f64 or fixed-point types #1680).
8. Ergonomic saving and loading (First class save / load support #1442) with checkpointing.
9. Cross-machine distributed processing.
10. System order visualization and modification.
11. Better testing and correctness tools. Archetype Invariants #1481 is a good start.

Domain specific features

1. Indexes (Prototype Indexing #1587, Allow entities to store multiple components with the same type #1527), for simple and correct grid-based simulations.
2. (Partial) differential equations (PDE) integrations.
3. Linear algebra integrations.
4. Machine learning integrations.
5. Native physics functionality.

[alec-deason]

It seems like there are at least two distinct use cases that could be interesting to think about. One is where bevy directly runs a scientific simulation, which seems to be what the description above is focused on. But there's also clearly a need for test bed simulations to use in robotics and other similar domains. This has been mentioned at least once in #1207. I assume that that use case has softer requirements around mathematical provability and reproducibility and is more like a game with the player out of the loop.

Related to that are the AI test bed setups that various people have built, either on top of existing games like StarCraft or using custom engines. It seems like the main missing feature to enable that is a way for the external AI tool to introspect game state, either by looking at a rendered view as in #1207 or somehow introspecting the ECS store directly. That ties in to other conversations about modding where the ability of non-bevy, or even non-rust, code to interact with the ECS is interesting.

[alice-i-cecile]

The two places I've seen that latter pattern ("scientific understanding") really clearly is in a) forest management planning b) educational software. Rather than a hacked together interface like notebooks, you end up wanting a real polished app (think RShiny) with knobs you can tweak and interventions you can pick.

For initial twiddling purposes, I think that live parameter tweaking / hot reloading tools ala bevy_inspector_egui are going to be the way to go, and can hopefully displace the notebook pattern in a sensible fashion.

[alec-deason]

Yeah, that makes sense as a second phase of exploratory stuff. You have the basic model, now you're going to do a lot of experimenting with parameters so it makes sense to have a less flexible but more fiddleable GUI view into it.

It strikes me that the process of exploring model parameters in a scientific context might be very similar to the process of tuning a game late in development. You have the structure of the model down but you need to work out what it's sensitive to, or what interventions have a desired effect. Whether that's species succession outcomes after different tree harvesting patterns or DPS patterns in response to different upgrade drops the parameter tuning task might benefit from the same kind of tooling.

What I'm imagining is something like bevy_inspector_egui or as you say some kind of RShiny style GUI toolkit backed by a persistent, version controllable settings system. So the parameters can be loaded from file and also manipulated and saved from the development GUI. I guess that's functionality that an editor would normally provide? But having something that leans into graphing and data visualization seems a bit different and interesting.

[aevyrie]

A third related use case: CAD software (ping @aevyrie). In effect, you're combining a scientific simulation with a classic GUI in a seamless fashion, and want very configurable controls.

CAD could definitely be implemented in either of alec-deason's cases, although that's true for any application. I suspect the majority of projects that don't need the realtime rendering/computing advantages garnered by an ECS will put all their business logic into a separate crate, and only use the bevy-specific features for building an application and UI. This library would presumably handle all the scientific computing in an async fashion.

Forgive the apparent tangent, but where I foresee Bevy really shining is if we develop tools that streamline the process of building an application. A WYSIWYG editor for GUI will definitely be a boon for this, but there are a lot of other things that add drag to the process. When I was building tolstack, these were the "simple" things like file ser/de, a native file explorer, autosaving, application settings, theming, graphing, and exporting simulation results to csv. Even though these aren't the things I would normally associate with scientific simulation, I think they solve a real need for people who want to make polished simulation tools. Maybe a useful comparison is that Bevy could be to scientific simulation tools what dear-imgui is to graphics programming.

[alec-deason]

When I think of bevy as a scientific tool I'm thinking of it mostly as a framework for simulation, and that's all ECS. I think you're right that having bevy be a first class application framework would make it interesting to anyone building an application, including a scientific one. But I think ECS oriented, agent based simulation is something that Bevy basically does now that is distinctive and interesting for some kinds of scientific software and if the goal is to make bevy attractive to scientific users (which I don't know that it is, but it would be neat) then you'd lean into that.

[aevyrie]

Yeah, you're completely right, that seems like the most natural use of Bevy - the ECS would make agent-based simulation really nice. I think we have totally different use cases in mind within the same(ish) domain 😄. Regardless, I think there are some common needs across the scientific landscape we have a common need for, namely graphing and stats. I saw egui recently added graphing - using that with the inspector tool would be a really seamless way of gleaning statistical insight from simulations interactively.

I find Bevy uniquely attractive from any other application framework for the CAD stuff I want to do because: 3D is first class, it's all in one fast language, I can target native/web, and it's open source. Godot was probably my second option, but I really wanted to do everything in Rust. I hope that gives more context to my earlier comment!

[alice-i-cecile]

Another critical domain-specific integration: GIS.

[aristaeus]

I want charts/graphing! But I'm not sure where to start. Is there an obvious Rust library to integrate, or will we need to roll our own?

[alice-i-cecile]

This is actually not quite what I meant, though it could be very useful. I was advocating for integrating with an in-process Rust graphing library, not shelling out to a totally external tool.

Ah I see. So I guess you port it out to an in-process vanilla format (e.g. dataframes from Polars), and then use that to write it out into a stable file format. Okay yeah, that seems much more sane.

[aevyrie]

Something like this would be nifty: https://www.flecs.dev/query_demo

[aristaeus]

As well as aggregating ECS data, it will be common to build a time series as the game/simulation runs. I currently just have a Vec<f32> component on entities and a system to populate it, but it is probably worth thinking about how to make this more ergonomic.

• Time series on entities
• Aggregate time series over several entities.

[alec-deason]

Aggregating and down sampling over a single entity's time series is probably also useful.

[Bauxitedev]

For basic plotting you can use egui-bevy. Here's an example from my program:

[alice-i-cecile]

Another general feature: easy parallel process replication to enable monte-carlo simulation. See this thread on Discord.

[CGMossa]

I've picked up my efforts of using bevy for my projects. At this point I'd love some pointers as to how to do this, in any way possible. Should I make an issue or something similar?

[alice-i-cecile]

Make an issue that asks for an example IMO. Please link to both this thread and bevyengine/rfcs#16 as part of that, which may be very useful for enabling this use case.

[aevyrie]

@alice-i-cecile and I were chatting about UOM, I am of the opinion that this is a fundamental primitive that Bevy might want to be opinionated on, so that dependent crates can interoperate. The uom crate might be a good choice. This will impact scaling (size), time, and physics.

With Bevy and the ECS being so friendly to plugins, being opinionated on units of measure may be critical to avoid ecosystem fracturing with CAD/simulation/game physics plugins.

[alice-i-cecile]

Agreed with all of the above. It appears that uom is more popular, more mature and better designed than measurements. My initial inspection of their API lines up with this.

dimensioned looks to be the other candidate we should consider, and seems quite similar to uom. I'm not sure on their relative strengths, and both seem reasonably active.

[alice-i-cecile]

Dimensioned seems somewhat less mature and much less popular: it seems like the major selling point relative to uom is no-std, which doesn't matter in our use case. Fairly strongly leaning towards uom as a result: interop outside the Bevy ecosystem is very nice too.

Note: this choice also has implications on our choice of rationals; it looks like they're using num-rational as the dependency which seems perfectly fine.

[aristaeus]

One thing to watch out for is handling vectors with units. uom isn't generic over the scalar, so instead of Length<Vec> you need to do Vec<Length>. That's probably a better API choice, but
a) GLAM types aren't generic over their scalar
b) When I tried doing this with nalgebra, it was painful enough that I gave up. I forget the details, but one issue is that SIMD will only work when the vectors are over native types.

[aevyrie]

These are good points. A potential solution would be that all internal ops could be optimized glam types, with only externally exposed functions thinly wrapped with uom or similar types. We'd have to think carefully about the implications of this, and where it is truly needed. E.g.: the rendering system probably wouldn't benefit from having units on Transforms or lighting parameters. However, the physics system might benefit from applying real measurement units to the underlying rendering scalars to prevent physics bugs and make the solver unit agnostic.

[adsick]

actually would be very nice to be able to draw some pointclouds/octrees e.t.c. in Bevy, I already have an application and I need to implement the visualization of the physical process.

[maciejwnuk]

I think it would be nice to start off as a plugin (https://github.com/bevyengine/bevy/blob/main/docs/plugins_guidelines.md) and maybe then move on to more "standalone" format.

[maciejwnuk]

I would even sign up for collab. 😃

[RedlineTriad]

Fixed time-step simulation without a frame-rate limit would also be nice.
Would also be useful for testing to just run bevy with a fixed time-step say 10 ms for 1000 frames.

My use-case would something like https://www.codingame.com/training/easy/mars-lander-episode-1
Where there a various situations and you have to write code to land your rocket.
You could try to simulate it at faster rate than normal, say 1000x but it would be much more elegant to just specify a time-step and then just go as fast as possible.

[ManevilleF]

You can specify time step constraints to systems

[RedlineTriad]

You can specify time step constraints to systems

But I don't know of any way to do it with unlimited frame rate, simulate 1000 seconds of physics in 1 second with 10ms time-step.

[alice-i-cecile]

A proper frame-rate control API (#1343) should go a long way to solving this need.

[jueewo]

I've been working in agent-based modelling and decision support systems and I'm considering to use bevy for future implementations, academically and for an upcoming client project. So right now I'm working on a first prototypic implementation and I'm interested to share experiences.