Discussing alternative ways of specifying system priority

[acmcarther]

Expanding upon from #71. Sorry for the wall of text, I've got a ton of thoughts about this issue and didn't have a lot of time to refine them down to a bite sized blurb. Thanks for reading, anyway!

My thoughts, mostly boil down to two things:

• That data dependencies aren't necessarily the same as runtime dependencies,
• That maintaining the "knowledge" of who goes before who in more than one place, at least with respect to a given system, is a bit of a pain.

Data dependency is not always the same as run order dependency

It turns out in at least my case that a data dependency is not the same as the run ordering. If that were the case, I'd have cycles in my execution strategy. One example (from my client)

[network::AdapterSystem]
-> [some interpreting system] Via incoming network events
-> [player::InputSystem] Via some interpreted state
-> [network::AdapterSystem] Via outbound player input event

I resolve this in my particular case by just not having the network system depend on things that supply outgoing events. It emits them first thing on the next tick.

I've also got cases where I enforce a "dependency" that's not directly visible via the data dependencies. One example from my client.

[network::AdapterSystem]
-> [synchronization::System] reconstructs state emitted by server, writing (LOTS of) component state
~> [renderer::System] uses that component state.

There are alternate approaches to solve both problems. It would be possible to break the adapter system into two systems, or to have the renderer system depend on some sentinel value. In my case, it was more difficult at an implementation level to do the former due to my underlying socket, and the latter is actually more complicated due to a Send issue with my renderer.

Centralizing the "knowledge" of system priority.

A different, and for me more compelling, reason is that maintaining the explicit priorities is just a pain in the butt when specifying them the current way.

I've found, at least in my case, that keeping the knowledge of dependency ordering localized, even if its often redundant, is less overhead than the alternatives. Those alternatives as I see them are either maintaining a huge set of constants with numerical priorities, or making systems responsible for knowing their priority in the raw. The former gets pretty busy and doesn't separate concerns well, and the latter IMO isn't viable since that priority value is intrinsically dependent on the values of its dependencies, and the number of other systems -- it becomes a maintenance headache.

Thus, at least in my swing at the problem, I put off actually numbering the systems, and just resolve those values at the last possible second when every system, and their explicit dependencies, is available.

Why now?

In some sense, I think this problem is more salient when the friction in passing events is lessened. I hacked together a broadcasting pub sub system (that uses specs under the hood), that lets emitters and receivers be less aware of each other. That increase in ergonomics (IMO, anyway), made the pain of manual system juggling a bit more obvious.

For morbidly curious, the implementation is embedded in my game repo (a test here). Its super naive and unoptimized though.

Post-submit addendum

I didn't realize until after the fact, but the fact that i use pubsub, and thus obscure the lineage of a piece of data, sorta necessitates a more direct way of specifying dependencies. I imaging there could be other cases as well where the data dependencies are similarly complicated.

[jeffparsons]

Now that I'm actually using Specs, I figure I'll add my AU$0.02, too:

My preferred API to meet my immediate needs would looks something like this from a consumer's perspective:

fn create_systems(&self) -> Result<(), specs::RunOrderOverconstrainedError>
    // ... (create systems)

    let render_handle = self.planner.add_system(render_sys, "render");
    let control_handle = self.planner.add_system(control_sys, "control");
    let net_in_handle = self.planner.add_system(incoming_network_events_sys, "net_in");
    let physics_handle = self.planner.add_system(physics_sys, "physics");
    
    planner.add_constraint(Constraint { sooner: control_handle, later: physics_handle })?;
    planner.add_constraint(Constraint { sooner: net_in_handle, later: physics_handle })?;
    planner.add_constraint(Constraint { sooner: physics_handle, later: render_handle })?;

    Ok(())
}

I prefer registering constraints based on system instances, not types; I might want a generic system with multiple instances that should run at different places in the order. I can provide examples if this point is contentious. I only mention it all because something linked from discussion here #71 appeared to imply specifying ordering between types of systems.

Implementation notes

I'm suggesting an opaque handle type so that Specs could internally give every system instance an ID with some wrapper around whatever system you pass it. That way users don't have to create this themselves.

The original "priority number"-based ordering could be emulated in this scheme as well, by adding run order constraints between system with adjacent priority orders, and then throwing them into the mix with any other systems whose priorities are defined through constraints.

NOTE: My understanding is that priority number based ordering is a strict subset of constraint based ordering; i.e. Specs won't start executing a system if there are any systems of higher priority still running / waiting, even if it has a spare thread. If this is not true, then I might be using Specs all wrong at the moment!

Checking that the system is not over-constrained would be as simple as performing a topological sort (e.g. https://github.com/gifnksm/topological-sort-rs). I haven't looked into the implementation of Specs enough to have an opinion on how best to implement the actual ordering at run-time, though.

[acmcarther]

For me an instance-based constraint style would work as well. This also seems potentially easier to implement since you'd not have to worry about TypeIds. Your handles also would remove my current need for an additional "naming" scheme.

I hadn't considered multiple instances of a system though -- do you have an example that you could share out of curiosity?

[jeffparsons]

I hadn't considered multiple instances of a system though -- do you have an example that you could share out of curiosity?

I haven't actually implemented any of these yet, and they'd be trivial to implement by making a separate wrapper type per instance, so take this with a grain of salt.

The main use case is I have in mind is systems where each instance does basically the same thing, but on different subsets of entities. I'm imagining an order of system execution along these lines:

• Handle user input, apply to player character.
• Handle network events, apply to entities (position, velocity, etc.).
• Physics 1: authoritative updates and dead reckoning for all entities that are either currently very close to the player, or very large (based on bounding boxes in scene hierarchy) so we can keep things nice and smooth where it's noticeable.
• Drawing things to screen.
• Physics 2: same as physics 1, but for however many less important entities we can get through by the time a deadline is up, based on some kind of priority queue based on some combination of their scale/distance, their sensitivity to slow updates, and how long since they received an update.

Even as I write this, I'm realising that this would probably be better represented as two separate systems that can share a bunch of logic that exists outside of the systems themselves. So perhaps there is actually no good reason to prefer per-instance priorities. 😄

[WaDelma]

For specifying system prorities daggy could be used.