Thread and Worker Redesign and Refactor

[withinboredom]

@AlliBalliBaba discovered an architecture that allows for Fibers to work. However, it would be good to discuss the overall architecture and design so we can work on it together instead of one giant PR.

It would also be a good idea to use build tags and/or 'feature flags'? so we can switch between the old and new implementation easily. Thus if there is a (unforeseen) problem with the new architecture, a user can fall back to the old implementation. It would also allow for easier benchmarking.

What are the current issues with the thread/worker design?

• juggling many states: ready, booting (aka, initial ready), serving a worker script, serving a request for a worker, serving a request for a non-worker, etc.
• juggling many WaitGroups: termination, stopping, starting, restarting, readying, etc.
• the watcher can interrupt at any point
• the worker script can exit (or error out)
• and probably more...

That being said, @AlliBalliBaba has probably the most experience for how this should work; I'm just introducing ... so I'll hand it off to them.

cc: @dunglas

[AlliBalliBaba]

#1137 was meant as a first step towards starting and stopping threads individually. I agree that it is not yet ideal and that there's additional complexity and race conditions. These race conditions don't matter yet though since we don't change what a thread does at runtime (yet).

I like the idea of a state machine as you mentioned in #1137. Each type of thread would have its own state machine, I'll give this a go when I have time.

I don't think we should keep around multiple implementations though, that kind of defeats the point of refactoring. If there are 'unforseen problems', people can just rollback to a previous version (any new feature can bring unforseen problems).

[withinboredom]

I don't think we should keep around multiple implementations though, that kind of defeats the point of refactoring.

How so? It means that we can merge "dead code" and fix bugs without dealing with huge merge conflicts.

If there are 'unforseen problems', people can just rollback to a previous version (any new feature can bring unforseen problems).

That's just a side-effect. The real value is in the ability to work on code that isn't released yet and have smaller PRs.

[AlliBalliBaba]

Refactoring requires also repurposing and removing existing code. I guess you can't go 'all the way' when the old code has to stay. Working on code that hasn't released yet is the entire point of having branches in git, a branch with too many changes just causes a lot of conflicts with 'main'. Keeping the old code around won't solve that problem though, since then you'd have to implement all changes twice, once in the new and once in the old code.

The problem with my branch was just that it already fixed issues that you were trying to refactor on main, so things were done twice unnecessarily.

[withinboredom]

I was thinking of something like this for the state machine:

stateDiagram-v2
    [*] --> Unitialized: startup
    Unitialized --> Booted: initializing php
    Booted --> Ready: assigned role
    Ready --> Active: Started worker script / request
    Active --> Restart: PHP exit
    Restart --> Ready: Reset thread
    Active --> Failure: Unexpected termination
    Failure --> Ready: Reset thread
    Active --> Drain: Signal from watcher
    Drain --> Ready: Reset thread
    Ready --> Failure: Worker script crash
    Ready --> Drain: Signal from watcher
    Drain --> Shutdown: Graceful shutdown
    Shutdown --> [*]

Loading

[AlliBalliBaba]

Something like this 👍 . The watcher will not drain the threads, just 'continue' the current work iteration for a restart.

[withinboredom]

That's the drain state: finish the current request and either shutdown or return to ready.

[AlliBalliBaba]

Hmm I guess the question is: should the state machine also handle dispatching requests? In that case having an 'active' and 'drain' state makes more sense. Otherwise the state machine cannot really affect what the thread is doing during a 'work' iteration anyways.
In other words: Worker threads and regular threads both enter this state at some point, where they are waiting for work via a request (is this the 'true' ready state?:

select {
    case: a request is incoming
    case: stop listening for requests
}

I'm not sure how the above select statement would look like with a state machine.

[AlliBalliBaba]

After trying a bit to get the state machine to work I realized that this doesn't feel like quite the right approach. There are 2 concepts at play here:

• the state of the underlying C thread and the loop that it's executing
• the state of the 'worker' that decides what we should do at specific points in the loop

The loop can only be at 5 points

• before startup
• before script execution
• after script execution
• waiting for request
• after shutdown

The state of the worker then helps decide what to do at these 5 points (should it wait, continue, stop the loop and so on). Thinking about it in terms of transitioning from one state to another makes this very complicated. Rather, we are reaching one of these hooks and then do something based on the state.

So a state machine should implement these 5 hooks and then then switch-case all possible states.

[AlliBalliBaba]

I went ahead and added a minimal implementation of thread states that removes the global WaitGroups #1137 . Would something like this be enough to merge as a starting point that fixes Fibers @withinboredom ? Doing the whole state machine at once is probably overkill.