Investigate standardized XState integration

[mweststrate]

This is a subject I want to research soon, so any upfront ideas, inputs, examples are welcome!

cc @RainerAtSpirit @mattruby @davidkpiano

[davidkpiano]

I have to look deeper into MST. Any state management solution is fully compatible with XState and assign() as long as an immutable object is returned. At a cursory glance, I'm not sure that MST is "immutable" by default (I could be wrong), and instead it wants to "own" the state and state changes, which might be in conflict with XState.

The reason XState prefers immutable data structures for context is because it needs to control when context changes occur, and it needs to maintain history (just like Redux). Also, it's preferred change mechanism is property based (like React's setState) because it allows analysis into how certain properties will change, and when they will change, instead of blindly assuming that assign() will change the entire context object.

By the way, I've been using immer heavily with XState. It works extremely well:

foo: {
  on: {
    SOME_EVENT: {
      actions: assign((ctx, e) => produce(ctx, draft => draft.count += e.value))
    }
  }
}

So if MST works similarly, it can fit well.

[mattruby]

MST is great at serializing and deserializing it's current state. I've created a few state machines using MST with a few enums. My first thought was a way to have MST own the actions and context. And xstate own what is called when. But things get interesting when I get into who initializes what when. It would also potentially useful to have an observable flavor of State. It would be neat to have an observable nextEvents and such.

I'm still having a hard time deciding how best to weave the tools together.

[RainerAtSpirit]

Here's an pretty rough POC of mst and xstate integration. https://github.com/RainerAtSpirit/xstatemst.
Like @mattruby I'm having a hard time to decicde, who should be responsible for what. If we e.g. want to use mobx-react observer to react to both context and state changes than we would need to have e.g. State.value and State.nextEvents observables. There're probably more though.

[mattruby]

At one point, I was thinking of creating an MST model that would match xstate's data model. So I would model my State in MST. But I think I'd end up having to create a whole machine interpreter.

[RainerAtSpirit]

I'd rather like using the xstate interpreter for that job.
What about having an XStateAble model that we can add as volatile prop to any other model?

.volatile(self => ({
    xstate: createXStateAble(machineDefinition, {
      actions: {
        fetchData: self.fetchData,
        updateData: self.updateData,
        showErrorMessage: self.showErrorMessage
      }
    })
  }))

xstateable would have the minimum amount of props that a) the UI require and b) are needed to rehydrate the combination at any point in time. Something along the line.

// createXStateAble .ts
import { Instance, types } from "mobx-state-tree"
import { Machine } from "xstate"
import { interpret } from "xstate/lib/interpreter"

export const createXStateAble = (machineDefinition: any, config: any) => {
  const machine = Machine(machineDefinition, config)

  const XStateable = types
    .model("XStateable", {
      machineDefinition: types.frozen(),
      value: types.optional(types.string, ""),
      nextEvents: types.array(types.string)
    })
    .volatile((self: any) => ({
      machine
    }))
    .volatile((self: any) => ({
      service: interpret(self.machine).onTransition(state => {
        self.setValue(state.value)
        self.setNextEvents(state.nextEvents)
      })
    }))
    .actions((self: any) => ({
      setValue(value: string) {
        self.value = value
      },
      setNextEvents(nextEvents: any) {
        self.nextEvents = nextEvents
      },
      afterCreate() {
        self.value = self.machine.initialState.value
        self.service.start()
      }
    }))

  const xstate = ((window as any).xstate = XStateable.create({
    machineDefinition
  }))

  return xstate
}

I've updated https://github.com/RainerAtSpirit/xstatemst. accordingly.

[RainerAtSpirit]

Now that async recipes has landed in immer 2.0 immerjs/immer@5fee518, immer looks like a better companion for xstate to me. As @davidkpiano outlined is plays nicely with assign and it's only purpose would be to produce a new ctx.
I still need to find some time to verify that assumption ;).

[davidkpiano]

Here is how nicely Immer (1.x) plays with XState. It's beautiful:

[RainerAtSpirit]

Immer meets xstate, here's my first pick: https://github.com/RainerAtSpirit/xstatemst/tree/immer.
This will work with immer 1 as well as it's not making usuage of immer 2 async feature.
As an alternative to @davidkpiano design pattern above, this is using an external updateContext action.

export const machine = Machine<
  IPromiseMachineContext,
  IPromiseMachineSchema,
  PromiseMachineEvents
>(machineConfig, {
  actions: {
    fetchData: async function fetchData(ctx, event) {
      const success = Math.random() < 0.5

      await delay(2000)
      if (success) {
        // service is a singleton that will be started/stopped within <App>
        service.send({
          type: "FULFILL",
          data: ["foo", "bar", "baz"]
        })
      } else {
        service.send({ type: "REJECT", message: "No luck today" })
      }
    },
    updateContext: assign((ctx, event) =>
      produce(ctx, draft => {
        switch (event.type) {
          case "FULFILL":
            draft.data = event.data
            draft.message = ""
            break
          case "REJECT":
            draft.message = event.message
            break
        }
      })
    )
  }
})

export const service = interpret(machine)

[mweststrate]

Have been working with a MST-xstate integration last week, first results look very promising! Stay tuned :)

[lu-zen]

Pls integrate mobx too 🙏

[aksonov]

@mweststrate Is there any progress about XState integration?

[mweststrate]

Recently, used the following utility successfully in a project:

import { types } from 'mobx-state-tree'
import { isObject } from 'util'
import { State } from 'xstate'
import { interpret } from 'xstate/lib/interpreter'

export function createXStateStore(machineInitializer) {
  return types
    .model('XStateStore', {
      value: types.optional(types.frozen(), undefined), // contains the .value of the current state
    })
    .volatile(() => ({
      currentState: null, // by making this part of the volatile state, the ref will be observable. Alternatively it could be put in an observable.box in the extend closure.
    }))
    .extend((self) => {
      function setState(state) {
        self.currentState = state

        if (isObject(state.value)) {
          self.value = state.value
        } else {
          self.value = { [state.value]: {} }
        }
      }

      let machine
      let interpreter

      return {
        views: {
          get machine() {
            return machine
          },
          get rootState() {
            return self.currentState.toStrings()[0] // The first one captures the top level step
          },
          get stepState() {
            const stateStrings = self.currentState.toStrings()
            const mostSpecific = stateStrings[stateStrings.length - 1] // The last one is the most specific state
            return mostSpecific.substr(this.rootState.length + 1) // cut of the top level name
          },
          matches(stateString = '') {
            return self.currentState.matches(stateString)
          },
        },
        actions: {
          sendEvent(event) {
            interpreter.send(event)
          },
          afterCreate() {
            machine = machineInitializer(self)
            interpreter = interpret(machine).onTransition(setState)
            // *if* there is some initial value provided, construct a state from that and start with it
            const state = self.value ? State.from(self.value) : undefined
            interpreter.start(state)
          },
        },
      }
    })
}

It worked great (feel free to create a lib out of it). For example with the following (partial) machine definition:

import { address, payment, confirmation } from './states'

export const CheckoutMachineDefinition = {
  id: 'checkout',
  initial: 'address',
  context: {
    address: {
      complete: false,
    },
    payment: {
      complete: false,
    },
  },
  meta: {
    order: [
      {
        name: 'address',
        title: 'Shipping',
        continueButtonTitle: 'Continue To Payment',
        event: 'CHANGE_TO_ADDRESS',
      },
      {
        name: 'payment',
        title: 'Payment',
        continueButtonTitle: 'Continue To Review Order',
        event: 'CHANGE_TO_PAYMENT',
      },
      {
        name: 'confirmation',
        title: 'Review',
        continueButtonTitle: 'Place Order',
        event: 'CHANGE_TO_CONFIRMATION',
      },
    ],
  },
  states: {
    address,
    payment,
    confirmation,
  },
  on: {
    CHANGE_TO_ADDRESS: '#checkout.address.review',
    CHANGE_TO_PAYMENT: '#payment',
    CHANGE_TO_CONFIRMATION: '#confirmation',
  },
}

It can be tested / used like this:

import { Machine } from 'xstate'
import { autorun } from 'mobx'
import { createXStateStore } from './index'
import { CheckoutMachineDefinition } from '../checkoutmachine'

describe('XStateStore', () => {
  const testable = createXStateStore(() =>
    Machine(CheckoutMachineDefinition, {
      guards: {
        validate: () => true,
      },
    }),
  )

  it('creates a default XStateStore', () => {
    const test = testable.create({
      machineDefinition: CheckoutMachineDefinition,
    })
    expect(test.toJSON()).toMatchSnapshot()
  })
  it('updates the value with a string', () => {
    const test = testable.create({ value: 'address' })
    expect(test.value).toEqual({ address: 'shipping' })
  })
  it.skip('updates the value with an empty object string', () => {
    const test = testable.create({ value: { address: '' } })
    expect(test.value).toEqual({ address: {} })
  })
  it('updates the value with an object', () => {
    const test = testable.create({
      value: {
        address: {
          shipping: {},
        },
      },
    })
    expect(test.value).toEqual({
      address: {
        shipping: {},
      },
    })
  })
  describe('stepState - ', () => {
    it('one level', () => {
      const test = testable.create({ value: 'payment' })
      expect(test.value).toMatchInlineSnapshot(`
                Object {
                  "payment": Object {},
                }
            `)
      expect(test.matches('payment')).toBeTruthy()
    })
    it('two levels', () => {
      const test = testable.create({
        value: { address: 'shipping' },
      })
      expect(test.stepState).toBe('shipping')
      expect(test.matches('address')).toBeTruthy()
      expect(test.matches('address.shipping')).toBeTruthy()
    })
    it('three levels', () => {
      const test = testable.create({
        value: { billingAddress: { test1: { test2: {} } } },
      })
      expect(test.stepState).toBe('test1.test2')
    })
  })
  describe('matches - ', () => {
    it('undefined argument', () => {
      try {
        const test = testable.create({ value: { address: { review: { editBilling: 'billing' } } } })
      } catch (e) {
        expect(e).toMatchInlineSnapshot(`[Error: Child state 'review' does not exist on 'address']`)
      }
    })
    it('full state', () => {
      const test = testable.create({ value: { billingAddress: { test1: { test2: {} } } } })
      expect(test.matches('billingAddress.test1.test2')).toBeTruthy()
    })
    it('parent state', () => {
      const test = testable.create({ value: { billingAddress: { test1: { test2: {} } } } })
      expect(test.matches('billingAddress')).toBeTruthy()
    })
    it('child state', () => {
      const test = testable.create({ value: { billingAddress: { test1: { test2: {} } } } })
      expect(test.matches('billingAddress.notThere')).toBeFalsy()
    })
  })
  describe('transitions are reactive - ', () => {
    it('value', () => {
      const store = testable.create()
      const states = []
      autorun(() => {
        states.push(store.value)
      })

      store.sendEvent('CHANGE_TO_ADDRESS')
      store.sendEvent('CHANGE_TO_CONFIRMATION')

      expect(states).toMatchSnapshot()
    })
    it('rootState', () => {
      const store = testable.create()
      const states = []
      autorun(() => {
        states.push(store.rootState)
      })

      store.sendEvent('CHANGE_TO_ADDRESS')
      store.sendEvent('CHANGE_TO_CONFIRMATION')

      expect(states).toMatchSnapshot()
    })
    it('matches', () => {
      const store = testable.create()
      const states = []
      autorun(() => {
        states.push(store.matches('address'))
      })

      store.sendEvent('CHANGE_TO_ADDRESS')
      store.sendEvent('CHANGE_TO_CONFIRMATION')

      expect(states).toMatchSnapshot()
    })
  })
})

So the MST store wraps around the machine definition and interpreter. It has two big benefits: One, the state is serializable as it is stored as a frozen, secondly, everything is observable, so using store.matches can be tracked in a computed view / component etc.

CC @mattruby

[davidkpiano]

I'd be glad to create a small @xstate/mobx-state-tree library.

[andraz]

Do one thing and do it well:
define the next legal state (of a variable) in relation to the current state (of the same variable).
This is a feature MST does not have, but would benefit greatly if it would have it.

My implementation of MXST (mobx-xstate-state-tree) in under 50 lines of code (+jsx demo lines).
https://codesandbox.io/s/mxst-mobxxstatestatetree-8nibz

We get an observable string called "state" which responds to commands sent via .send() into the MST.

Everything feels built into MST, so it behaves to an outside user simply as an YAO (yet another observable).

Put this model into a map node on parent MST, and you can have 50 machines (for example rows in a spreadsheet), all running on their own having their own state, responding in real time to send() commands automagically via observable HOC wrapped JSX. Powerfull stuff...

---

edit: did not notice that similar implementation was done few posts above just half a day before, we came to similar conclusions... this is a powerful pattern, could it be merged into MST core functionality?

[mweststrate]

@davidkpiano that would be awesome! Make sure to ping me if you have questions around MST or need review and such

[mattruby]

Just came upon this throwing a hail mary at google. This seems like a match made in heaven. It seems to me that xstate/FSM would serve as a way to conditionalize/contextualize MSTs, but if you're considering creating a dedicated integration, that would be far better than rolling my own!

I agree! I feel like MST as the context. Have the actions automatically wired up. The ability to easily use views as guards. Possibly have the state as a types.custom.

We have a pretty great integration in place right now, but I still think there's a great opportunity here. It's hard to determine where MST ends and xstate begins.

[loganpowell]

@mattruby can you share an example or your current work?

[andraz]

I've used my implementation few comments above in a recent project while also onboarding the coworker to all of the principles behind XState and MobX and digged in many refactorings to simplify it as much as possible.

The trick is in the fact you can make XState practically invisible (and less scary) to the outside, you just create a root model with:

• the { machine: MSTMachine } as a type in model
• an action (named send) which calls the machine.send()
• a view (named state) with a getter returning machine.state

From this point MST should take care of the view by itself when you call the action, everything happening at the root level.

p.s.: I later upgraded my implementation a bit because I needed to pass the machine configuration dynamically into the model. This requires you to use a function MSTMachine(config) which returns MSTMachine type configured internally with the configuration you wish to use (a factory function).

Simpler alternatives are to just have a machine.init(config) call to trigger after MST is created or you can even hardcode the config directly into the type (this could work if there are only one or two possible machine configs in your project).

Plugging the MSTMachine type into a call similar to { mymach: types.machine(config) } should integrate it seamlessly into MST.

Then it is a team decision if they prefer root.mymach.state & root.mymach.send() to the root.state & root.send() use pattern.

[mattruby]

@mattruby can you share an example or your current work?
I'm pretty much using Michel's impl: #1149 (comment)

We've tweaked it a little. But that's the core.

[loganpowell]

@davidkpiano when you say:

Any state management solution is fully compatible with XState and assign() as long as an immutable object is returned.

How do you check for immutability? I.e., is this a hard requirement (enforced), would returning a POJO break everything or both?

[davidkpiano]

XState doesn't check for immutability, but it's just a best practice. Mutating context can break in unpredictable ways though (as with any mutation); e.g....

const machine = Machine({
  context: { count: 0 },
  // ...
});

const firstMachine = machine;
const secondMachine = machine;

If you do assign(ctx => ctx.count++) in the firstMachine, then the context object of the secondMachine is also mutated.

I'm thinking of an API that may solve this though, via "late binding":

// ⚠️ tentative future API
const machine = Machine({
  context: () => ({ count: 0 }), // safe to mutate - completely new object!
  // ...
});

[andraz]

Solution is to define a map of machines:

const Store = types
  .model({
    machines: types.map(MSTMachine)
  })
  .actions(store => ({
    addMachine({ name, machineConfig }) {
      store.machine.set(name, { machineConfig })
    },
    // everything below is optional, store will work without it but code won't be as nice
    aClick() {
      store.machine('a').send('click')
    }
  }))
  .views(store => ({
    machine(name) {
      return store.machines.get(name)
    },
    get aState() {
      return store.machine('a').state
    }
  }))

const store = Store.create()

store.addMachine({
  name: 'a',
  machineConfig : customMachineConfiguration
})

Then link callbacks:

  <button onClick={store.aClick}>send click to someName</button>

And render their values:

const Autorefresh = observable(_ => <div>{store.aState}</div>)

Done.

How much of this could be automated is to be discussed. It would be great if all getters and click handlers could generate themselves "automagically", because it is just copy paste boilerplate code...

---

Keep it simple.

State.value should be a scalar, not some complex conglomerate in the form of an object (don't even get me started on the mutability of it, problems with memory leaks, garbage collection, pointers getting passed around when juniors fall into the shallow cloning hole etc...).

In short, state can be a boolean, integer, float or a string, that's it.

If it is more complex, the machine configuration is probably already few 100 lines long. This is a clear sign you can refactor your machine configuration into submachines defining each discrete value separately (separation of concerns). If you already did that, great, you're 90% done. now just separate submachines into literally "separate machines". Aka: standalone variables.

We are already using MST for the complex state because it is the best tool for it.

We can create machines on this level, to manipulate each key in that state precisely. This is where XState shines.

---

To illustrate the problem with an example:

Toolbar of bold/italic/underline can be controlled with

• jQuery (let's not even go there... using the live DOM's class to store the boolean state ?!)
• some long convoluted state machine logic tracking 3 separate states in submachines
• broken apart into 3 separate objects, implement each one with a (same) machine that toggles a boolean when receiving a 'click', organize the states in MST, then observe and glue them together in JSX.

[michaelstack]

What is the status of this issue?

Did everyone lose interest?

[mweststrate]

Afaik above solutions are still being used, just no one standardized it as ready to use documented OSS package :)

…

On Wed, Jan 5, 2022 at 12:29 AM Michael Stack ***@***.***> wrote: What is the status of this issue? Did everyone lose interest? — Reply to this email directly, view it on GitHub <#1149 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAN4NBFRMBLFJ55WOL4BPGLUUOGF7ANCNFSM4GSCG3KQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[Slooowpoke]

We're currently using a derivative of both the solutions provided above since the first solution offered an easy way to add (to xstate) our existing MST actions, the second, offers the .matches property and solved an issue with nested machine states (swapping: value: types.optional(types.string, ""), for value: types.optional(types.frozen(), {}),)

It all plays pretty nicely, though be aware that if you use MST stores in the xstate guards, you can end up with race conditions. To avoid them you just need to ensure that you treat store updates (that are being used in guards) as xstate services, not xstate actions.

[beepsoft]

@Slooowpoke can you maybe share your final solution?

[andraz]

Still using a similar pattern as above, upgraded with the latest developments in the architecture:

Moved the statecharts to the FAAS side where I am running "state transition as a service" on the BFF (backend for frontend) of micro frontends: sending the transition name in (by writing to a MST key monitored by onPatch which emits it to ws), getting state name back (BFF Redis holds the current state to know the context while keeping FAAS itself stateless).

MST remains the perfect solution for receiving data in from the BFF in real time: websocket listener just triggers an action on MST, done. Everything on UI after that is handled automatically by observers. Now I am literally driving the frontend UI with XState - from the back seat - in FAAS.

[Slooowpoke]

@Slooowpoke can you maybe share your final solution?

@beepsoft Of course, here it is.

I think we might be looking to move away from it though, there's been some issues with writing boilerplate to keep the two together. It might be an anti-pattern for xstate (and maybe mobx), but we've been using a fairly large state machine coupled together with a fairly large mobx-state-tree store. Ideally we'd have lots of smaller machines with final states, but coupling them (with the solution attached) would mean moving them all up to the top store and out of the machine (because we can't pass the services / actions / guards in easily).

I'm sure theres a better solution but the ones I've come to have all felt a bit clunky, equally since MST doesn't reflect the flow types we have to define each one when declaring them in mobx which is error prone (we introduced helper functions for this and there is a PR I've raised which would clear this up).

The solution I've been toying with recently is just using mobx with xstate. It uses an observable state as context and sits in front of the assign action, it gives the observability of mobx with everything else being the xstate way. I haven't tried it out much and it could probably be improved but it seems ok initially.

I'd be super keen to see what you've ended up with @andraz, it could be that I've gotten it all twisted and maybe they are a match made in heaven!

[redbar0n]

@andraz

What benefits have you been able to get by driving the frontend UI from the backend (instead of having the state machine stored on the frontend)?

How has it worked out with versioning (potential version skew between frontend UI and backend state machine) ?

[andraz]

The solution was ahead of its time in that it required a developer to be able to comprehend and maintain awareness of all the system's moving parts in order to upgrade it without causing any issues.

As is often the case in the field, many members of the team have only a basic understanding of key concepts, often outdated since 1995. These include jQuery, REST API calls, and the concept of a view change. Some also have a narrow view of the system's functionality, seeing only the transition of a Next.js URL, akin to how some may view PHP.

Training junior-level staff on advanced concepts is inefficient and time-consuming, leading to system failures and unmaintainable code. Upgrades are often unsuccessful due to the lack of understanding of the underlying architecture. As a result, significant time and resources are wasted on implementing minor changes, such as 100s of lines of 80% unused, duplicated code, to achieve a 3-line upgrade.

In light of these experiences, I have adopted an alternative approach. I utilise Zustand or even fall back to internal useState for the majority of trivial cases. This enables me to create small, self-contained stores that are inherent to the wrapper as much as possible. This also ensures that there is no leakage of state outside of the React component that is driving the state.

In order to address the issue of absent remote control functionality, I have adopted an RPC approach (also known as a subscription pattern). This involves the backend broadcasting a message via web socket to indicate a change, after which the React component is responsible for subscribing to this event and triggering the relevant callbacks internally.

If we wish to have an idea of the frontend state in the backend, the backend component can listen for those same events and have the same state representation located on the backend. As it is all JS, it would be straightforward to copy the code, but maintaining it in sync over time is likely to be challenging

p.s.: In terms of the state transitions themselves, the same overall issue persists. When the designers and project leads are only capable of understanding the most basic "click menu here and it opens page there" concepts of junior level PHP applications - there is no need for state transitions from the home page to the level 1 subpage; it can be implemented as a simple array.

State transitions are likely only to be applicable in the case of well-matured applications when upgrading. These are applications that undergo a complete rewrite by a developer who has in-depth knowledge of the subject matter and is able to conduct an exhaustive analysis of the existing application, identifying all states and implementing a state chart accordingly.

[coolsoftwaretyler]

Hey folks,

This is less of an issue and more of a discussion. I'm going to move it to GitHub discussions which will preserve the thoughts there, keep it discoverable, and allow people to chime in from time to time with additional thoughts and questions.

Thanks for all the time, and if any of y'all are still interested in contributing to MobX-State-Tree, we are still around and happy to field PRs and such!