New Signal Primitive

[bowheart]

The current plan is for Zedux v2 to introduce a new signal primitive to Zedux.

Overview

Zedux is already technically a signals implementation, but its atoms have different design goals than signals. Atoms are designed to be fully autonomous state containers - with side effects attached, an optional promise, a store to manage its public state (and potentially more internal stores), and optional exports defining the atom's public API. Signals are designed to be lightning fast, mega lightweight state containers that shuttle updates around as efficiently as possible. Zedux being the multi-paradigm state manager it is should have both.

While Zedux has some advantages over most signals libs, there are some features that all signals libs have as first-class citizens that Zedux can only mimic pretty clumsily right now. The main examples:

• effects with auto-tracked dependencies. In Zedux, you'd have to use an impure atom or selector to mimic these
• mySignal()/mySignal.get() to auto-track dependencies in any reactive context. In Zedux, you have to first get a reference to the atom getters, which is easy in most cases. The main exception is in nested function calls - you have to either pass the atom getters object through the call stack or use ecosystem._evaluationStack.atomGetters.get(myAtom) (assuming you have a reference to the ecosystem).

Zedux has debatably better APIs than signals in these regards:

• computeds/memos. Zedux's selectors are basically these, except just plain functions. And Zedux's composable stores are actually able to "reverse propagate" changes, so they don't have the same need to be readonly as computed signals. This means that Zedux atoms are capable of everything that both signal and computed can do. However, this is a little misleading since you'd have to learn stores too to take advantage of reverse propagation.
• untrack. The means to prevent signals from registering dependencies. Zedux uses the atom getters on the ecosystem - e.g. ecosystem.get(myAtom).
• Zedux stores are capable of being consumed as both streams of state and streams of actions. Signals are just streams of state.
• Zedux tracks the full list of indefinitely nested reasons explaining every state update for every node in the graph.
• Composed stores can "reverse propagate" changes to child stores. Composed signals have to be readonly.

However, Zedux's stores have a number of problems:

• Subscribing directly to a store skips the atom graph. This can lead to "state tearing" in rare cases. While there are easy workarounds, this has always been an annoying design flaw with Zedux.
• Stores rely on the ecosystem's scheduler to properly propagate changes between composed stores. Any stores created outside the ecosystem function slightly differently and have edge cases we've yet to resolve. Since stores are really never created outside atoms, this is a low prio problem.
• Creating composed stores is clumsy right now, requiring both an injectStore with manual createStore call and a store.use call to keep the parent store in sync with child store reference changes.
• All injected stores cause the atom to reevaluate unless subscribe: false is manually specified on child stores (the currently recommended approach).
• Metadata is untyped right now. A simple fix for uncomposed stores, but it's pretty difficult for composed stores because:
• Composed stores can't be fully agnostic about where they are in the store hierarchy - effects subscribers have to use e.g. action.payload.payload.meta to access metadata passed to a grandchild store or action.meta to access metadata passed to itself.
• A store.use call is required to keep unstable child stores up-to-date in a parent store. Plus calling store.use always dispatches an extra, useless prime action that could actually cause problems if another atom updated itself synchronously in response to that action (a pointless situation, but still another annoying design flaw in Zedux).
• Stores are a completely different paradigm from atoms. You have to learn both get(myAtom) and store.getState(), both myAtomInstance.addDependent(cb, options) and store.subscribe({ effect, error, next }). We heavily favor working with atoms directly, so Zedux is even lacking some features when working with stores directly, such as the ability to subscribe to a store in a selector (with atoms, you just get(myAtom)).

Using the new GraphNode class introduced in #114, we should be able to create an official signal primitive for Zedux that has all the capabilities of Zedux's powerful store model and that naturally fixes every one of these problems.

Main API

The simplest signal is created inside an atom via injectSignal:

const counterAtom = atom('counter', () => {
  const signal = injectSignal(0)

  return signal // make this signal _the_ signal controlling this atom's state
})

Signals will replace stores as the atom state controller. Current plan is for Zedux v2 to support both, v3 to switch to signals only (for built-in atom instances - stores can still be used by e.g. a LegacyAtomTemplate).

That means that v2 will have a breaking change where atoms that don't specify a store will create a signal to manage their state, not a store.

Basic usage cheatsheet:

const signal = injectSignal('initial state', { hydrate, reactive })
const signalWithExpensiveInit = injectSignal(() => makeExpensiveState())

signal.get() // reactive! (registers a graph dep when called in reactive contexts, except the atom owning the signal)
signal.getOnce() // non-reactive

signal.set('new state')
signal.set(state => ({ num: 1, str: state })) // function overload

The reactive option replaces injectStore's subscribe option and works the same. To hydrate a signal, either set hydrate: true or set hydrate to a function mapping the atom's transformed (if the atom config specifies hydrate) hydration (if the ecosystem received a hydration for the current atom) to the initial value of the signal.

const signal = injectSignal('default state', { hydrate: true })
const transformedHydration = injectSignal('default state', { hydrate: hydration => hydration.someField })

As with stores, specifying hydrate is optional. If it isn't specified, Zedux will hydrate the signal immediately after the initial evaluation, triggering an extra evaluation unless reactive: false was specified.

Computed Signals

There are no computed signals - selectors already cover it:

const signal = injectSignal('a')
const computedSignal = ({ get }: AtomGetters) => get(signal) + 'b'
// or
const computedSignal2 = () => signal.get() + 'b'

Proxies and Transactions

Instead of store.setStateDeep, signals have signal.mutate:

signal.mutate(deepPartial) // undefined values are ignored and deletion is impossible with this overload
signal.mutate(proxiedState => {
  proxiedState.str = 'mutated state'
})

The new signal.mutate method finally introduces immer-style proxies to Zedux with a naturally opt-in API - just stick to .set if you don't like mutations.

Zedux proxies come supercharged with transactions - every mutation generates a list of ordered add, remove, or update objects that will be accepted natively by upcoming Zedux features to e.g. efficiently synchronize signals between web workers and the main process without sending the full state 😮. Opt in to this efficiency by simply using .mutate.

Observe these transactions by registering event listeners on the signal:

Events

Instead of store.dispatch, signals have signal.send. Instead of store.subscribe, signals have signal.on. Events will also replace all built-in and custom metadata types of stores.

Yep, to attach metadata to an update, "send" an event with it. Just like stores, signals can send metadata either by itself or attached to an update.

Sending events:

// map event names to their payload types (or undefined) for TS support
const signalWithEvents = injectSignal<string, { myEventName: string }>('the state')
signalWithEvents.send('myEventName', 'must be a string')
signalWithEvents.send({ myEventName: 'my payload' }) // object form

// the object form can be used to send multiple events together:
signal.send({ myEventName: 'my payload', [ZeduxBatch]: true }) // the built-in event types are always valid, though some wouldn't make sense

// any events can be sent along with any state update
signal.set('new state', { myEventName: 'my payload', [ZeduxBatch]: true })
signal.mutate(deepPartial, { myEventName: 'my payload' })

The built-in event types include:

• batch - Replaces the current batch meta type. No payload
• change - The most common. Sent when the state changes due to a .set or .mutate call. Essentially replaces the current hydrate action type (can be distinguished from mutate (previously "merge") events by checking if a mutate event was sent alongside the change). The payload is a single, indefinitely-nested EvaluationReason.
• cycle - Sent on life cycle status change. Signals only have Active and Destroyed statuses (TODO: finalize)
• ignore - Replaces the current ignore meta type. No payload. TODO: are we removing this?
• mutate - Replaces the current merge action type

The current delegate, hydrate, inherit, merge, and prime action/meta types of stores will go away. The hydrate action type will now be the assumed default since reducers and .dispatch are gone.

Listening to events:

// `.on` manually registers a graph edge
const cleanup = signal.on('mutate', transactions => {
  transactions[0] // for the above mutation: { path: ['str'], type: 'update', value: 'mutated state' }
})

// all callbacks receive the full events map as their second parameter:
signal.on('eventName', (eventPayload, eventMap) => {});

cleanup() // call the returned cleanup function to remove the graph edge

signal.on('change', (reason, events) => {
  // reason is an indefinitely nested EvaluationReason object
  const { newState, oldState, reasons } = reason

  // events is an object mapping any events sent with the state update to their payloads
  // mutations will always send a `mutate` event with a transactions list
  if (events.mutate) {}

  // the event that triggered this listener is also in the map (though redundant here):
  events.change === reason
})

signal.on('cycle', (newStatus, oldStatus) => {})

// custom events:
signalWithEvents.on('myEventName', theString => {})

// pass no event name to be notified of all events. In this case, the event map is the first param passed to the callback
signal.on((eventMap) => {})

// listening to other built-in types is possible, but probably useless
signal.on(ZeduxBatch, () => {})
signal.on(ZeduxIgnore, () => {})

Mapped Signals

One of the big advantages stores have over signals (and hence one of the reasons we haven't seriously considered switching to signals before) is that they can "reverse propagate" their changes - changing a parent store's state is exactly the same as changing the state of the child store(s) directly. Zedux works out which store is controlling which pieces of state and "delegates" the update to the appropriate store.

Signals are a no-go without similar functionality. The typical model for computed (readonly) signals isn't good enough:

const counter = signal(1)
const doubledCounter = computed(() => counter() * 2)

doubledCounter.get() // 2
doubledCounter.set(4) // error! Signals have no way of knowing that the value of `doubledCounter` originated from the `counter` signal.

An atom should be able to inject any number of signals and then compose them together into a single top-level signal that represents the "public state" of the atom. That's how stores work. It's one of the key ingredients to Zedux's beautifully composable, React-esque architecture.

To this end, we'll introduce a second signal primitive: The mapped signal.

const signalA = injectSignal('a')
const signalB = injectSignal('b')
const signalC = injectSignal('c')

const mappedSignal = injectMappedSignal({
  a: signalA,
  nested: {
    b: signalB,
    c: signalC,
  },
})

mappedSignal.get() // { a: 'a', nested: { b: 'b', c: 'c' } }

// updating this signal delegates the change to all signals affected
mappedSignal.set(state => ({ a: 'aa', ...state })) // only updates signalA
mappedSignal.mutate(state => {
  state.nested.b = 'bb' // only updates signalB
})

return mappedSignal // the mapped signal can then be returned as _the_ signal of the atom instance.

This will give signals every capability they need to replace stores. Plus this API is more succinct than the store equivalent:

const storeA = injectStore('a')
const storeB = injectStore('b')
const storeC = injectAtomInstance(otherAtom).store

const parentStore = injectStore(() => createStore({
  a: signalA,
  nested: {
    b: signalB,
    c: signalC,
  },
}))

// if any stores are unstable references (storeC here), `.use` is required here to prevent the parent store from holding onto dead child stores
parentStore.use({
  a: signalA,
  nested: {
    b: signalB,
    c: signalC,
  },
})

Mapped signals also flatten out events - instead of seeing nested action.payload.payload.meta properties in deeply composed stores, all stores see the same single-level event map - eventMap.eventName.

Not a blocker, but we'd ideally get a TS error when trying to map multiple signals together that have matching event names with non-matching types:

const signalA = injectSignal<number, { conflictingName: string }>(1)
const signalB = injectSignal<number, { conflictingName: number }>(2)

const mappedSignal = injectMappedSignal({
  a: signalA,
  b: signalB, // TypeError (ideally)! `conflictingName: string` isn't assignable to type `conflictingName: number`
})

I know that's possible with a certain format for generics and parameters. I'm not sure if it's possible with the desired API. I'm pretty sure TS would infer the overlapping event payloads as unions and we'd be able to make any calls to .on unable to pass a parameter that satisfies all constraints. Maybe that's good enough?

In this situation, since Zedux will forward all events sent to the mapped signal to every inner signal, those inner signals' event listeners could get events that match the name but not the payload type.

This would probably be very rare in practice and is fixed in either case by either changing the event name on one of the inner signals or by making both payload types match.

No Reducers

Reducers are seriously old and feeling more and more clunky and outdated. We want to get Zedux completely off of them natively. Signals will instigate their demise.

Since reducers are simply pure functions, they'll still be usable manually e.g. when migrating from Redux:

const signal = injectSignal(myReducer(initialState, { type: 'prime' }))

signal.set(state => myReducer(state, { payload, type }))

But of course it'll be recommended to just not. Use atom exports to name state updaters.

How Does This Solve the Problems with Stores?

• Subscribing directly to a store skips the atom graph

Signal instances are graph nodes. The containing atom registers graph edges on all injected signals (that don't specify subscribe: false). Anything that uses the atom's internal signals also registers graph edges on them. The graph now handles all notifications. Nothing can skip it.

• Stores rely on the ecosystem's scheduler to properly propagate changes between composed stores

Signals will also rely on the ecosystem's propagation system. However, mapped (aka composed) signals will not be creatable outside atoms. No inconsistent behavior.

• Creating composed stores is clumsy

injectMappedSignal is an API dedicated specifically for composition. It accepts a signal and always keeps its mapped signals up-to-date automatically. One function call instead of three.

• All injected stores cause the atom to reevaluate

The containing atom registers graph edges on all injected signals. The graph already always prevents dependents from running multiple times from one update.

• Metadata is untyped

Signal events will be fully typed. A signal's event types will be shared with mapped signals that wrap it.

• Composed stores can't be fully agnostic about where they are in the store hierarchy

Events will be flattened in mapped signals - every signal, everywhere in the hierarchy will receive the same event map.

• A store.use call is required to keep unstable child stores up-to-date

injectMappedSignal handles this directly, no extra considerations needed.

• Stores are a completely different paradigm from atoms

Signals are the same paradigm! All Zedux's atom helpers will just work with signals - get(myAtom) and get(mySignal), myAtomInstance.on(event, cb) and mySignal.on(event, cb)

All APIs

All together, this task involves fully creating these exports in the @zedux/atoms package (some to be possibly moved to @zedux/core in v3)

• injectSignal
• injectMappedSignal
• SignalInstance class
• Ecosystem#signal - a method for creating new signal instances scoped to the ecosystem.
• signal factory - whenever we implement standalone signals (see below)

Extra Considerations

Signal Instances Don't Have Templates

Either that or the template will be the initial state or state factory function passed to injectSignal. TODO: decide which. The Template generic will reflect this.

The exception will be for standalone signals, if we make those:

Standalone Signals

This will probably not be part of Zedux v2 initial release, but maybe a minor version afterward. It would be nice to be able to create signals outside of atoms that can be consumed inside or completely outside atoms:

const counterSignal = signal(0)

counterSignal.on('change', ({ newState }) => {})

This gives the core package a use - it can be a tiny, barebones package that package maintainers can use for managing simple internal state. They can then expose a signal which can be consumed in an end-user's atoms.

There will not be composed signals outside atoms. If mapped signals or derivations like selectors are needed, use atoms.

Any standalone signals would have to create a "clone" when used inside atoms. They'd essentially become singleton signal templates mapped to a single signal instance inside the ecosystem. It would be Zedux's job to keep the clone (aka signal instance) in sync with the standalone signal (aka signal template). Signal templates would become the only templates to hold state.

Inline .on Calls

It would be nice if we could call .on inline in an atom state factory:

const counterAtom = atom('counter', () => {
  const signal = injectSignal(0)

  signal.on('change', logChange)

  return signal
})

This would apply to all graph nodes (that's where the on method comes from). Any such .on calls would register a graph edge on the signal and be automatically cleaned up when the atom is destroyed. Only .on calls during initial evaluation would be registered. Calls on subsequent evaluations are ignored.

This is a nice-to-have.

Delegated Events

The current plan is for a mapped signal to forward all events dispatched directly to it (not events created implicitly via delegated set and mutate calls) to all internal signals. This means those inner signals can get events their types don't specify.

This is similar to how it works now and we've found it really isn't a problem. However, it can be annoying remembering to ignore other cases in catch-all handlers:

const signal = injectSignal(0) // no custom events

signal.on(eventMap => {
  // the types for all the built-in events are known:
  if (eventMap.batch) return
  if (eventMap.change) return
  if (eventMap.cycle) return
  if (eventMap.ignore) return

  // mutation is the only known event not handled so far.
  // But we can't assume that `eventMap` has a mutation if we made it here:
  // This signal could receive events we don't know about.
  if (eventMap.mutation) return

  // ignore unknown events
})

Perhaps we should consider a different API that specifies the strings as runtime code - types inferred rather than specified directly. This way Zedux can look at the strings to only forward events to the inner signal(s) that support them.