Proof of Concept: Enhancer Overhaul

[gaearon]

Heavily inspired by #1576 but taken further.
(I temporarily removed JSDoc because everything inside moved.)

The public API for regular consumers stays the same. The public API for people who write store enhancers is simplified.

We add a new concept. I tentatively call it “store base” but its name doesn’t really matter yet. I encourage you to avoid bikeshedding on the name for now, and consider the concept instead.

The “store base” thing is exactly what we currently pass to the middleware. It’s a stripped down version of the store that offers no subscriptions. It’s just { dispatch, getState }. In this PR, we consider Redux if it was implemented around this abstraction.

Since a “store base” doesn’t have a subscription mechanism, a function that creates the store base accepts (reducer, initialState, onChange) as arguments. The mechanism of notifying subscribers is implemented at a higher level by the store.

Making onChange an argument is important for keeping the “store base” a useful composition abstraction. For example, a “store base” enhancer may want to wrap onChange to debounce calls to all store subscribers without knowing anything about the subscriber logic.

I might be wrong but I have a feeling that “store base” enhancers can completely replace store enhancers as the main Redux extension mechanism. Not also do they offer wrapping reducer and initialState, but they also have a number of other benefits that store enhancers don’t:

• They are easier to write because only two methods need to be returned: getState and dispatch.
• Unlike store enhancers, “store base” enhancers don’t need to worry about wrapping replaceReducer or copy-pasting our $$observable implementation since both of those things are implemented at the store level which “seals” the composed store base enhancers.
• “Store base” enhancers that implement batching wouldn’t need to worry about tracking the changes to our change emitter mechanism because they would be able to wrap (to delay or debounce) onChange as they see fit without knowing what’s inside of it.
• If store “seals off” the enhanced “store base” chain as shown in this PR, enhancers won’t be able to put arbitrary methods on the store. (Arguably some enhancers do that today, but the overall system is less fragile if we just disallow this.)
• The INIT action goes through all enhancers and middleware because it is emitted outside the “store base” chain in the store itself. The store has the ability to enforce that the initial action is dispatched synchronously, as it can check the current state right after the first dispatch. (Middleware not dispatched through on init #465)
• This would also leave us with just one API for applying enhancers (the modern one) because the old way would give you a store without a subscribe method.

This doesn’t even quite introduce a new concept, as we already pass a thing with the same API to middleware. We’re just giving it a name (rather than a vague “middleware API”) and using more prominently. Also, it is only relevant to people who write store enhancers.

The only downside I can think of so far is the documentation / tutorial update / ecosystem costs, but luckily we never documented enhancers thoroughly, and we only have a few popular ones. In fact many some enhancers should just work unless they forget to pass all arguments, add custom methods, or specifically handle subscribe (in which case they’d be better off handling onChange instead anyway). And we don’t need to be stuck in an update limbo because we can release a 4.0 alpha that switches enhancer argument to mean a “store base” enhancer, and send some PRs to popular enhancers, and then roll it out when everybody is happy.

Thoughts? I know this is a big change, but we barely had any breaking changes since the release, and if there are no other problems, I feel this moves Redux closer to what I originally tried to pull off with separating dispatching from subscribing when I was first thinking about Redux. I’d be much more comfortable with this store extension mechanism than what we have now, but if I’m wrong please help me understand why.

@acdlite @zalmoxisus @tappleby @tomkis1 @yelouafi @timdorr @lukewestby

[gaearon]

Note that these benefits above are based on my gut feeling. I’ll still need to verify all these problems are actually solved by this abstraction.

[acdlite]

On first glance this seems really great — I like the separation of dispatch logic and subscription logic. But I don't understand this sentence:

This would also leave us with just one API for applying enhancers (the modern one) because the old way would give you a store without a subscribe method.

Why wouldn't one of today's enhancers continue to work the old way with this new createStore(), which has the same signature as before?

[timdorr]

Interesting. So, at it's core, this can be thought of similarly to rearranging a traditional class inheritance. That is, rather than the enhancer extending the Redux "class", Redux inherits from something that implements a particular interface. (Not sure if that analogy is helpful, but it's what springs to mind at the moment).

Or another way to think of it: Inversion of Control (IoC). So, this isn't unprecedented.

[acdlite]

Should we have the "real" store extend the store base here via spread, so that enhancers can add their own properties besides dispatch and getState?

return {
  ...storeBase,
  dispatch,
  getState,
  subscribe,
  // ... and so on
}

E.g. this would allow the observable interface to be safely be implemented as an enhancer. Not that we'd necessarily want to do that — just an example of how alternative event systems* are possible with this new enhancer architecture.

*not that they weren't possible before, but now they're safer

[acdlite]

In other words, I don't see the benefit of "sealing" the enhancers, and it seems like you lose some extensibility as well.

[gaearon]

The problem I tried to solve is that anything other than { getState, dispatch } is hard to compose, and enhancers have no knowledge of the past enhancers. If any enhancer could add a method like subscribe(), it would be a pain to later batch subscriptions in any later enhancer (this is pretty much why https://github.com/tappleby/redux-batched-subscribe is currently using a less performant subscription mechanism which was copy pasted from earlier versions of Redux). If any enhancer could add [$$observable], then any further enhancer changing getState would cause it to report incorrect values because the inner enhancer uses the inner getState.

Sealing off limits the domain of enhancers to:

• Hijacking reducer
• Hijacking initialState
• Hijacking dispatch
• Hijacking getState
• Hijacking onChange

All these things are guaranteed to be orthogonal: it is always possible to write an enhancer that changes some of them and it is guaranteed that this won’t trip up enhancers before or after it.

However as soon as we let enhancers define non-orthogonal methods (such as [$$observable] which depends on subscribe and getState), this introduces subtle failures. If a default enhancer included subscribe, a thing that wants to delay subscribe would have to reimplement its logic.

I feel good that all existing use cases seem covered by (reducer, initialState, onChange) => { dispatch, getState }. I would be wary of making the system more expressive because this caused issues in the past. Most enhancers so far had to work around this freedom rather than enjoy it. Finally, for what it’s worth, it is always possible to really wrap createStore if you really need it. I’m just not convinced this is any close to being a common case for the enhancers I have seen.

[acdlite]

The more I think about this the more I really like this, Dan. I hope we can think of a good name for "store base." In my view, a "store base" is really the meat of Redux, whereas what we now think of as a "store" is a store base plus some subscription stuff that really isn't all that important (other than as an an integration point for projects like React Redux), plus some extra opinions (actions should be serializable, automatic INIT actions) that are advisable for the majority of people, but in the grand scheme of things aren't super crucial either.

[jokeyrhyme]

So, effectively, Redux comes with a default enhancer that adds the subscribe() method.

[acdlite]

@jokeyrhyme Yeah pretty much

[acdlite]

@jokeyrhyme Actually, the way I'd model it in my head is

// Pseudo-code, not literal functions
store = eventSystem(enhancers(storeBase))

where the store base handles state updates and createStore adds an event system.

[johanneslumpe]

I can somewhat understand where the idea about sealing off the base comes from, but I think that this is a bit too opinionated. If someone wants to extend the store with custom methods, they should be able to do so. It might be useful to put a big warning there if you do that, but just disallowing out of the box seems like a harsh change.

[jokeyrhyme]

We have replaceReducer, do you think there is a case for a replaceEnhancer, or otherwise dynamically attaching enhancers after the creation of the store?

It might be nice for performance, for example, to only attach the dev-tools enhancer while the dev-tools panel is visible. Or to be able to instantiate and attach a history enhancer after the creation of the store.

[yelouafi]

I think this is far better than the actual way. Dispatching the INIT action after store creation 'and' using the enhanced dispatch would solve a bunch of startup issues like #1240 or other similar issues.

A small remark on the ActionTypes.INIT. The action is not actually exposed as a public API. Shouldn't be exposed as a kind of 'lifecycle event' ?

For example, to resolve issues like #1240 the middleware should wait first for the INIT action before dispatching its first action.

[zalmoxisus]

Like the concept a lot. Having onChange instead of subscribers will help to avoid a lot of edge cases, like the store being replaced by other enhancers and having the old state in our enhancer's subscriber.

The transition for Chrome extension would be a bit difficult as it is updated automatically, but keeping also old redux-devtools-instrument with deprecation warnings for the earlier Redux versions (probably when the third argument is not defined) should help.

[gaearon]

@acdlite

Why wouldn't one of today's enhancers continue to work the old way with this new createStore(), which has the same signature as before?

Today’s enhancers would work. However we can throw inside createStore if the enhancer returned any other methods than { getState, dispatch }. This way, if the enhancers want to keep working with the modern API (the only one we document right now), they will be forced to drop support for the old API. Since the majority of people now pass enhancers as an argument, enhancers will be pressured to implement compat with 4.x for the modern way.

In other words, today’s 3.x enhancers technically can work on 4.x stores, but as people will ask the authors to make them passable as third arguments, we’ll end up with 4.x-only enhancers that can only be applied as an argument because they only return { dispatch, getState } as enforced by createStore().

[gaearon]

@timdorr

That is, rather than the enhancer extending the Redux "class", Redux inherits from something that implements a particular interface. (Not sure if that analogy is helpful, but it's what springs to mind at the moment).

If we were to draw a class analogy, then Store constructor accepts the enhancer “inheritance chain” as an argument, applies it to the StoreBase, and then contains an instance of FinalStoreBase while implementing a superset of StoreBase API, and delegates to it. It recreates the instance of StoreBase (with the whole chain) during replaceReducer.

[gaearon]

@johanneslumpe

If someone wants to extend the store with custom methods, they should be able to do so.

They are able to, but since this breaks composition of enhancers, they would do it outside our API.
You can always do

store.myCustomWhatever = ...

It just doesn’t need we need to bless this anymore because we found the sweet extension point that solves most of the use cases I have seen in the wild with enhancers.

[gaearon]

@jokeyrhyme

We have replaceReducer, do you think there is a case for a replaceEnhancer, or otherwise dynamically attaching enhancers after the creation of the store?

I think we wouldn’t be able to add this because there is no guarantee that the existing state generated by one enhancer tree would be compatible with the next enhancer tree. (Indeed, in the case of switching DevTools off, it wouldn’t be.) At this point you might as well create the store again directly.

[gaearon]

@yelouafi

A small remark on the ActionTypes.INIT. The action is not actually exposed as a public API. Shouldn't be exposed as a kind of 'lifecycle event'? For example, to resolve issues like #1240 the middleware should wait first for the INIT action before dispatching its first action.

Yeah, I haven’t thought much about this yet. With this model we’re in a much better place to enforce constraints (e.g. we can throw if the middleware failed to synchronously pass INIT down). However we’ll need to think more about what behavior makes sense for the initial actions.

[tomkis]

I definitely like this change, looks like a clear separation of concerns. Tried to think about it in all my enhancers and didn't come with any potential issue.

[gaearon]

By the way this gives us an opportunity to finally pass the state as an argument to the listeners. 😄
We now have a guarantee that it will be the final state provided by enhancers by the time we call them.

[acdlite]

Copying part of my comment above here in case it gets hidden by a new commit:

But part of me does think it would be nice if there were a "core enhancer" that implemented this stuff, and was automatically applied by createStore. Then createStoreBase could be exported separately (contingent on finding a better name for it), which would allow for alternative createStore-like APIs to be experimented on in userland without having to rewrite much of anything.

Another way of putting this is that I think "store base" is a better way for userland to experiment with alternative (perhaps higher-level) APIs, rather than composing createStore. Aside from implementing different event systems (which could still be compatible with React Redux, as long as they expose the same methods), they could also do things like apply default enhancers and so forth in a cleaner, safer way.

[acdlite]

Hey, now we can throw here if the enhancer returns the wrong thing! Sweet

[jridgewell]

With this, initialState will never be garbage collected. You'd either need to move the initial storeBase creation outside of replaceReducer, or null it out after this.

[jimbolla]

So I started cataloging store enhancers with the help of @markerikson. I've got this google spreadsheet in progress. I briefly looked at the code of all of these, to get a feel for what they're doing and how.

I think this PR would make a lot of these enhancers very unhappy. There's a lot of enhancers that add their own properties to the store, or do interesting things with the subscribe mechanics, as examples. Or a lot of them just subscribe. Taking those abilities away means they'll have to forego being a store enhancer and instead do their business outside of createStore, pushing the setup burden into the application code.

[gaearon]

do interesting things with the subscribe mechanics, as examples. Or a lot of them just subscribe.

Wouldn’t this still be possible with custom onChange? As far as I remember, this change doesn’t take away any capabilities from enhancers. It just makes those capabilities easier to implement since instead of shadowing the subscription mechanism you can just call onChange whenever.

[gaearon]

Could I ask you to pick a minimal enhancer example that seems inconvertible to the new approach?

Apart from custom methods/properties (IMO it’s a bad pattern and in this case it’s fine to force users to wrap, since otherwise they’d be looking for those methods/properties in Redux docs).

[Parakleta]

I wasn't sure whether to start a new issue or comment here, but I think what I am struggling with is not possible in the current design of Redux and would need to be added as a 4.0 change. Essentially it is currently not possible to have middlewares emit actions that need to be processed by enhancers higher than them in the chain. The idea of a middleware emitting something that needs pre-processing before hitting the reducers appears to be accepted as a good thing because the middleware enhancer ensures that they are composed in a way that lifts all new actions to the top of the middleware chain. Giving enhancers access to the top-of-chain dispatch method ensures that new actions emitted by other enhancers are run through the full enhancer/middleware chain.

For middleware a distinction is made between next and dispatch, where next pushes the action further down the chain, and dispatch lifts it back up to the top. I would suggest that the new enhancer mechanism proposed here should have in addition to onChange another function such as liftAction which passes actions back up the chain. This is similar to the concept of the final store from #2214.

One additional advantage of this is that middleware no longer needs to be special, and is in fact just another enhancer.

[markerikson]

@Parakleta : I think the difference is that middleware can dispatch back to the start of the middleware pipeline. So, nothing is going outside the bounds of applyMiddleware itself. In addition, it's a deliberate part of the middleware design that they get both next and dispatch as references.

In contrast, enhancers are intended to be layered on top of each other without knowing what's before or after.

Can you give an example use case for when something like "going back to the outermost enhancer" might be necessary?

[Parakleta]

Essentially any time a middleware (or by extension an enhancer) emits a new action (in my case particularly async action emitters) I think those should be subject to the same enhancements as new actions emitted from other sources.

The absence of this ability is what has caused a range of issues to be raised regarding incompatibilities with the router enhancer and other middlewares, makes batching difficult, and I suspect is going to cause problems when redux-observable is converted from a middleware to an enhancer.

The issue is raised in #1051, #1240, and #1294 but kind of left unresolved. Related are all of the issues in other projects that cross-link with those issues. Quite a few of those external projects seem to be hoping that this PR will fix the problem for them.

In the very simplest case if I have a batching enhancer and some middleware if I compose them batch -> middleware then I cannot batch actions emitted by the middleware. If I compose them middleware -> batch then my batched actions must be in a form that can be ignored by the middleware but then after unbatching will not be processed by the middleware. I have seen a workable solution in https://github.com/abc123s/redux-batch-enhancer which unpacks the batched actions in a middleware and brackets them with PUSH/POP actions which a later enhancer uses to enable/disable batching, but this feels clumsier than it need be. With a top level dispatch it just becomes a matter of adding an enhancer with a function like:

var depth = 0;
function dispatch(action) {
    try {
        depth += 1;
        store.dispatch(action);
    } finally {
        depth -= 1;
    }
    if (depth === 0) {
        onChange();
    }
    return action;
}

I don't know. For every use case that I can probably come up with there is probably a work around anyway, but the crux of the issue is I think that the only dispatch available to enhancers is not actually dispatch, but rather next. If enhancers only manipulate actions then this is fine because they do their manipulation and pass it on, but if enhancers are allowed to generate actions then this restriction is not OK. If enhancers are not allowed to generate new actions then middleware shouldn't be an enhancer.

[markerikson]

This has been sitting open forever, and I think we're unlikely to do anything about it in the near future. The PR exists, we can always refer back to it for discussion later. Maybe something to consider for a notional Redux v5 or something.