Code splitting in large webapps

[gaearon]

I've heard from several people they already want to use this in production, which is a pretty crazy idea, if you ask me 😉 . Still, it's better to consider production use cases early. One of such use cases is code splitting.

Large apps don't want to carry all the code in one JS bundle. Webpack and Browserify allow you to split your app into several parts. One core part loads first, the rest is loaded on demand. We want to support this.

Currently Redux forces you to declare all Stores at the root. This is sensible because if you register them lazily as components subscribe to them, some Stores might miss some actions. This is totally not obvious and fragile.

However, defining all Stores at the root robs us of some of the benefits of code splitting, as all Stores will have to be included in the application's entry point bundle.

So here's my proposal. (I haven't thought it through at all; tell me if it's silly.)

I want to support several <Root>s throughout the application. Whenever a new <Root> is mounted under an existing one, instead of initializing a new dispatcher, it adds its stores to the parent dispatcher. They do not receive the actions they missed—fair game IMO.

Open questions:

• Is the Store state ever destroyed?
• Should it be destroyed when a child <Root> unmounts?
• Should it be destroyed when a particular store key is removed from the stores prop?
• Is it time to rename <Root> to something like <Dispatcher>?

I don't know if it's a good design or not, just something to get the ball rolling.
I want to have some consistent state lifecycle story that works with big apps.

I don't like this idea. If the code loads, it should start handling the actions immediately; not when some view mounts. And what if the new view mounts, and then unmounts? Its <Root> will be gone, poof! But we don't want to erase its state.

Perhaps, a better idea is to rely on React! We got <Root> at the top. (Yo, let's call it <Dispatcher> ;-). Okay, so we got <Dispatcher> at the top. And it has a stores prop. (Not in the decorator version, but we're looking at an advanced use case.) And we got React. And React lets you change props. Get it?

// ------
// Core App
// ------

// StoreRegistry.js

let allStores = {};
let emitChange = null;

export function register(newStores) {
  allStores = { ...allStores, ...newStores }
  emitChange(allStores);
}

export function setChangeListener(listener) {
  if (emitChange) throw new Error('Can set listener once.'); // lol
  emitChange = listener;
  emitChange(allStores);
}

// App.js

import { Dispatcher } from 'redux';
import * as StoreRegistry form './stores/registry';
import * as coreStores from './stores/core';

StoreRegistry.register(coreStores);

class App extends Component {
  constructor(props) {
    super(props);
    StoreRegistry.setChangeListener(() => this.handleStoresChange);
  }

  handleStoresChange(stores) {
    if (this.state) {
      this.setState({ stores });
    } else {
      this.state = { stores };
    }
  }

  render() {
    return (
      <Dispatcher stores={this.state.stores}>
        <SomeRootView />
      </Dispatcher>
    );
  }
}


// ------
// Dynamic module loaded with code splitting
// ------

import * as StoreRegistry form './stores/registry';
import * as extraStores from './stores/extraStores';

// Boom! Will register code-splitted stores.
StoreRegistry.register(extraStores);

// Note that we want to register them when the code loads, not when view mounts.
// The view may never mount, but we want them to start listening to actions ASAP.
// Their state is never destroyed (consistent with the code never being unloaded).

Thoughts?

cc @vslinko

[vslinko]

I don't care right now about bundles in my production app so this solution looks good because it should be implemented on app-level.
When I'll think about bundles I'll use this approach.

[jordangarcia]

Having the ability to lazily load stores is pretty nice for code splitting optimizations.

At Optimizely we organize everything in modules. Currently in production we have over 50 modules that are spread out through several JS bundles.

Here is an excerpt from NuclearJS's README:

Modules

The prescribed way of code organization in NuclearJS is to group all stores, actions and getters of the same domain in a module.

Example Module File Structure

For the flux-chat example we will create a chat module that holds all of the domain logic for the chat aspect. For smaller projects there may only need to be one module, but for larger projects using many modules can decouple your codebase and make it much easier to manage.

modules/chat
├── stores/
    └── thread-store.js
    └── current-thread-id-store.js
├── actions.js // exports functions that call flux.dispatch
├── action-types.js // constants for the flux action types
├── getters.js // getters exposed by the module providing read access to module's stores
├── index.js // MAIN ENTRY POINT - facade that exposes a public api for the module
└── tests.js // module unit tests that test the modules stores, getters, and actions

modules/chat/index.js

var flux = require('../../flux')

flux.registerStores({
  currentThreadID: require('./stores/current-thread-id-store'),
  threads: require('./stores/thread-store'),
})

module.exports = {
  actions: require('./actions'),

  getters: require('./getters'),
}

• Modules expose a single public API, the index.js file. It is improper for an outside piece of code to require any file within the module except the index.js file.
• Stores are registered lazily through the module's index.js. This may seem weird at first, but in NuclearJS stores are more of an implementation detail and not ever directly referenceable.
• Data access to the module's store values is done entirely through the getters it exposes. This provides a decoupling between the store implementation and how the outside world references the state that a module manages. A getter is a contract between the outside world and the module that a particular piece of information is accessible. The evaluator of a getter does not care about the underlying store representation.

By grouping everything together in the module and registering stores as a side effect of requiring the modules entry point then components / other parts of the system are free to require whatever modules they need and they are guaranteed that all stores needed will be registered.

This allows us to code split however we'd like which is usually based around components and not flux.

Not saying you should adopt 100% this route but really reinforcing the concept of organizing by domain and not by type.

[jordangarcia]

This is sensible because if you register them lazily as components subscribe to them, some Stores might miss some actions. This is totally not obvious and fragile.

By grouping stores and actions in a module then the dependency between components <==> modues and modules <==> modules is made explicit. If a module needs to invoke another modules actions or wants its store to handle another modules action then it must explicitly require the other module and the stores will always be loaded.

[gaearon]

@jordangarcia, thanks for chiming in! NuclearJS is an inspiration :-)

Yeah. Dividing source by “domains” and registering sources as a side effect of relevant domain root module execution makes sense to me. That's what I suggested in the original post, too. (Although via a slightly different mechanism.)

[emmenko]

<3 this idea!

Another production (not-ready-to-use-yet) use case that could help.
In my current project I have a "modules"-based structure, something like

lib
-- app
-- core
-- modules
-- -- dashboard
-- -- orders

We use DI in all modules (except for core) and inject core into them.
When the app starts, we load all modules (in our case they simply export theirs routes) and dynamically build the application sitemap with the module routes.

Currently we are using a classical Flux implementation, so we have one instance of the dispatcher that we inject in each module. Each module also has its own stores and actions.

Now if I use redux approach here, I would have to do it this way:

• each module also exports its stores, so that the overall top-level root component can gather all of them
• or each module has it's own root component (so stores are not shared outside the module). This also means that there would be multiple root components (which I don't know if it's meant to work like this)

Now, with this proposal, I can imagine to just inject StoreRegistry into the modules and they simply register themselves. Then I would just define root on the application top level.

Am I thinking correct or am I missing some fundamental point? Thanks :)

[jquense]

So this is of interest to me as well as we do a lot of code splitting in our Big App.

We actually have two concerns that are being addressed by our current set up

1. lazy load reducers
2. ensure that a view isn't depending on a state branch that doesn't have its reducer loaded

One, is fairly easy to fix with the current redux api, we just have a createReducer() wrapper that does something like:

 function createReducer(reducer){
  globalReducers[reducer.name] = reducer
  redex.replaceDispatcher(reCreateDispatcher())
  return reducer
}

This has the effect of letting reducers register themselves, a la traditional flux. That isn't partcularly necessary for code splitting but it helps us with number 2 on the list. We created a custom @connect decorator that takes reducers as an input and then only passes that branch to the view via the select function, this requires the dev to import the reducer and pass it to the decorator, which ensures that the reducer is loaded when a view depends on it.

@connect(userReducer)

we could also use reducer string names, but using the module instance lets webpack build the bundles a lot easier.

All in all the setup is working for us, and has the benefit of when you move, rename, or remove a reducer you get a compile time error for views that depend on it. None of this is to suggest that this is a good idea for a redux core api, but I though i'd chime in with our use case.

things that are still annoying:

• composing reducers to avoid waitFor means that you can't lazy load the constituent reducers, which normally doesn't matter unless one store is particularly computationally heavy but is only used in a specific area

[gaearon]

@jquense

These are great points, thanks for raising them. I think it will be easier to brainstorm solutions if somebody's up for making redux-code-splitting-example similar to huge-apps example inside React Router examples folder.

Wanna give it a try?

[jquense]

@gaearon I'd be happy to try. Fair warning it may not happy super quickly. you want a PR or should I just build an example repo?

[gaearon]

A separate repo would be better.

[gaearon]

Superseded by #350.

[gaearon]

FYI today I'd probably code split reducers like this: https://gist.github.com/gaearon/0a2213881b5d53973514

[dferber90]

I've got a working example of reducer splitting at my webapp example in case anyone is interested. Using a different approach in which the innermost route's component must export the complete reducer. That way the reducer is predictable and doesn't depend on browsing history, although that's a detail.

Description of the approach is here.

I've improved that example in the meantime, but haven't pushed the changes yet. The reducer splitting hasn't changed though.