Stores as web workers?

[richardbutler]

I haven't dug into the actual codebase itself all that much before posting this, but what would be involved to allow store handler functions to be async? This would allow stores to be handed off to their own threads, as Web Workers, for complex reduce logic.

For example, I have a huge stack of financial data, fetched into a worker thread. I repeatedly ask the worker to reduce this (fixed) data for me depending on various filtering parameters (downsampling that data for a time series graph, for instance). However, I can't pass the data back from the main thread to the worker each time as there is too much transfer overhead, so I need to keep the original unfiltered state in the worker.

Is this something middleware could support, or does that lie further up the chain?

[gaearon]

Yeah, I think it's doable, probably via higher-order stores (more powerful than middleware).

Example of HOS for Redux DevTools (soon to be published): https://github.com/gaearon/redux/blob/e019f334ebfda3a99ba8bb7377b8496e059d594a/examples/counter/redux-devtools/index.js

The idea is that HOSs have createStore -> createStore signature. applyMiddleware is a HOS itself.
We'll have more docs later, but in the meantime:

https://github.com/gaearon/redux/blob/cdaa3e81ffdf49e25ce39eeed37affc8f0c590f7/docs/higher-order-stores.md

[tappleby]

Here's a prototype I was playing around with last week, the web worker is created in the middleware which is probably the wrong spot, since you have no way to terminate it: https://gist.github.com/tappleby/342754c4a3fe02d82ec0

Depending on your stack you might be able to use RxJs + redux-rx

[gaearon]

which is probably the wrong spot, since you have no way to terminate it

Yeah as soon as you want some control, HOS is a better fit than middleware.
With HOS, declaration would look like

const composedCreateStore = compose(
  applyMiddleware(
    thunkMiddleware, 
    dispatchIdMiddleware, 
    () => promiseMiddleware, 
    loggerMiddleware
  ),
  webWorker('/build/worker.js'), // higher-order store
  createStore
);

This would allow you to grab something like store.webWorkerStore that processes “lifted” actions like START or STOP, but store would keep looking as a “normal” Store to the app.

I'm sorry I can't explain more right now, but you can read devTools HOS for more information. Here's a smaller persistState HOS (which needs to be a HOS because it may hijack the initial state).

[richardbutler]

No need to apologise, great to see it's already in the pipeline - HOS look very powerful. As we're just passing messages around as actions, web workers should fit nicely into the pipe.

I guess the additional complexity is figuring out when all "async" stores have finished processing the action, so we can update the components. Shouldn't be too problematic with promises though.
Looking at persistState, looks like you already have that covered. 👍

[gaearon]

Let's continue this discussion in #313.