FR: make panel state persist to database

[brettdh]

As described in #1252, panels can sometimes come back with this error:

Data for this panel isn't available anymore. Please reload the page and retry.

This is made more likely by stateless execution environments such as AWS Lambda - if the request that generates the state is handled by a different lambda than the request that fetches the state (which can happen frequently), this error will appear.

We can improve matters by storing panel state in the database and looking it up when the panel JS requests it.

[tim-schilling]

I've started on this and immediately ran into the reason why it doesn't exist. The difficulty is the serialization and deserialization of the toolbar. Since it's all done in memory today, there is none. This might require a bit of refactoring to the application so that the base toolbar and panels have its data in a format that can be serialized.

[tim-schilling]

To get the toolbar to store its state in a persistent manner is likely a large refactor of the application. It may be faster @brettdh for you to create another library that extends the toolbar to export the data or rendered templates on each request.

[brettdh]

@tim-schilling do you know of any other such libraries that extend the toolbar in any way? Not sure where I'd get started without poking into its internals.

[tim-schilling]

I don't know of other third party applications that do something similar. Unfortunately, the toolbar doesn't have any signals to hook into that would make this a bit easier to manage. If you want to avoid that, then I think subclassing DebugToolbar and overriding render_toolbar or monkey-patching render_toolbar are your best bets.

[tim-schilling]

This may be required to support async / channels (#1413). The rational is that channels (ASGI) uses multiple processes when using runserver. This may cause different requests for a specific store id to come up empty as the store is currently in memory. Getting the toolbar to store the data in a more multi-process manner may involve some type of serialization. This serialization could also support storing the data in a database, redis or some other storage backend.