Using a shared Worker (instead of SharedWorker)

[rhashimoto]

TL;DR It's possible to share an OPFS database connection across multiple browser tabs. There's a toy demo page to show how this could be done, working around the lack of OPFS in SharedWorker. I think this will turn out to be the best way to use OPFS.

Earlier I outlined an access handle pooling approach to take advantage of the new more synchronous access handles of the Origin Private File System API (OPFS). The result is a completely synchronous VFS that doesn't need Asyncify or SharedArrayBuffer to bridge to asynchronous calls. A proof-of-concept implementation works quite well - I think this is as fast as OPFS can go, which is also faster than IndexedDB except for write transaction overhead and perhaps on concurrent reads. The main drawback of this VFS is it only supports one database connection, and while multiple browser tabs could detach and reattach their VFS cooperatively, that would be expensive, probably prohibitively expensive. What can we do if we want to support access from multiple contexts?

An alternative way to support multiple tabs accessing the same database without detach/attach on every transaction is to multiplex requests from tabs onto a single connection, i.e. handling arbitration at the application level instead of inside SQLite. The question is where would the database live? Ideally the database could go into a SharedWorker but unfortunately that won't work because OPFS FileSystemSyncAccessHandle isn't available in SharedWorker. In order to share access to an OPFS-based SQLite database, the database has to live in a regular Worker belonging to one of the tabs.

Sharing a Worker on one tab with other same-origin tabs seems like a real headache. How do you decide which tab is the one that exposes its Worker? What happens if the user closes that tab? Fortunately, however, getting this to work turns out to be easier than expected if you take advantage of some of the newer browser APIs.

There's a proof of concept for sharing a simple arithmetic service across multiple tabs without putting the service in SharedWorker at this link (source). Try it by opening the page on a few browser tabs. The first tab you open will initially be designated as the service provider. If you close or reload that tab, another one will automatically become the provider. Clicking the buttons will make calls to the service from the clicked tab to the provider tab (including when both are the same tab). Use the slow operation button to verify that overlapping calls works, and that an exception is thrown if the service moves during a call (a real app would retry on this exception). Going from this toy page to a shared database connection shouldn't be difficult (proof left as an exercise for the reader demos).

How does this work? It does actually use a small SharedWorker to pass MessagePorts between tabs, but it is not involved with calls to the service (an alternative implementation replacing SharedWorker with a service worker is linked later in this thread). Web Locks turn out to be a great way to watch the lifetime of a tab and initiate the migration of the service to a new tab. And BroadcastChannel tells all the tabs to execute that migration. Those mechanics are only a couple hundred lines of code, and that includes providing a (simplified) Comlink-style proxy so the application doesn't have to deal with raw messages.

The OPFS access handle pooling approach now works on all the major browsers. With this option to support multiple browser tabs, I think this is going to be the best way to implement an OPFS-backed database. IndexedDB will likely remain superior for applications prioritizing lower write transaction overhead, the option to trade durability for performance, concurrent reads, and browser support. Everything else can enjoy OPFS faster I/O, no Asyncify, no COOP/COEP headers, and never invalidating the read cache.

[tantaman]

Wow, this is great work.

The main drawback of that approach is it only supports one database connection

I think the biggest drawback here is related to transaction management. You'd need to do some locking to exclude one tab (or worker) from starting a transaction while another tab has an open transaction. If one tab has a long running transaction then that could starve other tabs or workers.

Each worker/tab having their own connection allows long-ish read transactions to at least time-slice their reads so each reader can jointly make progress. A transaction locking mechanism over a single connection would preclude this.

Maybe this drawback isn't that big of a deal though. Were you thinking of other downsides to only having a single DB connection?

I've been assuming that many connections doesn't really buy much in the browser since:

• Many connections to the same DB from the same thread can't improve concurrency given the WASM instance is constrained to running in a single thread.
• In some not very in-depth testing, spinning up many instances of the WASM module in many threads and connecting to the same DB file in each, reads appeared to be doing time-slicing rather than being truly concurrent.

[rhashimoto]

I think the biggest drawback here is related to transaction management. You'd need to do some locking to exclude one tab (or worker) from starting a transaction while another tab has an open transaction.

Yes, that's correct - the application has to arbitrate access. I don't think this is very difficult, though. I think you can approach it in one of two ways:

1. Each call to the database service consists of a complete transaction (or multiple complete transactions), and the service manages a request queue and processes them sequentially, or
2. By convention, each tab acquires a Web Lock to make calls to the database service, and each "session" begins with a command to rollback any incomplete write transaction in a previous session.

If one tab has a long running transaction then that could starve other tabs or workers.

That's true, but it's only a difference with read transactions. Write transactions will block other tabs even with multiple connections.

Each worker/tab having their own connection allows long-ish read transactions to at least time-slice their reads so each reader can jointly make progress. A transaction locking mechanism over a single connection would preclude this.

I think you would be able to do this time slicing with a variant on (2) above, where readers acquire the Web Lock in shared mode. This would put the granularity at a statement instead of a transaction.

Were you thinking of other downsides to only having a single DB connection?

Technically, the lack of read concurrency is a limitation of OPFS access handles - only one context at a time can have an access handle to a file. So even with multiple connections you would have to work around that, e.g. the original OPFS VFS I wrote uses the File interface for read transactions because of this. But I don't know where the breakeven point is between slower reads that allow concurrency and faster reads that don't. I suspect that more people will be happier with the latter.

There are a couple more issues (at least) that need to be considered with this approach:

• When you submit a write transaction and an exception occurs because the service migrated to another tab, you won't actually know if the transaction reached the database or not (e.g. a tab could die right before or after finishing a commit). So simply resubmitting the transaction may not produce the correct result. Care must be taken either to make each write transaction idempotent or to produce a side effect that can be checked for conditional resubmission.

• Only database state will be preserved on a migration; connection state will be lost. So something like CREATE TEMPORARY TABLE or ATTACHing an auxiliary database would need to be re-established on migration, either by the new database service provider or by the interested tabs.

In some not very in-depth testing, spinning up many instances of the WASM module in many threads and connecting to the same DB file in each, reads appeared to be doing time-slicing rather than being truly concurrent.

This will depend on the VFS you use. Are you using the stock IDBBatchAtomicVFS class? Currently all the wa-sqlite example VFS classes that use locks do only exclusive locking, i.e. no concurrency, but switching to allow shared locks is a one-line change. If you're already doing this, then it could be a bug - maybe I'm using an IndexedDB "readwrite" transaction somewhere that should be "readonly".

[tantaman]

This will depend on the VFS you use. Are you using the stock IDBBatchAtomicVFS class? Currently all the wa-sqlite example VFS classes that use locks do only exclusive locking

Ah, thanks for pointing that out. I am using the stock IDBBatchAtomicVFS.

[rhashimoto]

Care must be taken either to make each write transaction idempotent or to produce a side effect that can be checked for conditional resubmission.

If application transactions aren't idempotent and there is nothing already in the database to check whether the last transaction attempt committed, it's possible to add that with an extra table:

-- Setup table to record a transaction id for a tab
-- and a trigger that doesn't allow repeating the previous id.
CREATE TABLE IF NOT EXISTS LastTx (tabId PRIMARY KEY, txId);
CREATE TRIGGER IF NOT EXISTS RollbackDuplicateTx
  BEFORE UPDATE ON LastTx
  WHEN NEW.txId=OLD.txId BEGIN SELECT RAISE(ROLLBACK,'dup'); END;

-- Then write transactions could look like this:
BEGIN IMMEDIATE;
INSERT INTO LastTx VALUES('my-random-tab-id', 'some-transaction-id')
  ON CONFLICT(tabId) DO UPDATE SET txId='some-transaction-id';

-- Actual application SQL goes here...
COMMIT;

A verbatim retry on a migration exception should then succeed only if the prior attempt failed and vice versa. I have no experience with creating triggers so this could probably be improved, but that's the basic idea.

[gabrielgrant]

@rhashimoto am i understanding correctly that this last-transaction-id check may break if there are two (or more) transaction write requests in-flight simultaneously? (if both have been written, and then are retried in the same order, they will be applied again, right?)

[rhashimoto]

I'm not sure whether you're talking about two write transactions from one tab or two write transactions from separate tabs.

I wasn't considering an design where write transactions could be streamed so one tab could have multiple transactions in flight - I was assuming a tab would wait for each transaction to complete before submitting the next. But if you did stream transactions, you could use ordered transaction ids (e.g. a counter).

For handling write transactions from multiple tabs, note that the approach above saves a separate transaction id for each tab - one doesn't overwrite the other. So I think this works. Retries across multiple tabs might not happen in the same order as the original tries (and so could produce different database contents), but typically the original order was arbitrary anyway.

I haven't tried this at all, let alone under stress, so I could be wrong.

[rhashimoto]

One of the advantages of multiplexing requests from different contexts onto a single database connection is that you can set PRAGMA locking_mode=exclusive on that connection. This has a couple benefits for performance:

• SQLite doesn't make any per-transaction locking calls to the VFS.
• SQLite knows its read cache remains valid across transactions.

This means that sharing one connection could be faster even for a VFS that would allow multiple connections, depending on your workload. For example, I measured how fast IDBBatchAtomicVFS is on repeating a small write transaction (one update and one insert) as fast as possible on my aging Mac mini:

locking mode	transactions per second
normal	53.6
exclusive	65.8

This is a decent increase in performance. At least part of the reason is that SQLite doesn't need to make a round trip to the file system to make sure that nothing has changed in its cache.

IDBBatchAtomicVFS has a "relaxed" durability mode that lets you trade durability for performance. As a refresher, durability guarantees that once a write transaction completes it is stored on persistent media. Relaxing durability means that in some cases when service is interrupted (e.g. crash, power failure, etc.) one or more of the most recent transactions may be missing on reconnection. Durability is critically important for some applications and not very important for others. How does performance compare if we relax durability on IDBBatchAtomicVFS?

locking mode	transactions per second
normal	133.5
exclusive	1722.2

Wow! What's happening here with exclusive mode? What's happening is we're not waiting for IndexedDB at all. We don't wait for writes because we're allowing relaxed durability, and we don't wait for reads because the data is in the cache. No round trips to storage. Effectively we're streaming data to IndexedDB.

This is a pretty interesting option if your application can tolerate reduced durability. Also note that IDBBatchAtomicVFS should work in a SharedWorker so it wouldn't need to be a migrating service (but it could be in case it doesn't work in SharedWorker for some reason).

[tantaman]

Any thoughts on supporting this setup for Android where SharedWorkers are not yet an option?

One idea I had was to:

1. Use dedicated workers
2. To coordinate between workers, each dedicated worker requests a weblock where the resource name is the db name. Only the tab with the lock gets to proceed. If the tab is closed, the browser ensures another tab immediately takes over.
3. Use a signaling broadcast channel, with some static name, to indicate when a new tab has joined
4. This signaling channel is used to create names for dedicated broadcast channels between the active worker and different tabs. Other workers would record this information as well since they could be promoted in the future.
5. Something around heartbeating and reaping dead broadcast channels that were created in (4) but become irrelevant as tabs close

I worry, however, that a broadcast channel isn't as performant as a MessagePort.

[rhashimoto]

I didn't know SharedWorker doesn't work on Android. That's disappointing.

I think your idea will work. It will send all the database requests and responses to all the participating contexts, so that's potentially a lot of structured cloning. You can implement #​5 by having each tab acquire an exclusive WebLock with its name before it sends the name over the BroadcastChannel to announce itself. Then other tabs can request a shared lock on the tab name to clean up any resources associated with that name. No heartbeat necessary.

An alternative to your idea is to coordinate communication with a service worker. This is the fallback I originally had in mind, though I have not tried it. I think this will allow establishing a MessageChannel between two tabs by forwarding a MessagePort over MessageChannels between each tab and the service worker. It would be more trouble to set up than using SharedWorker but should behave equivalently once the connections are established. I could understand being reluctant to introduce a service worker, though.

[rhashimoto]

An alternative to your idea is to coordinate communication with a service worker. This is the fallback I originally had in mind, though I have not tried it.

@tantaman I tried it. It turns out to be surprisingly easy to use a service worker to establish a MessageChannel between two tabs, as long as one knows the other's clientId (which it can get via BroadcastChannel). It doesn't require adding any state to the service worker, so the ephemeral service worker lifetime isn't a factor at all.

You can play with an alternative implementation of SharedService that replaces SharedWorker with a service worker here. Both SharedService implementations have the same application interface, except for starting the service worker (which is not inside the class because the application might have its own service worker).

[tantaman]

working around the lack of OPFS in SharedWorker

Was this just a case in the past?

navigator.storage.getDirectory().then((opfsRoot) => {
  console.log(opfsRoot);
});

Returns an access handle in a shared worker context in both Firefox and Chrome. Can't seem to find the SharedWorker logs in Safari.

[rhashimoto]

You might be able to get asynchronous handles, but I'm pretty sure synchronous handles (which are the performant ones we're interested in) won't work. The reasoning is that a shared resource shouldn't block.

The spec shows that FileSystemFileHandle is defined on both Window and Worker (which I think includes SharedWorker) but its createSyncAccessHandle() method is only defined on DedicatedWorker.

[tantaman]

ah, right. Makes sense.

[AntonOfTheWoods]

I have this successfully working for the demo worker but would like to use the same system to set up a second, distinct worker ( const worker = new Worker('./second-worker.js', { type: 'module' });). My initial assumption was that if I did something like the following:

  // Connect Worker and SharedService.
  const worker = new Worker('./ahp-worker.js', { type: 'module' });
  const sharedService = new SharedService(SHARED_SERVICE_NAME, async () => {
    const providerPort = await new Promise(resolve => {
      worker.addEventListener('message', event => {
        resolve(event.ports[0]);
      }, { once: true });
      worker.postMessage(null);
    });
    return providerPort;
  });
  sharedService.activate();

  const two = new Worker('./separate-non-sqlite-worker2.js', { type: 'module' });
  const sharedService2 = new SharedService(SHARED_SERVICE_NAME + "2", async () => {
    const providerPort = await new Promise(resolve => {
      two.addEventListener('message', event => {
        resolve(event.ports[0]);
      }, { once: true });
      two.postMessage(null);
    });
    return providerPort;
  });
  sharedService2.activate();

That I would be able to use sharedService2 sharedService2.proxy.aservice("Hello there!");. However, it seems that only the first service activated can actually be used. I thought it might be the BroadcastChannel name so put the initialisation in the constructor this.#clientChannel = new BroadcastChannel('SharedService_' + this.#serviceName); (and also in the lock request) but this doesn't seem to make a difference. Whichever is declared first takes control.

Is this probably something to do with the SharedWorker/ServiceWorker clientId management? Any pointers on how to get multiple workers working together? Thanks!

[AntonOfTheWoods]

Thanks, I spent quite a while on it and there is definitely an issue regarding having the clientId in there more than once for different services. I didn't manage to find a solution though, sorry!

[rhashimoto]

The problem (or at least the first problem) is that every instance of SharedService is acquiring a Web Lock on the context clientId, so all instances after the first will block. It should just take the lock once per context no matter how many instances there are. I'll make that fix.

The lock is used for the service provider to detect the death of a service consumer so it can clean up any connection resources. It would probably be better design to use separate locks, e.g. by sending a lock name in the service request, to allow connections to different services to end independently. I'm not going to make that change though because (1) most service connections will last for the life of the context, (2) this is just a proof of concept, and (3) I'm really lazy.

[AntonOfTheWoods]

I took a few stabs in the dark by appending the service name to the client ID but it broke things so thought I'd ping back here to get an opinion. I'll confess I don't see how to make it possible to just make one service provider able to manage multiple independent services without them being able to start/end independently but either of those does everything I need so if you have time that would be awesome!

[rhashimoto]

Fix inbound landed. There is a simple two-service demo (without SQLite) here (source).

[AntonOfTheWoods]

You are a gentleman and a scholar, fine sir! Thank you! I think this code definitely deserves to migrate to a "neat little typescript lib" and if I have the time at some point will try and make that happen. This project is awesome!

[sampbarrow]

I've built a basic library that does this (for the purposes of using sqlite wasm or wa-sqlite), basically a backend agnostic comlink. Generifying it now and I'll probably just throw it on github open source. One layer does the processing, the other layer negotiates communication channels, so you can plug in a simple tab->shared/dedicated worker messaging channel, a dumb non-optimized broadcast channel or a more advanced channel that manages a migrating worker and creates/closes new channels as needed.

Also supports observables so you can subscribe to database updates. I'm doing this with sqlite and it works great, broadcasts out new query results when something happens to the db, and since the comparison logic is in the worker it doesn't bog down the main thread.

[AntonOfTheWoods]

That sounds awesome. I have been suggesting the electric-sql folks move on from the IDB-only setup they have now, and having a robust system with observables might just well convince them. Please let us know when you have something on gh and I will loudly ping them!

[sampbarrow]

Almost done. It's working great for me, but there's a lot of messy code, channels being left open and things like that which I'm cleaning up now. I can expose a class with methods returning promises and rxjs observables, and call those methods from another tab as if they were local to that thread and it's all seamless.

Here is some architecture, I'm open to feedback if anyone has any.

• At the core of it is a sender/receiver which translates command data into function calls and responses into observables and promises.
• On top of that you have a channel abstraction. A channel is just an observable and and observer for command data to go in and out of. A channel can be backed by anything, a worker onmessage/postmessage, a broadcast channel, a migrating worker, theoretically could even work fine with a local callback. The first couple of implementations end there.

So like I said, comlink, but backend agnostic. You call expose(object) on the worker and wrap(new Worker) on the frontend, literally 2 lines of code.

Migrating workers work like this. I did steal most of this from @rhashimoto (would be great to hear your thoughts by the way) but I left out the service worker part.

1. There's a named "lookup" channel - all other channel names are autogenerated. You can have multiple named lookup channels for multiple migrating services.
2. When a new worker registers (restricted to only one at a time by a web lock), it broadcasts its availability, and the ID of its dedicated registration channel.
3. When a client (ie a tab) starts, it broadcasts a request for worker availability.
4. When a worker and a client become aware of each other, they do a registration process over a the registration channel. The client creates two dedicated broadcast channels for itself, one for commands and one for responses. I did this because I figured it would keep load off of the other tabs if I kept these channels separate. The client also acquires an arbitrary web lock. The IDs of the channels and the web lock are passed to the migrating worker.
5. Once a worker and client are connected they can exchange commands issued by the client, responses to those commands, and the worker can push responses on demand (ie observables). Unsubscription logic and everything is automatic.
6. If a client dies, its web lock dies, and the worker sees that and drops all of its subscriptions.
7. If a worker dies, a new worker broadcasts its availability, all of the clients drop their old connections we start over from number 4 with all the clients. They also resubcribe to their observables on the new worker automatically but they do NOT reissue pending commands where the return types are promises.

I'm relying on the type system to govern the runtime behavior of observables vs promises, I pass a merged observable and promise back from all calls that are delegated to the worker, behavior is determined lazily by how the return object is treated (meaning do you call "then" or "subscribe" on it). It is all typed appropriately using a mapped type over the exposed type so if you follow the type system there shouldn't be any mixups.

Sqlite shared access pool plugs right into this and works fine observables and all. This doesn't solve the sqlite transaction issue automatically but that's pretty basic to do once this is in place, I wanted to leave this portion of it generic. I'll share my sqlite code too but it's really basic compared to this.

There are also a couple of things like automatic debouncing and batching of messages which has helped when I need to call a worker function a few hundred (or thousand) times in a loop, but those are optional.

I'm a little concerned about the performance of broadcast channels, but for me, it seems okay so far, I haven't benchmarked it but it doesnt feel very different than the regular worker I was using before. And this includes me sending 20mb uint8arrays back and forth pretty regularly in what feels like a few milliseconds.

In any case it is MUCH faster than using the other sqlite opfs methods. No comparison. I can open dozens of tabs, all with active observables watching the database, whereas before it would start to choke after a handful.

Oh and re performance - one thing I forgot to mention. It's very easy to automatically drop and resubcribe tabs using the browser's visibility events if you don't want to leave broadcast channels open for every tab. At some point I'll benchmark this and see if that's worth adding as a flag of some sort.

[sampbarrow]

I put up some code at https://github.com/sampbarrow/observable-worker

It's not at all done, but it's working pretty smoothly, ironed out some bugs today. There's some kind of issue where rxjs treats the promise+observables as promises in some cases (happens on my deployment but not my dev instance) which gives a cryptic error and crashes, so I might have to change the structure of the proxy object entirely to be safe. For modularity if I do that I'll add a dummy wrapper so you can swap in a local object if needed.

Chrome seems to be give a small grace period to let a worker finish previous requests before terminating it, so for very short queries, closing the current active worker tab even during an active query in another tab doesn't necessarily fail, which is nice.

I'd like to add the ability to pass worker related params (timeout, autoRetry) on a per-call basis so I'll be implementing that soon.

Usage is simple. For a migrating worker, in your main thread(s), do this:

const service = wrapMigrating<{YourServiceClass}>()

Then you can call service.remote.methodOfYourServiceClassThatReturnsAnObservable().subscribe(value => ...)

In the worker, do this.

exposeMigrating({ target: new YourServiceClass() })

You can pass a { context: string } to both of these calls to specifically name the lookup channel. this way you can have multiple migrating services.

[Matt-TOTW]

Hi rhashimoto, thanks for this demo, it's excellent, but I have a quick question. Could this be done with just broadcast channels? If communication between the main and worker threads was done via broadcast channels, wouldn't that be simpler than setting up the shared worker and passing message ports around? We'd use web locks to establish the Providing tab and only that one would respond to any message requests. I haven't actually implemented my suggestion - I have successfully implemented your solution with some modest modifications and it works great, thanks! 😊 - but I'm just wondering if there is a simpler way to do it. It would also be good as shared workers aren't universally supported.
Am I missing something? Perhaps there's something I haven't thought of that would mean it won't work.
Thanks for reading!

[Matt-TOTW]

I should've read the thread more carefully. I see that tantaman has already raised this and you've looked into it. I will give it a try. Thanks again.

[rhashimoto]

As you have probably read above, it is possible to use a service worker instead of a SharedWorker to establish a MessageChannel between two contexts on separate tabs (or the same tab).

Using only BroadcastChannel to communicate between tabs can work, especially if message traffic is light. For heavier traffic, using MessageChannel can improve performance by ensuring that messages are sent only to contexts that need them.

[Matt-TOTW]

Thanks for the reply. I like the idea of the broadcast channels only approach but I do hear what you are saying in regards to traffic being sent to contexts that don't need them. It seems to me though that I could simply close the connection to the channel on contexts that are not actively using the service (and reopen a connection as needed). This wouldn't work very well if there was an expectation that the un-focused tabs would still be active in some way, but in my use case I can expect that the un-focused tabs would be essentially idle.
I have written a simple implementation and I will put it up somewhere soon in case it's useful for anyone.

[Matt-TOTW]

I ended up going in a slightly different direction. Instead of opening and closing connections to a single channel, I ended up creating a channel for each pair of provider-secondary tabs. So 10 tabs = 10 channels.
There is one more common channel to facilitate setup and provider changes.
This way, each channel should only be occupied with relevant traffic, excepting the common channel.

The main benefit as far as I see it is that it's conceptually a bit simpler and doesn't require a shared or service worker. I don't know about performance difference. From using it I can say that I don't notice a difference, but then in my use case the actual amount of data being transferred across contexts is pretty light - a few hundred KBs at most.

Sorry it's a bit of a mess. I liked the idea of doing it functionally for some reason and I also wanted to keep it sync so that it could be initialised in the global scope without await.

If you have time, please let me know what you think. Ta!

https://github.com/Matt-TOTW/shared-service/blob/master/src/sharedService.ts

[sampbarrow]

My implementation works similarly but also has typescript types and is abstracted so you can also use the same api for dedicated workers. also handles observables. https://github.com/sampbarrow/observable-worker - you are free to use any of the code if you want. i wouldnt say its production ready though.

All of the shared worker stuff is in deprecated.ts (im planning on switching to one of the newer sqlite implementations that can handle multiple tabs on its own).