extend GC over comms protocol #3306
Assignees
Labels
Milestone
Comments
warner
added a commit
that referenced
this issue
Jun 14, 2021
9 tasks
warner
added a commit
that referenced
this issue
Jun 19, 2021
This is the big switch to activate comms GC. All the new code is in
`gc-comms.js`, which creates the "gcKit".
`dispatch()` was changed to assemble GC actions arriving from the kernel into
a single "GC" structure, with all three kinds of actions.
* this anticipates a future kernel change that lumps all three GC-related
dispatch types into a single message, and mirrors the remote protocol
which does exactly that
* we first filter out any kfrefs (vrefs) for meta-objects, like the
"controller" (root object) and the receivers, because we aren't prepared
to drop these
* after all dispatch operations (message send, promise resolution
notification, and all GC operations), we call the new `processGC()` to
perform any necessary GC actiity
`gc-comms.js` follows the same pattern as `delivery.js`:
* two inbound conversion functions (`gcFromKernel` and `gcFromRemote`) to
handle GC requests arriving from either source
* two outbound conversion functions (`gcToKernel` and `gcToRemote`) to emit
syscalls or remote-protocol messages that express GC requests
* the end-of-crank `processGC()` function
* this calls `processMaybeFree()` to check on everything whose refcount
touched zero during the crank
* that returns a set of actions which need to be taken
* the actions are sorted into the kernel or remote that is affected, then
request messages are generated for syscalls and/or transmission
refs #3306
Merged
warner
added a commit
that referenced
this issue
Jun 19, 2021
This is the big switch to activate comms GC. All the new code is in
`gc-comms.js`, which creates the "gcKit".
`dispatch()` was changed to assemble GC actions arriving from the kernel into
a single "GC" structure, with all three kinds of actions.
* this anticipates a future kernel change that lumps all three GC-related
dispatch types into a single message, and mirrors the remote protocol
which does exactly that
* we first filter out any kfrefs (vrefs) for meta-objects, like the
"controller" (root object) and the receivers, because we aren't prepared
to drop these
* after all dispatch operations (message send, promise resolution
notification, and all GC operations), we call the new `processGC()` to
perform any necessary GC actiity
`gc-comms.js` follows the same pattern as `delivery.js`:
* two inbound conversion functions (`gcFromKernel` and `gcFromRemote`) to
handle GC requests arriving from either source
* two outbound conversion functions (`gcToKernel` and `gcToRemote`) to emit
syscalls or remote-protocol messages that express GC requests
* the end-of-crank `processGC()` function
* this calls `processMaybeFree()` to check on everything whose refcount
touched zero during the crank
* that returns a set of actions which need to be taken
* the actions are sorted into the kernel or remote that is affected, then
request messages are generated for syscalls and/or transmission
refs #3306
warner
added a commit
that referenced
this issue
Jun 19, 2021
This is the big switch to activate comms GC. All the new code is in
`gc-comms.js`, which creates the "gcKit".
`dispatch()` was changed to assemble GC actions arriving from the kernel into
a single "GC" structure, with all three kinds of actions.
* this anticipates a future kernel change that lumps all three GC-related
dispatch types into a single message, and mirrors the remote protocol
which does exactly that
* we first filter out any kfrefs (vrefs) for meta-objects, like the
"controller" (root object) and the receivers, because we aren't prepared
to drop these
* after all dispatch operations (message send, promise resolution
notification, and all GC operations), we call the new `processGC()` to
perform any necessary GC actiity
`gc-comms.js` follows the same pattern as `delivery.js`:
* two inbound conversion functions (`gcFromKernel` and `gcFromRemote`) to
handle GC requests arriving from either source
* two outbound conversion functions (`gcToKernel` and `gcToRemote`) to emit
syscalls or remote-protocol messages that express GC requests
* the end-of-crank `processGC()` function
* this calls `processMaybeFree()` to check on everything whose refcount
touched zero during the crank
* that returns a set of actions which need to be taken
* the actions are sorted into the kernel or remote that is affected, then
request messages are generated for syscalls and/or transmission
refs #3306
warner
added a commit
that referenced
this issue
Jun 19, 2021
This is the big switch to activate comms GC. All the new code is in
`gc-comms.js`, which creates the "gcKit".
`dispatch()` was changed to assemble GC actions arriving from the kernel into
a single "GC" structure, with all three kinds of actions.
* this anticipates a future kernel change that lumps all three GC-related
dispatch types into a single message, and mirrors the remote protocol
which does exactly that
* we first filter out any kfrefs (vrefs) for meta-objects, like the
"controller" (root object) and the receivers, because we aren't prepared
to drop these
* after all dispatch operations (message send, promise resolution
notification, and all GC operations), we call the new `processGC()` to
perform any necessary GC actiity
`gc-comms.js` follows the same pattern as `delivery.js`:
* two inbound conversion functions (`gcFromKernel` and `gcFromRemote`) to
handle GC requests arriving from either source
* two outbound conversion functions (`gcToKernel` and `gcToRemote`) to emit
syscalls or remote-protocol messages that express GC requests
* the end-of-crank `processGC()` function
* this calls `processMaybeFree()` to check on everything whose refcount
touched zero during the crank
* that returns a set of actions which need to be taken
* the actions are sorted into the kernel or remote that is affected, then
request messages are generated for syscalls and/or transmission
refs #3306
warner
added a commit
that referenced
this issue
Jun 19, 2021
This is the big switch to activate comms GC. All the new code is in
`gc-comms.js`, which creates the "gcKit".
`dispatch()` was changed to assemble GC actions arriving from the kernel into
a single "GC" structure, with all three kinds of actions.
* this anticipates a future kernel change that lumps all three GC-related
dispatch types into a single message, and mirrors the remote protocol
which does exactly that
* we first filter out any kfrefs (vrefs) for meta-objects, like the
"controller" (root object) and the receivers, because we aren't prepared
to drop these
* after all dispatch operations (message send, promise resolution
notification, and all GC operations), we call the new `processGC()` to
perform any necessary GC actiity
`gc-comms.js` follows the same pattern as `delivery.js`:
* two inbound conversion functions (`gcFromKernel` and `gcFromRemote`) to
handle GC requests arriving from either source
* two outbound conversion functions (`gcToKernel` and `gcToRemote`) to emit
syscalls or remote-protocol messages that express GC requests
* the end-of-crank `processGC()` function
* this calls `processMaybeFree()` to check on everything whose refcount
touched zero during the crank
* that returns a set of actions which need to be taken
* the actions are sorted into the kernel or remote that is affected, then
request messages are generated for syscalls and/or transmission
refs #3306
warner
added a commit
that referenced
this issue
Jun 20, 2021
This is the big switch to activate comms GC. All the new code is in
`gc-comms.js`, which creates the "gcKit".
`dispatch()` was changed to assemble GC actions arriving from the kernel into
a single "GC" structure, with all three kinds of actions.
* this anticipates a future kernel change that lumps all three GC-related
dispatch types into a single message, and mirrors the remote protocol
which does exactly that
* we first filter out any kfrefs (vrefs) for meta-objects, like the
"controller" (root object) and the receivers, because we aren't prepared
to drop these
* after all dispatch operations (message send, promise resolution
notification, and all GC operations), we call the new `processGC()` to
perform any necessary GC actiity
`gc-comms.js` follows the same pattern as `delivery.js`:
* two inbound conversion functions (`gcFromKernel` and `gcFromRemote`) to
handle GC requests arriving from either source
* two outbound conversion functions (`gcToKernel` and `gcToRemote`) to emit
syscalls or remote-protocol messages that express GC requests
* the end-of-crank `processGC()` function
* this calls `processMaybeFree()` to check on everything whose refcount
touched zero during the crank
* that returns a set of actions which need to be taken
* the actions are sorted into the kernel or remote that is affected, then
request messages are generated for syscalls and/or transmission
refs #3306
warner
added a commit
that referenced
this issue
Jun 20, 2021
When the comms vat resolves a promise, it begins a two-part retirement process. Right after it sends the `resolve` message to the remote, it deletes the outbound half of the c-list, because this promise ID is partially retired, and the comms vat won't be referencing it in any outbound messages again. The inbound half is retained until the other side acknowledges the resolution message (i.e. some inbound messages arrives which demonstrates awareness of the `notify`, by including an `ackSeqNum` at least as large as the `seqNum` of the outbound `notify`). This changes the code to retain both sides of the c-list entry until the ack is received. I think this winds up being slightly cleaner, and most importantly it retains our ability to map local-ref to remote-ref, which enables an upcoming change to `deleteRemoteMapping` / `deleteKernelMapping` to only take a single argument (lref, not lref+rret), which is a lot cleaner, and avoids some error cases. refs #3306
warner
added a commit
that referenced
this issue
Jun 20, 2021
This is the big switch to activate comms GC. All the new code is in
`gc-comms.js`, which creates the "gcKit".
`dispatch()` was changed to assemble GC actions arriving from the kernel into
a single "GC" structure, with all three kinds of actions.
* this anticipates a future kernel change that lumps all three GC-related
dispatch types into a single message, and mirrors the remote protocol
which does exactly that
* we first filter out any kfrefs (vrefs) for meta-objects, like the
"controller" (root object) and the receivers, because we aren't prepared
to drop these
* after all dispatch operations (message send, promise resolution
notification, and all GC operations), we call the new `processGC()` to
perform any necessary GC actiity
`gc-comms.js` follows the same pattern as `delivery.js`:
* two inbound conversion functions (`gcFromKernel` and `gcFromRemote`) to
handle GC requests arriving from either source
* two outbound conversion functions (`gcToKernel` and `gcToRemote`) to emit
syscalls or remote-protocol messages that express GC requests
* the end-of-crank `processGC()` function
* this calls `processMaybeFree()` to check on everything whose refcount
touched zero during the crank
* that returns a set of actions which need to be taken
* the actions are sorted into the kernel or remote that is affected, then
request messages are generated for syscalls and/or transmission
refs #3306
warner
added a commit
that referenced
this issue
Jun 20, 2021
This is the big switch to activate comms GC. All the new code is in
`gc-comms.js`, which creates the "gcKit".
`dispatch()` was changed to assemble GC actions arriving from the kernel into
a single "GC" structure, with all three kinds of actions.
* this anticipates a future kernel change that lumps all three GC-related
dispatch types into a single message, and mirrors the remote protocol
which does exactly that
* we first filter out any kfrefs (vrefs) for meta-objects, like the
"controller" (root object) and the receivers, because we aren't prepared
to drop these
* after all dispatch operations (message send, promise resolution
notification, and all GC operations), we call the new `processGC()` to
perform any necessary GC actiity
`gc-comms.js` follows the same pattern as `delivery.js`:
* two inbound conversion functions (`gcFromKernel` and `gcFromRemote`) to
handle GC requests arriving from either source
* two outbound conversion functions (`gcToKernel` and `gcToRemote`) to emit
syscalls or remote-protocol messages that express GC requests
* the end-of-crank `processGC()` function
* this calls `processMaybeFree()` to check on everything whose refcount
touched zero during the crank
* that returns a set of actions which need to be taken
* the actions are sorted into the kernel or remote that is affected, then
request messages are generated for syscalls and/or transmission
refs #3306
warner
added a commit
that referenced
this issue
Jun 20, 2021
This is the big switch to activate comms GC. All the new code is in
`gc-comms.js`, which creates the "gcKit".
`dispatch()` was changed to assemble GC actions arriving from the kernel into
a single "GC" structure, with all three kinds of actions.
* this anticipates a future kernel change that lumps all three GC-related
dispatch types into a single message, and mirrors the remote protocol
which does exactly that
* we first filter out any kfrefs (vrefs) for meta-objects, like the
"controller" (root object) and the receivers, because we aren't prepared
to drop these
* after all dispatch operations (message send, promise resolution
notification, and all GC operations), we call the new `processGC()` to
perform any necessary GC actiity
`gc-comms.js` follows the same pattern as `delivery.js`:
* two inbound conversion functions (`gcFromKernel` and `gcFromRemote`) to
handle GC requests arriving from either source
* two outbound conversion functions (`gcToKernel` and `gcToRemote`) to emit
syscalls or remote-protocol messages that express GC requests
* the end-of-crank `processGC()` function
* this calls `processMaybeFree()` to check on everything whose refcount
touched zero during the crank
* that returns a set of actions which need to be taken
* the actions are sorted into the kernel or remote that is affected, then
request messages are generated for syscalls and/or transmission
refs #3306
warner
added a commit
that referenced
this issue
Jun 20, 2021
When the comms vat resolves a promise, it begins a two-part retirement process. Right after it sends the `resolve` message to the remote, it deletes the outbound half of the c-list, because this promise ID is partially retired, and the comms vat won't be referencing it in any outbound messages again. The inbound half is retained until the other side acknowledges the resolution message (i.e. some inbound messages arrives which demonstrates awareness of the `notify`, by including an `ackSeqNum` at least as large as the `seqNum` of the outbound `notify`). This changes the code to retain both sides of the c-list entry until the ack is received. I think this winds up being slightly cleaner, and most importantly it retains our ability to map local-ref to remote-ref, which enables an upcoming change to `deleteRemoteMapping` / `deleteKernelMapping` to only take a single argument (lref, not lref+rret), which is a lot cleaner, and avoids some error cases. refs #3306
warner
added a commit
that referenced
this issue
Jun 20, 2021
This is the big switch to activate comms GC. All the new code is in
`gc-comms.js`, which creates the "gcKit".
`dispatch()` was changed to assemble GC actions arriving from the kernel into
a single "GC" structure, with all three kinds of actions.
* this anticipates a future kernel change that lumps all three GC-related
dispatch types into a single message, and mirrors the remote protocol
which does exactly that
* we first filter out any kfrefs (vrefs) for meta-objects, like the
"controller" (root object) and the receivers, because we aren't prepared
to drop these
* after all dispatch operations (message send, promise resolution
notification, and all GC operations), we call the new `processGC()` to
perform any necessary GC actiity
`gc-comms.js` follows the same pattern as `delivery.js`:
* two inbound conversion functions (`gcFromKernel` and `gcFromRemote`) to
handle GC requests arriving from either source
* two outbound conversion functions (`gcToKernel` and `gcToRemote`) to emit
syscalls or remote-protocol messages that express GC requests
* the end-of-crank `processGC()` function
* this calls `processMaybeFree()` to check on everything whose refcount
touched zero during the crank
* that returns a set of actions which need to be taken
* the actions are sorted into the kernel or remote that is affected, then
request messages are generated for syscalls and/or transmission
refs #3306
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
What is the problem being solved
Our #3106 GC plan covers inter-vat intra-SwingSet references, but doesn't address inter-SwingSet references (which travel through the comms vat). The next step is to extend the existing
dropExport/retireExport/retireImportplan through the comms vat.From the kernel's point of view, the comms vat is just another vat, so it sends and expects the same GC deliveries/syscalls as any other. From the comms vat's point of view, the kernel is just another remote (albeit much more synchronous, which reduces the management overhead somewhat).
The task is to build refcount tracking into the comms tables, then react to inbound GC messages (from either the kernel or remotes) by changing these refcounts. This may cause the comms vat to emit outbound GC messages (either to the kernel as a syscall, or to another remote as a protocol message).
Design
(theses are just some quick protocol notes, I'll edit them into a proper design later)
isReachableflag to all clist entriesreachableis the sum of theisReachableflags from all importers, plus one for each resolved promise or auxdata that references the objectrecognizableis the count of all importing clist entries, plus resolved promises and auxdatadropExport, it clears theisReachableflag for that importer's clist, which decrements thereachablerefcountreachablehits zero, comms sends adropExportto the exporter and clears theisReachableflag on the exporter's clist entry (again, maybe not really necessary)retireExport, it deletes the importer's clist entry, which decrements therecognizablerefcountrecognizablehits zero, comms sends aretireExportto the exporter and deletes the clist entrydropImport(which will always be from an exporter):dropImportfrom sender-space to local (comms) space, then deletes the exporting c-list entryreachablecount must already be zero, else the sender ofdropImportdid something wrongdropImportto each importer, then deletes their clist entrydropImportto the importerIf comms receives a
retireExportorretireImportfor a ref that is not in the c-list, it should just ignore it. There are two reasons/phases where this might happen. The "normal" one is during a race between the importer sendingretireExportand the exporter sendingretireImport. We could choose to track this race in the same way we handle the retirement of promises:retireExport, and tracks the (sent seqnum, rref) pair in an ordered listretireImportarrives before that point, ignore it: there was a race, no big dealretireImportarrives after that point (i.e. neither the clist nor the orderedretireExport-sent list knows the rref): this is the weird case, we might decide to kill the connection, or log-but-ignore, or just ignoreretireImportThe algorithm for comms is mostly simpler than the kernel because:
maybeFreeKrefsand the durablegcActionsset can both be ephemeralprocessNextGCActioncode that prioritizes one over the other, and actions won't be negated by an earlier re-introductionHowever it's slightly more complex because of the need to distinguish between informed and ignorant inbound messages, and the possibility that we choose to log or kill-connection when a
retireImport/retireExportarrives after we know the race window has closed.The text was updated successfully, but these errors were encountered: