Start MatrixRTC in embedded mode #3679
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Šimon Brandner <[email protected]>
SimonBrandner
requested review from
richvdh and
robintown
and removed request for
a team
robintown
approved these changes
Aug 28, 2023
github-merge-queue bot
pushed a commit
that referenced
this pull request
Sep 12, 2023
* Add hacky option to disable the actual calling part of group calls. So we can try using livekit instead. * Put LiveKit info into the `m.call` state event (#3522) * Put LK info into state Signed-off-by: Šimon Brandner <[email protected]> * Update to the new way the LK service works Signed-off-by: Šimon Brandner <[email protected]> --------- Signed-off-by: Šimon Brandner <[email protected]> * Send 'contentLoaded' event As per comment, so we can start digging ourselves out of the widget API hole we're currently in. * Add comment on updating the livekit service URL * Appease CI on `livekit` branch (#3566) * Update codeowners on `livekit` branch (#3567) * add getOpenIdToken to embedded client backend Signed-off-by: Timo K <[email protected]> * add test and update comment Signed-off-by: Timo K <[email protected]> * Merge `develop` into `livekit` (#3569) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Richard van der Hoff <[email protected]> Co-authored-by: Michael Telatynski <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: RiotRobot <[email protected]> Co-authored-by: Florian Duros <[email protected]> Co-authored-by: Kerry <[email protected]> Co-authored-by: David Baker <[email protected]> Co-authored-by: Erik Johnston <[email protected]> Co-authored-by: Valere <[email protected]> Co-authored-by: Hubert Chathi <[email protected]> Close IDB database before deleting it to prevent spurious unexpected close errors (#3478) Fix export type `GeneratedSecretStorageKey` (#3479) Fix order of things in `crypto-api.ts` (#3491) Fix bug where switching media caused media in subsequent calls to fail (#3489) fixes (#3515) fix the integ tests, where #3509 etc fix the unit tests. fix breakage on node 16 (#3527) Fix an instance of failed to decrypt error when an in flight `/keys/query` fails. (#3486) Fix `TypedEventEmitter::removeAllListeners(void)` not working (#3561) * Revert "Merge `develop` into `livekit`" (#3572) * Don't update calls with no livekit URL & expose method to update it instead and generally simplify a bit: change it to a single string rather than an array of structs. * Fix other instances of passing focusInfo / livekit url * Add temporary setter * WIP refactor for removing m.call events * Always remember rtcsessions since we need to only have one instance * Fix tests * Fix import loop * Fix more cyclic imports & tests * Test session joining * Attempt to make tests happy * Always leave calls in the tests to clean up * comment + desperate attempt to work out what's failing * More test debugging * Okay, so these ones are fine? * Stop more timers and hopefully have happy tests * Test no rejoin * Test malformed m.call.member events * Test event emitting and also move some code to a more sensible place in the file * Test getActiveFoci() * Test event emitting (and also fix it) * Test membership updating & pruning on join * Test getOldestMembership() * Test member event renewal * Don't start the rtc manager until the client has synced Then we can initialise from the state once it's completed. * Fix type * Remove listeners added in constructor * Stop the client here too * Stop the client here also also * ARGH. Disable tests to work out which one is causing the exception * Disable everything * Re-jig to avoid setting listeners in the constructor and re-enable tests * No need to rename this anymore * argh, remove the right listener * Is it this test??? * Re-enable some tests * Try mocking getRooms to return something valid * Re-enable other tests * Give up trying to get the tests to work sensibly and deal with getRooms() returning nothing * Oops, don't enable the ones that were skipped before * One more try at the sensible way * Didn't work, go back to the hack way. * Log when we manage to send the member event update * Support `getOpenIdToken()` in embedded mode (#3676) * Call `sendContentLoaded()` (#3677) * Start MatrixRTC in embedded mode (#3679) * Reschedule the membership event check * Bump widget api version * Add mock for sendContentLoaded() * More log detail * Fix tests and also better assert because the tests were passing undefined which was considered fine because we were only checking for null. * Simplify updateCallMembershipEvent a bit * Split up updateCallMembershipEvent some more * Typo Co-authored-by: Daniel Abramov <[email protected]> * Expand comment * Add comment * More comments * Better comment * Sesson * Rename some variables * Comment * Remove unused method * Wrap updatecallMembershipEvent so it only runs one at a time * Do another update if another one is triggered while the update happens * Make triggerCallMembershipEventUpdate async * Fix test & some missed timer removals * Mark session manager as unstable --------- Signed-off-by: Šimon Brandner <[email protected]> Signed-off-by: Timo K <[email protected]> Co-authored-by: Šimon Brandner <[email protected]> Co-authored-by: Timo K <[email protected]> Co-authored-by: Timo <[email protected]> Co-authored-by: Daniel Abramov <[email protected]>
github-merge-queue bot
pushed a commit
that referenced
this pull request
Oct 31, 2023
* WIP refactor for removing m.call events * Always remember rtcsessions since we need to only have one instance * Fix tests * Fix import loop * Fix more cyclic imports & tests * Test session joining * Attempt to make tests happy * Always leave calls in the tests to clean up * comment + desperate attempt to work out what's failing * More test debugging * Okay, so these ones are fine? * Stop more timers and hopefully have happy tests * Test no rejoin * Test malformed m.call.member events * Test event emitting and also move some code to a more sensible place in the file * Test getActiveFoci() * Test event emitting (and also fix it) * Test membership updating & pruning on join * Test getOldestMembership() * Test member event renewal * Don't start the rtc manager until the client has synced Then we can initialise from the state once it's completed. * Fix type * Remove listeners added in constructor * Stop the client here too * Stop the client here also also * ARGH. Disable tests to work out which one is causing the exception * Disable everything * Re-jig to avoid setting listeners in the constructor and re-enable tests * No need to rename this anymore * argh, remove the right listener * Is it this test??? * Re-enable some tests * Try mocking getRooms to return something valid * Re-enable other tests * Give up trying to get the tests to work sensibly and deal with getRooms() returning nothing * Oops, don't enable the ones that were skipped before * One more try at the sensible way * Didn't work, go back to the hack way. * Log when we manage to send the member event update * Support `getOpenIdToken()` in embedded mode (#3676) * Call `sendContentLoaded()` (#3677) * Start MatrixRTC in embedded mode (#3679) * Reschedule the membership event check * Bump widget api version * Add mock for sendContentLoaded() * Embeded mode pre-requisites Signed-off-by: Šimon Brandner <[email protected]> * Embeded mode E2EE Signed-off-by: Šimon Brandner <[email protected]> * Encryption condition Signed-off-by: Šimon Brandner <[email protected]> * Revert "Embeded mode pre-requisites" This reverts commit 8cd7370. * Get back event type Signed-off-by: Šimon Brandner <[email protected]> fds Signed-off-by: Šimon Brandner <[email protected]> * Change embedded E2EE implementation Signed-off-by: Šimon Brandner <[email protected]> * More log detail * Fix tests and also better assert because the tests were passing undefined which was considered fine because we were only checking for null. * Simplify updateCallMembershipEvent a bit * Split up updateCallMembershipEvent some more * Use `crypto.getRandomValues()` Signed-off-by: Šimon Brandner <[email protected]> * Rename to `membershipToUserAndDeviceId()` Signed-off-by: Šimon Brandner <[email protected]> * Better error Signed-off-by: Šimon Brandner <[email protected]> * Add log line Signed-off-by: Šimon Brandner <[email protected]> * Add comment Signed-off-by: Šimon Brandner <[email protected]> * Send call ID in enc events (also a small refactor) Signed-off-by: Šimon Brandner <[email protected]> * Revert making `joinRoomSession()` async Signed-off-by: Šimon Brandner <[email protected]> * Make `client` `private` again Signed-off-by: Šimon Brandner <[email protected]> * Just use `toString()` Signed-off-by: Šimon Brandner <[email protected]> * Fix `callId` check Signed-off-by: Šimon Brandner <[email protected]> * Fix map Signed-off-by: Šimon Brandner <[email protected]> * Fix map compare Signed-off-by: Šimon Brandner <[email protected]> * Fix emitting Signed-off-by: Šimon Brandner <[email protected]> * Explicit logging Signed-off-by: Šimon Brandner <[email protected]> * Refactor Signed-off-by: Šimon Brandner <[email protected]> * Make `updateEncryptionKeyEvent()` public Signed-off-by: Šimon Brandner <[email protected]> * Only update keys based on others Signed-off-by: Šimon Brandner <[email protected]> * Fix call order Signed-off-by: Šimon Brandner <[email protected]> * Improve logging Signed-off-by: Šimon Brandner <[email protected]> * Avoid races Signed-off-by: Šimon Brandner <[email protected]> * Revert "Avoid races" This reverts commit f65ed72. * Add try-catch Signed-off-by: Šimon Brandner <[email protected]> * Make `updateEncryptionKeyEvent()` private Signed-off-by: Šimon Brandner <[email protected]> * Handle indices and throttling Signed-off-by: Šimon Brandner <[email protected]> * Fix merge mistakes Signed-off-by: Šimon Brandner <[email protected]> * Mort post-merge fixes Signed-off-by: Šimon Brandner <[email protected]> * Split out key generation from key sending And send all keys in a key event (changes the format of the key event) rather than just the one we just generated. * Remember and clear the timeout for the send key event So we don't schedule more key updates if one is already pending. Also don't update the last sent time when we didn't actually send the keys. * Make key event resends more robust * Attempt to make tests pass * crypto wasn't defined at all * Hopefully get interface right * Fix key format on the wire to base64 * Add comment * More standard method order * Rename encryptMedia The js-sdk doesn't do media and therefore doesn't do media encryption * Stop logging encryption keys now * Use regular base64 It's not going in a URL, so no need * Re-add base64url randomstring was using it. Also give it a test. * Add tests for randomstring * Switch between either browser or node crypto Let's see if this will work... * Obviously crypto has already solved this * Some tests for MatrixRTCSession key stuff * Test keys object contents * Change keys event format To move away from m. keys * Test key event retries * Test onCallEncryption * Test event sending & spam prevention * Test event cancelation * Test onCallEncryption called * Some errors didn't have data * Fix binary key comparison & add log line * Fix compare function with undefined values * Remove more key logging * Check content.keys is an array * Check key index & key * Better function name * Tests too --------- Signed-off-by: Šimon Brandner <[email protected]> Co-authored-by: David Baker <[email protected]> Co-authored-by: David Baker <[email protected]>
github-merge-queue bot
pushed a commit
that referenced
this pull request
Nov 7, 2023
* WIP refactor for removing m.call events * Always remember rtcsessions since we need to only have one instance * Fix tests * Fix import loop * Fix more cyclic imports & tests * Test session joining * Attempt to make tests happy * Always leave calls in the tests to clean up * comment + desperate attempt to work out what's failing * More test debugging * Okay, so these ones are fine? * Stop more timers and hopefully have happy tests * Test no rejoin * Test malformed m.call.member events * Test event emitting and also move some code to a more sensible place in the file * Test getActiveFoci() * Test event emitting (and also fix it) * Test membership updating & pruning on join * Test getOldestMembership() * Test member event renewal * Don't start the rtc manager until the client has synced Then we can initialise from the state once it's completed. * Fix type * Remove listeners added in constructor * Stop the client here too * Stop the client here also also * ARGH. Disable tests to work out which one is causing the exception * Disable everything * Re-jig to avoid setting listeners in the constructor and re-enable tests * No need to rename this anymore * argh, remove the right listener * Is it this test??? * Re-enable some tests * Try mocking getRooms to return something valid * Re-enable other tests * Give up trying to get the tests to work sensibly and deal with getRooms() returning nothing * Oops, don't enable the ones that were skipped before * One more try at the sensible way * Didn't work, go back to the hack way. * Log when we manage to send the member event update * Support `getOpenIdToken()` in embedded mode (#3676) * Call `sendContentLoaded()` (#3677) * Start MatrixRTC in embedded mode (#3679) * Reschedule the membership event check * Bump widget api version * Add mock for sendContentLoaded() * Embeded mode pre-requisites Signed-off-by: Šimon Brandner <[email protected]> * Embeded mode E2EE Signed-off-by: Šimon Brandner <[email protected]> * Encryption condition Signed-off-by: Šimon Brandner <[email protected]> * Revert "Embeded mode pre-requisites" This reverts commit 8cd7370. * Get back event type Signed-off-by: Šimon Brandner <[email protected]> fds Signed-off-by: Šimon Brandner <[email protected]> * Change embedded E2EE implementation Signed-off-by: Šimon Brandner <[email protected]> * More log detail * Fix tests and also better assert because the tests were passing undefined which was considered fine because we were only checking for null. * Simplify updateCallMembershipEvent a bit * Split up updateCallMembershipEvent some more * Use `crypto.getRandomValues()` Signed-off-by: Šimon Brandner <[email protected]> * Rename to `membershipToUserAndDeviceId()` Signed-off-by: Šimon Brandner <[email protected]> * Better error Signed-off-by: Šimon Brandner <[email protected]> * Add log line Signed-off-by: Šimon Brandner <[email protected]> * Add comment Signed-off-by: Šimon Brandner <[email protected]> * Send call ID in enc events (also a small refactor) Signed-off-by: Šimon Brandner <[email protected]> * Revert making `joinRoomSession()` async Signed-off-by: Šimon Brandner <[email protected]> * Make `client` `private` again Signed-off-by: Šimon Brandner <[email protected]> * Just use `toString()` Signed-off-by: Šimon Brandner <[email protected]> * Fix `callId` check Signed-off-by: Šimon Brandner <[email protected]> * Fix map Signed-off-by: Šimon Brandner <[email protected]> * Fix map compare Signed-off-by: Šimon Brandner <[email protected]> * Fix emitting Signed-off-by: Šimon Brandner <[email protected]> * Explicit logging Signed-off-by: Šimon Brandner <[email protected]> * Refactor Signed-off-by: Šimon Brandner <[email protected]> * Make `updateEncryptionKeyEvent()` public Signed-off-by: Šimon Brandner <[email protected]> * Only update keys based on others Signed-off-by: Šimon Brandner <[email protected]> * Fix call order Signed-off-by: Šimon Brandner <[email protected]> * Improve logging Signed-off-by: Šimon Brandner <[email protected]> * Avoid races Signed-off-by: Šimon Brandner <[email protected]> * Revert "Avoid races" This reverts commit f65ed72. * Add try-catch Signed-off-by: Šimon Brandner <[email protected]> * Make `updateEncryptionKeyEvent()` private Signed-off-by: Šimon Brandner <[email protected]> * Handle indices and throttling Signed-off-by: Šimon Brandner <[email protected]> * Fix merge mistakes Signed-off-by: Šimon Brandner <[email protected]> * Mort post-merge fixes Signed-off-by: Šimon Brandner <[email protected]> * Split out key generation from key sending And send all keys in a key event (changes the format of the key event) rather than just the one we just generated. * Remember and clear the timeout for the send key event So we don't schedule more key updates if one is already pending. Also don't update the last sent time when we didn't actually send the keys. * Make key event resends more robust * Attempt to make tests pass * crypto wasn't defined at all * Hopefully get interface right * Fix key format on the wire to base64 * Add comment * More standard method order * Rename encryptMedia The js-sdk doesn't do media and therefore doesn't do media encryption * Stop logging encryption keys now * Use regular base64 It's not going in a URL, so no need * Re-add base64url randomstring was using it. Also give it a test. * Add tests for randomstring * Switch between either browser or node crypto Let's see if this will work... * Obviously crypto has already solved this * Some tests for MatrixRTCSession key stuff * Test keys object contents * Change keys event format To move away from m. keys * Test key event retries * Test onCallEncryption * Test event sending & spam prevention * Test event cancelation * Test onCallEncryption called * Better before/after member comparison Only trigger for when members actually join, and just generally make it a bit more understandable. * Rotate per-participant keys when a member leaves With a delay borth before making a new key, to try to batch up multiple people leaving into a single key change, and a delay before actually using the new key to allow time for it to arrive. This increasingly feels like storing our own sender key in the same set is suboptimal because we're starting to have to treat it more & more specially. * Some errors didn't have data * Fix binary key comparison & add log line * Fix compare function with undefined values * Test key rotation * Test caught a merge bug! * The missing word was, 'delay' * More input validation --------- Signed-off-by: Šimon Brandner <[email protected]> Co-authored-by: Šimon Brandner <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR currently has none of the required changelog labels.
Add one of:
T-Deprecation,T-Enhancement,T-Defect,T-Taskto indicate what type of change this is plusX-Breaking-Changeif it's a breaking change.