Introduce MatrixRTCSession lower level group call primitive (#3663)

* Add hacky option to disable the actual calling part of group calls. So we can try using livekit instead. * Put LiveKit info into the `m.call` state event (#3522) * Put LK info into state Signed-off-by: Šimon Brandner <[email protected]> * Update to the new way the LK service works Signed-off-by: Šimon Brandner <[email protected]> --------- Signed-off-by: Šimon Brandner <[email protected]> * Send 'contentLoaded' event As per comment, so we can start digging ourselves out of the widget API hole we're currently in. * Add comment on updating the livekit service URL * Appease CI on `livekit` branch (#3566) * Update codeowners on `livekit` branch (#3567) * add getOpenIdToken to embedded client backend Signed-off-by: Timo K <[email protected]> * add test and update comment Signed-off-by: Timo K <[email protected]> * Merge `develop` into `livekit` (#3569) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Richard van der Hoff <[email protected]> Co-authored-by: Michael Telatynski <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: RiotRobot <[email protected]> Co-authored-by: Florian Duros <[email protected]> Co-authored-by: Kerry <[email protected]> Co-authored-by: David Baker <[email protected]> Co-authored-by: Erik Johnston <[email protected]> Co-authored-by: Valere <[email protected]> Co-authored-by: Hubert Chathi <[email protected]> Close IDB database before deleting it to prevent spurious unexpected close errors (#3478) Fix export type `GeneratedSecretStorageKey` (#3479) Fix order of things in `crypto-api.ts` (#3491) Fix bug where switching media caused media in subsequent calls to fail (#3489) fixes (#3515) fix the integ tests, where #3509 etc fix the unit tests. fix breakage on node 16 (#3527) Fix an instance of failed to decrypt error when an in flight `/keys/query` fails. (#3486) Fix `TypedEventEmitter::removeAllListeners(void)` not working (#3561) * Revert "Merge `develop` into `livekit`" (#3572) * Don't update calls with no livekit URL & expose method to update it instead and generally simplify a bit: change it to a single string rather than an array of structs. * Fix other instances of passing focusInfo / livekit url * Add temporary setter * WIP refactor for removing m.call events * Always remember rtcsessions since we need to only have one instance * Fix tests * Fix import loop * Fix more cyclic imports & tests * Test session joining * Attempt to make tests happy * Always leave calls in the tests to clean up * comment + desperate attempt to work out what's failing * More test debugging * Okay, so these ones are fine? * Stop more timers and hopefully have happy tests * Test no rejoin * Test malformed m.call.member events * Test event emitting and also move some code to a more sensible place in the file * Test getActiveFoci() * Test event emitting (and also fix it) * Test membership updating & pruning on join * Test getOldestMembership() * Test member event renewal * Don't start the rtc manager until the client has synced Then we can initialise from the state once it's completed. * Fix type * Remove listeners added in constructor * Stop the client here too * Stop the client here also also * ARGH. Disable tests to work out which one is causing the exception * Disable everything * Re-jig to avoid setting listeners in the constructor and re-enable tests * No need to rename this anymore * argh, remove the right listener * Is it this test??? * Re-enable some tests * Try mocking getRooms to return something valid * Re-enable other tests * Give up trying to get the tests to work sensibly and deal with getRooms() returning nothing * Oops, don't enable the ones that were skipped before * One more try at the sensible way * Didn't work, go back to the hack way. * Log when we manage to send the member event update * Support `getOpenIdToken()` in embedded mode (#3676) * Call `sendContentLoaded()` (#3677) * Start MatrixRTC in embedded mode (#3679) * Reschedule the membership event check * Bump widget api version * Add mock for sendContentLoaded() * More log detail * Fix tests and also better assert because the tests were passing undefined which was considered fine because we were only checking for null. * Simplify updateCallMembershipEvent a bit * Split up updateCallMembershipEvent some more * Typo Co-authored-by: Daniel Abramov <[email protected]> * Expand comment * Add comment * More comments * Better comment * Sesson * Rename some variables * Comment * Remove unused method * Wrap updatecallMembershipEvent so it only runs one at a time * Do another update if another one is triggered while the update happens * Make triggerCallMembershipEventUpdate async * Fix test & some missed timer removals * Mark session manager as unstable --------- Signed-off-by: Šimon Brandner <[email protected]> Signed-off-by: Timo K <[email protected]> Co-authored-by: Šimon Brandner <[email protected]> Co-authored-by: Timo K <[email protected]> Co-authored-by: Timo <[email protected]> Co-authored-by: Daniel Abramov <[email protected]>
matrix-org · Sep 12, 2023 · 6836720 · 6836720
1 parent 6f51747
commit 6836720
Show file tree

Hide file tree

Showing 18 changed files with 1,516 additions and 20 deletions.
diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS
@@ -1,6 +1 @@
-*                         @matrix-org/element-web
-/.github/workflows/**     @matrix-org/element-web-app-team
-/package.json             @matrix-org/element-web-app-team
-/yarn.lock                @matrix-org/element-web-app-team
-/src/webrtc               @matrix-org/element-call-reviewers
-/spec/*/webrtc            @matrix-org/element-call-reviewers
+* @matrix-org/element-call-reviewers
diff --git a/package.json b/package.json
@@ -62,7 +62,7 @@
         "jwt-decode": "^3.1.2",
         "loglevel": "^1.7.1",
         "matrix-events-sdk": "0.0.1",
-        "matrix-widget-api": "^1.5.0",
+        "matrix-widget-api": "^1.6.0",
         "oidc-client-ts": "^2.2.4",
         "p-retry": "4",
         "sdp-transform": "^2.14.1",

diff --git a/spec/test-utils/webrtc.ts b/spec/test-utils/webrtc.ts
@@ -485,6 +485,7 @@ export class MockCallMatrixClient extends TypedEventEmitter<EmittedEvents, Emitt
 
     public getRooms = jest.fn<Room[], []>().mockReturnValue([]);
     public getRoom = jest.fn();
+    public getFoci = jest.fn();
 
     public supportsThreads(): boolean {
         return true;

diff --git a/spec/unit/embedded.spec.ts b/spec/unit/embedded.spec.ts
@@ -23,7 +23,14 @@ limitations under the License.
 // eslint-disable-next-line no-restricted-imports
 import { EventEmitter } from "events";
 import { MockedObject } from "jest-mock";
-import { WidgetApi, WidgetApiToWidgetAction, MatrixCapabilities, ITurnServer, IRoomEvent } from "matrix-widget-api";
+import {
+    WidgetApi,
+    WidgetApiToWidgetAction,
+    MatrixCapabilities,
+    ITurnServer,
+    IRoomEvent,
+    IOpenIDCredentials,
+} from "matrix-widget-api";
 
 import { createRoomWidgetClient, MsgType } from "../../src/matrix";
 import { MatrixClient, ClientEvent, ITurnServer as IClientTurnServer } from "../../src/client";
@@ -33,6 +40,12 @@ import { MatrixEvent } from "../../src/models/event";
 import { ToDeviceBatch } from "../../src/models/ToDeviceMessage";
 import { DeviceInfo } from "../../src/crypto/deviceinfo";
 
+const testOIDCToken = {
+    access_token: "12345678",
+    expires_in: "10",
+    matrix_server_name: "homeserver.oabc",
+    token_type: "Bearer",
+};
 class MockWidgetApi extends EventEmitter {
     public start = jest.fn();
     public requestCapability = jest.fn();
@@ -49,8 +62,15 @@ class MockWidgetApi extends EventEmitter {
     public sendRoomEvent = jest.fn(() => ({ event_id: `$${Math.random()}` }));
     public sendStateEvent = jest.fn();
     public sendToDevice = jest.fn();
+    public requestOpenIDConnectToken = jest.fn(() => {
+        return testOIDCToken;
+        return new Promise<IOpenIDCredentials>(() => {
+            return testOIDCToken;
+        });
+    });
     public readStateEvents = jest.fn(() => []);
     public getTurnServers = jest.fn(() => []);
+    public sendContentLoaded = jest.fn();
 
     public transport = { reply: jest.fn() };
 }
@@ -285,7 +305,12 @@ describe("RoomWidgetClient", () => {
             expect(await emittedSync).toEqual(SyncState.Syncing);
         });
     });
-
+    describe("oidc token", () => {
+        it("requests an oidc token", async () => {
+            await makeClient({});
+            expect(await client.getOpenIdToken()).toStrictEqual(testOIDCToken);
+        });
+    });
     it("gets TURN servers", async () => {
         const server1: ITurnServer = {
             uris: [

diff --git a/spec/unit/matrixrtc/CallMembership.spec.ts b/spec/unit/matrixrtc/CallMembership.spec.ts
@@ -0,0 +1,139 @@
+/*
+Copyright 2023 The Matrix.org Foundation C.I.C.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+import { MatrixEvent } from "../../../src";
+import { CallMembership, CallMembershipData } from "../../../src/matrixrtc/CallMembership";
+
+const membershipTemplate: CallMembershipData = {
+    call_id: "",
+    scope: "m.room",
+    application: "m.call",
+    device_id: "AAAAAAA",
+    expires: 5000,
+};
+
+function makeMockEvent(originTs = 0): MatrixEvent {
+    return {
+        getTs: jest.fn().mockReturnValue(originTs),
+        sender: {
+            userId: "@alice:example.org",
+        },
+    } as unknown as MatrixEvent;
+}
+
+describe("CallMembership", () => {
+    it("rejects membership with no expiry", () => {
+        expect(() => {
+            new CallMembership(makeMockEvent(), Object.assign({}, membershipTemplate, { expires: undefined }));
+        }).toThrow();
+    });
+
+    it("rejects membership with no device_id", () => {
+        expect(() => {
+            new CallMembership(makeMockEvent(), Object.assign({}, membershipTemplate, { device_id: undefined }));
+        }).toThrow();
+    });
+
+    it("rejects membership with no call_id", () => {
+        expect(() => {
+            new CallMembership(makeMockEvent(), Object.assign({}, membershipTemplate, { call_id: undefined }));
+        }).toThrow();
+    });
+
+    it("rejects membership with no scope", () => {
+        expect(() => {
+            new CallMembership(makeMockEvent(), Object.assign({}, membershipTemplate, { scope: undefined }));
+        }).toThrow();
+    });
+
+    it("uses event timestamp if no created_ts", () => {
+        const membership = new CallMembership(makeMockEvent(12345), membershipTemplate);
+        expect(membership.createdTs()).toEqual(12345);
+    });
+
+    it("uses created_ts if present", () => {
+        const membership = new CallMembership(
+            makeMockEvent(12345),
+            Object.assign({}, membershipTemplate, { created_ts: 67890 }),
+        );
+        expect(membership.createdTs()).toEqual(67890);
+    });
+
+    it("computes absolute expiry time", () => {
+        const membership = new CallMembership(makeMockEvent(1000), membershipTemplate);
+        expect(membership.getAbsoluteExpiry()).toEqual(5000 + 1000);
+    });
+
+    it("considers memberships unexpired if local age low enough", () => {
+        const fakeEvent = makeMockEvent(1000);
+        fakeEvent.getLocalAge = jest.fn().mockReturnValue(3000);
+        const membership = new CallMembership(fakeEvent, membershipTemplate);
+        expect(membership.isExpired()).toEqual(false);
+    });
+
+    it("considers memberships expired when local age large", () => {
+        const fakeEvent = makeMockEvent(1000);
+        fakeEvent.getLocalAge = jest.fn().mockReturnValue(6000);
+        const membership = new CallMembership(fakeEvent, membershipTemplate);
+        expect(membership.isExpired()).toEqual(true);
+    });
+
+    it("returns active foci", () => {
+        const fakeEvent = makeMockEvent();
+        const mockFocus = { type: "this_is_a_mock_focus" };
+        const membership = new CallMembership(
+            fakeEvent,
+            Object.assign({}, membershipTemplate, { foci_active: [mockFocus] }),
+        );
+        expect(membership.getActiveFoci()).toEqual([mockFocus]);
+    });
+
+    describe("expiry calculation", () => {
+        let fakeEvent: MatrixEvent;
+        let membership: CallMembership;
+
+        beforeEach(() => {
+            // server origin timestamp for this event is 1000
+            fakeEvent = makeMockEvent(1000);
+            // our clock would have been at 2000 at the creation time (our clock at event receive time - age)
+            // (ie. the local clock is 1 second ahead of the servers' clocks)
+            fakeEvent.localTimestamp = 2000;
+
+            // for simplicity's sake, we say that the event's age is zero
+            fakeEvent.getLocalAge = jest.fn().mockReturnValue(0);
+
+            membership = new CallMembership(fakeEvent!, membershipTemplate);
+
+            jest.useFakeTimers();
+        });
+
+        afterEach(() => {
+            jest.useRealTimers();
+        });
+
+        it("converts expiry time into local clock", () => {
+            // for sanity's sake, make sure the server-relative expiry time is what we expect
+            expect(membership.getAbsoluteExpiry()).toEqual(6000);
+            // therefore the expiry time converted to our clock should be 1 second later
+            expect(membership.getLocalExpiry()).toEqual(7000);
+        });
+
+        it("calculates time until expiry", () => {
+            jest.setSystemTime(2000);
+            expect(membership.getMsUntilExpiry()).toEqual(5000);
+        });
+    });
+});