Switch Stream (Video + Screen), best pratice

[Mozurok]

Hello, first of all congratulations for the excellent work that you have been doing Muaz.

I wanted to get a doubt about the stream switch between video and screen.

Because we have problems when we share two video / screen sources from Google Chrome for Mozilla Firefox.

Would you like to know the best way to switch from video to screen, or do we have some way of transmitting the two at the same time from the same source?

When i use:

  openConnection() {
    this.connection.session = {
      audio: true,
      video: true,
      data: true,
    };

    this.connection.sdpConstraints.mandatory = {
      OfferToReceiveAudio: true,
      OfferToReceiveVideo: true,
    };

    this.connection.mediaConstraints = {
      audio: true,
      video: true,
    };

    this.connection.open(this.predefinedRoomId);
    webroomsocketio.joinRoomAtivity(this.predefinedRoomId);
    this.setState({
      connect: true,
    });
  }

after

  shareScreen() {
    this.stopStream(this.state.srcStreamId);
    this.connection.addStream({
      screen: true,
      oneway: true,
    });
    this.setState({
      shareScreen: true,
    });
  }

works normal when they are between the same browsers, however when it is two different browsers it triggers an error
DOMException: "Found multiple different webrtc track ids in m-section 1. The behavior here is undefined."
and does not load the additional stream, just keeps the existing one.

Thank you very much for your attention.

[muaz-khan]

Did you check this demo?

• https://rtcmulticonnection.herokuapp.com/demos/video-and-screen-sharing.html

I'm using replaceTrack to switch between camera and screen.

To support parallel video-tracks, we can either clone camera-track for a secondary stream or use captureStream on a <video> or <canvas> element to capture an empty video track. In this case we will replace only that empty-track whenever needed. This helps us share both camera and screen at the same time.

[matiaslopezd]

If someone need replace tracks like video or audio with this function can do that. I have a class, that is the reason that I have the connection like this.connection.

Obviously need have a screenshare handler for known when screen track change state to ended.

setTrack(track, type) {
  return new Promise((resolve, reject) => {
    // Check track and type are valid
    if (!track || !type) reject(new Error('You don\'t set track or type track.'));
    // Check that track is not ended
    if (track.readyState === 'ended') reject(new Error('You don\'t can\'t replace with an "ended" track.'));
    // Each all participants for send screen track
    const foo = this.connection.getAllParticipants().forEach((pid) => {
      // Get peer connection of every participant
      const { peer } = this.connection.peers[pid];
      // Check that peer have senders or next peer
      if (!peer.getSenders) return;
      // Get all 
      peer.getSenders().forEach((sender) => {
        // Check the sender of peer and track, or next sender
        if (!sender || !sender.track) return;
        //  Get the track type and replace track with the new stream if match
        if (sender.track.kind === type && track) sender.replaceTrack(track);
      });
    });
    // Wait for replace to all peers and ready
    Promise.all(foo).then(() => resolve()).catch(() => reject());
  });
}

[chieuduong-bocasay]

hi @matiaslopezd

The method setTrack for switch camera and microphone ?

I making a function switch microphone.
But the rtcmulticonnection is no more support Change Cameras/Microphone.
https://www.npmjs.com/package/@fi1osof/rtcmulticonnection-v3

Can you help me how can make it ?!

Thanks
Chieu Duong

[AdamWarlock001]

hi i m using RTCMULTICONECCTION but
const constraints = {
"video": {
"facingMode":
{ "ideal": "environment" }
}
};

const stream = await navigator.mediaDevices.getUserMedia(constraints);

it does not work

connection.sdpConstraints.mandatory = {
OfferToReceiveAudio: true,
OfferToReceiveVideo: true,

};

I don't know how to default the rear camera

[matiaslopezd]

@AdamWarlock001 you need check the default deviceId of mobile. I mean, knowing the ID of front camera you can request getUserMedia with deviceId of rear camera.

A not precise example is:

const devices = await navigator.mediaDevices.enumerateDevices();

After this you need filter by kind of device like video, get the deviceId and add to constraints of getUserMedia.

[Karthikdb]

If someone need replace tracks like video or audio with this function can do that. I have a class, that is the reason that I have the connection like this.connection.

Obviously need have a screenshare handler for known when screen track change state to ended.

setTrack(track, type) {
  return new Promise((resolve, reject) => {
    // Check track and type are valid
    if (!track || !type) reject(new Error('You don\'t set track or type track.'));
    // Check that track is not ended
    if (track.readyState === 'ended') reject(new Error('You don\'t can\'t replace with an "ended" track.'));
    // Each all participants for send screen track
    const foo = this.connection.getAllParticipants().forEach((pid) => {
      // Get peer connection of every participant
      const { peer } = this.connection.peers[pid];
      // Check that peer have senders or next peer
      if (!peer.getSenders) return;
      // Get all 
      peer.getSenders().forEach((sender) => {
        // Check the sender of peer and track, or next sender
        if (!sender || !sender.track) return;
        //  Get the track type and replace track with the new stream if match
        if (sender.track.kind === type && track) sender.replaceTrack(track);
      });
    });
    // Wait for replace to all peers and ready
    Promise.all(foo).then(() => resolve()).catch(() => reject());
  });
}

Is this works??