Streams and Effects

The Webex Meetings Web SDK version 3.0 introduces new APIs designed to facilitate the management of local media streams for audio, video, and screen sharing. This article explains how the Streams object functions with two newly added effects: VirtualBackgroundEffect and NoiseReductionEffect. Those effects and their corresponding methods are available in the plugin-meetings package.

See the [kitchen sink app](https://webex.github.io/webex-js-sdk/samples/browser-plugin-meetings) for examples of the effects.

Streams

The following new Streams classes have been introduced:

1. LocalCameraStream
2. LocalMicrophone
3. LocalDisplayStream
4. LocalSystemAudioStream

Import the Streams classes into your applications using the plugin-meetings package:

import {
  LocalCameraStream,
  LocalMicrophoneStream,
  LocalDisplayStream,
  LocalSystemAudioStream,
  createCameraStream,
  createMicrophoneStream,
  createDisplayStream,
  createDisplayStreamWithAudio
} from '@webex/plugin-meetings';

Create Streams

To create a stream, camera, microphone, or display, invoke the respective method while passing any required parameters:

const cameraStream = await createCameraStream(cameraConstraints);

const microphoneStream = await createMicrophoneStream(audioConstraints);

const [localShareVideoStream, localShareAudioStream] = await createDisplayStreamWithAudio();

Here are the supported constraints:

const cameraConstraints = {
    deviceId?: ConstrainDOMString;
    width?: ConstrainULong;
    height?: ConstrainULong;
    aspectRatio?: ConstrainDouble;
    frameRate?: ConstrainDouble;
    facingMode?: ConstrainDOMString;
};

const audioConstraints = {
    deviceId?: string;
    autoGainControl?: boolean;
    echoCancellation?: boolean;
    noiseSuppression?: boolean;
};

Virtual Background Effect

The virtual background can take the form of an image, an MP4 video, or a user's actual background with an applied blur. The blur option offers additional flexibility including adjustable strength and quality levels.

Higher blur settings will put a greater computational demand on your computer.

Use the Meetings object's createVirtualBackgroundEffect() method to apply effects:

await webex.meetings.createVirtualBackgroundEffect(options);

Asynchronous	Yes
Parameters	options Name Description Values Required generator Determines where the model runs (on the main thread or a background thread). string local worker Optional. Defaults toworker. frameRate Determines how many frames are sent to the model per second. number 0-60 Optional. Defaults to 30. quality Determines the accuracy of the model (higher accuracy requires additional CPU resources). string LOW, MEDIUM, HIGH, ULTRA Optional. Defaults to LOW. mirror Whether the output image should be flipped horizontally. boolean Optional. Defaults to false. mode Determines the type of background to render behind a user. string BLUR, IMAGE, VIDEO Optional. Defaults to BLUR. blurStrength Determines how strongly the background is blurred. string WEAK, MODERATE, STRONG, STRONGER, STRONGEST Required in BLURmode. Defaults to STRONG. bgImageUrl Path to a background image that replaces the original background. String Fully qualified URL. Required in IMAGEmode. bgVideoUrl Path to a background video that replaces the original background. String. Fully qualified URL (MP4 only). Required in VIDEOmode
Returns	A promise that returns the virtual background effect.

Apply a Virtual Background Effect

To apply a virtual background effect to your camera Stream:

1. Create a local camera stream with any desired constraints:

   const cameraConstraints = { width: 640, height: 480 };
   
   const cameraStream = await createCameraStream(cameraConstraints);

2. Add the camera stream to the video srcObject:

   meetingStreamsLocalVideo.srcObject = cameraStream.outputStream;

3. Create the virtual background effect using the createVirtualBackgroundEffect() method:

   const effect = await webex.meetings.createVirtualBackgroundEffect();

   Since no parameters are specified, the BLUR background effect is created.

4. After creating the background effect, use the cameraStream object's addEffect() method to apply the effect to the camera stream:

   const effect = await cameraStream.addEffect(effect);

5. Enable the effect to observe it in action on the video:

   await effect.enable();

Apply the Noise Reduction Effect

The noise reduction effect is designed to eliminate background noise from an audio stream, ensuring clear audio during calls. To apply this effect, call the Meetings object's createNoiseReductionEffect() method with any desired options:

await webex.meetings.createNoiseReductionEffect(options);

Asynchronous	Yes
Parameters	options Description Values Required audioContext An optional AudioContext for custom behavior. For example: const audioContext = new AudioContext({sampleRate: 48000}); Object No mode Determines whether to run in WORKLET mode or LEGACY mode for older browsers. string WORKLET LEGACY Defaults to WORKLET.
Returns	A promise that returns the noise reduction effect.

Apply the Noise Reduction Effect

To apply the noise reduction effect to a microphone stream:

const microphoneStream = await createMicrophoneStream();

const effect = await webex.meetings.createNoiseReductionEffect();

meetingStreamsLocalAudio.srcObject = microphoneStream.outputStream;

const effect = await microphoneStream.addEffect(effect);

await effect.enable();

Effect Helper Methods

Both effects offer the following helper methods:

• effect.disable() - Disables the effect.
• effect.dispose() - Removes the effect.
• effect.setEnabled(true|false) - Single method to enable or disable the effect. Pass true to enable the effect and false to disable.
• stream.getEffectByKind(effectName) - Return the effect added on the stream (either noise-reduction-effect or virtual-background-effect).
• stream.getEffects() - Retrieve all effects added to the stream.
• stream.disposeEffects() - Remove all effects from the stream.