Closed Captions

Closed Captions

Closed Captions offer a text alternative to spoken dialogue, enabling participants in a meeting to read spoken content. This becomes particularly useful in ensuring accessibility and understanding of the content being discussed during a meeting. During the meeting, closed captions can be enabled or disabled. This functionality is available starting from SDK version 3.10.0.

API Usage Examples for Closed Captions

1. To check if Closed captions are allowed

Before you start using Closed Captions during a meeting, check to make sure they are allowed in the ongoing meeting to avoid any issues.

var isAllowed = call.isClosedCaptionAllowed 

2. To check if Closed captions are enabled

Check if Closed Captions are enabled for the current meeting.

var isEnabled = call.isClosedCaptionEnabled 

3. Toggling Closed Captions

Use a toggle to turn Closed Captions on or off during a meeting. It's an asynchronous process, the CompletionHandler will indicate if the operation was successful or not

Note It is recommended to rely on onClosedCaptionsInfoChanged callback, when captions are enabled to get the latest info changes.

call.toggleClosedCaption(enable: Bool, completionHandler: { isOn in
    print("Setting toggleClosedCaption to: \(enable), operation returned: \(isOn)")
}) 

4. Registering a Listener for Information Changes

To get the latest Closed Captions info for current spoken language and translation language, register a listener on a active call. Make sure for this the captions are allowed and enabled.

call.onClosedCaptionsInfoChanged = { info in
    print(info.canChangeSpokenLanguage)
    print(info.currentSpokenLanguage)
    print(info.currentTranslationLanguage)
    print(info.spokenLanguages)
    print(info.translationLanguages)
} 

5. Retrieve Closed Caption info object.

Use this API to get the Closed Captions info object, which will have information such as the current spoken and translation languages. Captions need to be enabled for using this API.

let ccInfo = call.getClosedCaptionsInfo 

6. Setting spoken language

In instances that demand a change in the spoken language, setCurrentSpokenLanguage() API can be used. However, it's crucial to note that spoken language can only be changed by the host, to ensure controlled communication flow.

let ccInfo = call.setCurrentSpokenLanguage(language: LanguageItem, completionHandler: {
    error in
    print(error)
}) 

7. Setting translation language

This API is used to change the language of closed captions. This is the language in which the text is notified to the application

let ccInfo = call.setCurrentTranslationLanguage(language: LanguageItem, completionHandler: {
    error in
    print(error)
}) 

8. Accessing All Received Closed Captions

Get a full list of all Closed Captions from the start of the meeting to now, letting users see previous captions.

let captionItems = call.getClosedCaptions() 

9. Managing Incoming Closed Captions

Set up a listener to manage and show new Closed Captions as they arrive during a call

call.onClosedCaptionArrived = { caption in
    // Implementation details...
} 

CaptionItem object holds caption text, speaker ID, display name, and an isFinal flag. onCaptionArrived() is triggered initially with each spoken word ('isFinal=false'), and upon the speaker’s pause or stop, it refines the text for accuracy and context, triggering again with 'isFinal=true'. And 'isFinal=true' indicates that the sentence is complete.

For example : Speaker: "Hello developer how are you" will trigger following callbacks

1st callback : hello -> isFinal=false
2nd callback : hello developer -> isFinal=false
3rd callback : hello developer how are -> isFinal=false
4th callback : hello developer how are you and -> isFinal=false (here "and" was false detection)

Speaker took a pause or stopped talking

5th callback : hello developer, how are you ? -> isFinal=true (Normalised statement in terms of context and grammar)

Conclusion

With these API examples, developers can craft an inclusive and enriched user experience, ensuring that communication during meetings is accessible and comprehendible to all participants. Utilizing these API functionalities fosters a supportive and collaborative environment, notably in multilingual and diverse settings.