diff --git a/changelog.md b/changelog.md index e0b44bff..4a32f5e2 100644 --- a/changelog.md +++ b/changelog.md @@ -17,13 +17,6 @@ We continuously add new features and enhancements, fix critical bugs, and regula - **Added support for generating Trackers in Spanish Language** (LABS)
[Read more here](/docs/streamingapi/code-snippets/receive-trackers-in-spanish). - -![sdk](/img/sdk-icon.png) -- **Web SDK Availability** (BETA + LABS)
-Availability of web based JavaScript SDK for Streaming API. Some of the features include transcribing real-time audio, connecting, muting/unmutung, reconnecting, subscribing to events and stopping connections. -[Read more here](/docs/web-sdk/overview). - - ### 22 Oct 2021 ![api update](/img/api-update.png) - **Added support for generating Summary for only the new Transcripts of a Conversation** (LABS)
diff --git a/web-sdk/labs-features.md b/web-sdk/labs-features.md deleted file mode 100644 index 936dd867..00000000 --- a/web-sdk/labs-features.md +++ /dev/null @@ -1,158 +0,0 @@ ---- -id: web-sdk-labs -title: Web SDK (Labs) -sidebar_label: Labs Features - ---- - -import Tabs from '@theme/Tabs'; -import TabItem from '@theme/TabItem'; - ---- - - -You can use labs features by setting your basePath in the init call to `https://api-labs.symbl.ai`. - -## New Configurations - -| Parameter | Required | Description | -| -------| ---------- | --------- | -|`disconnectonOnStopRequest` | Optional, default: true | If set to `false` the WebSocket will be set to a non-processing state if the `stop_request` event is set. In this state, the connection can be re-opened if the `start_request` event is sent. If `true` the WebSocket connection will close as normal. -|`disconnectOnStopRequestTimeout` | Optional | Accepts a value of 0 to 1800 seconds. Indicates how long this connection will remain in a non-processing state before timing out. | -|`noConnectionTimeout` | Optional | Accepts a value of 0 to 1800 seconds. Indicates how long a connection will remain active even when no one is connected. By using the same connectionId anyone can reconnect to this WebSocket before it times out completely.| -|`sourceNode` | Optional, default: null | For passing in an external `MediaStreamAudioSourceNode` object. By default the Web SDK will handle audio context and source nodes on it's own, though if you wish to handle that externally we've provided that option.| -|`config.encoding` | Optional, default: 'linear16' | Accepts either 'opus' or 'linear16'. For linear16, you must set the sampleRateHertz option. For opus the sampleRateHertz should always be 48000. | -| `handlers.ondevicechange` | Optional | By default Symbl Web SDK will provide the ondevicehandler logic, which just takes the new device and sends the sample rate over to our servers. If you wish to override this logic you can do so by passing an ondevicechange function into the handlers section of the config. You can assign a function to symbl.deviceChanged as a callback to when the event is fired. -| `reconnectOnError` | Optional, default: true | If true the Web SDK will attempt to reconnect to the WebSocket in case of error. - -### Example - -```js -const id = btoa("symbl-ai-is-the-best"); - -const connectionConfig = { - id, - insightTypes: ['action_item', 'question'], - disonnectOnStopRequest: false, - disconnectOnStopRequestTimeout: 300, - noConnectionTimeout: 300, - sourceNode: sourceNode, - reconnectOnError: true, - config { - encoding: 'opus', - sampleRateHertz: 48000 - }, - handlers: { - ondevicechange: () => { - alert('device changed!'); - }, - ... - } - ... -} - -... - -// Creates the WebSocket in a non-processing state -const stream = await symbl.createStream(connectionConfig); - -// Send the start request -await symbl.unmute(stream); -``` - -### Using createStream to start a Realtime Request -Creating a stream using `symbl.startRealtimeRequest(config)` has been deprecated in favor of `symbl.createStream(config)`. For `createStream`, the WebSocket is started in a non processing state. You must send the start request before processing any audio. - -The `createStream` function returns a stream object. In order to start the connection you can call `symbl.unmute(stream)`. Unmute will send the start request and start the audio streaming. - -### Using Mute/Unmute to Pause a Connection -If you set the `disconnectOnStopRequest` flag to false you can use `symbl.mute(stream)` and `symbl.unmute(stream)` to suspend and resume the connection. Muting the connection makes it so you're not being billed during times of silence. - -#### unmute(stream) -Receive the stream received from createStream as argument. Unmutes the audio by setting gain value to 1. If disconnectOnStopRequest config is set to false the start request will be sent to the Websocket and the audio context will start. - -##### mute(stream) -Receive the stream received from createStream as argument. Mutes the audio by setting gain value to 0. If disconnectOnStopRequest config is set to false the stop will be sent to the Websocket and the audio context will be suspended. - -### Use disconnectOnStopRequest to Pause and Resume a Stream -```js -symbl.init({ - appId: '', - appSecret: '', - // accessToken: '', // can be used instead of appId and appSecret - basePath: 'https://api-labs.symbl.ai', -}); - -const id = btoa("symbl-ai-is-the-best"); - -const connectionConfig = { - id, - insightTypes: ['action_item', 'question'], - disconnectOnStopRequest: false, - disconnectOnStopRequestTimeout: 300, - noConnectionTimeout: 300, - config: { - meetingTitle: 'My Test Meeting ' + id, - confidenceThreshold: 0.7, - timezoneOffset: 480, // Offset in minutes from UTC - languageCode: 'en-US', - encoding: 'opus', - sampleRateHertz: 48000 - }, - speaker: { - // Optional, if not specified, will simply not send an email in the end. - userId: '', // Update with valid email - name: '' - }, - handlers: { - /** - * This will return live speech-to-text transcription of the call. - */ - onSpeechDetected: (data) => { - if (data) { - const {punctuated} = data - console.log('Live: ', punctuated && punctuated.transcript) - console.log(''); - } - // console.log('onSpeechDetected ', JSON.stringify(data, null, 2)); - }, - /** - * When processed messages are available, this callback will be called. - */ - onMessageResponse: (data) => { - // console.log('onMessageResponse', JSON.stringify(data, null, 2)) - }, - /** - * When Symbl detects an insight, this callback will be called. - */ - onInsightResponse: (data) => { - // console.log('onInsightResponse', JSON.stringify(data, null, 2)) - }, - /** - * When Symbl detects a topic, this callback will be called. - */ - onTopicResponse: (data) => { - // console.log('onTopicResponse', JSON.stringify(data, null, 2)) - } - } -}; - -(async () => { - // Creates the WebSocket in a non-processing state - const stream = await symbl.createStream(connectionConfig); - - // Send the start request - await symbl.unmute(stream); - - // Suspend the stream after 10 seconds - window.setTimeout(() => { - await symbl.mute(stream); - }, 10000) - - // Re-send the start request to resume the stream after another 10 seconds - window.setTimeout(() => { - await symbl.unmute(stream); - }, 10000) -})(); - -``` \ No newline at end of file diff --git a/web-sdk/muting-and-unmuting-connected-device.md b/web-sdk/muting-and-unmuting-connected-device.md deleted file mode 100644 index 6c299a41..00000000 --- a/web-sdk/muting-and-unmuting-connected-device.md +++ /dev/null @@ -1,13 +0,0 @@ ---- -id: muting-and-unmuting-connected-device -title: Muting and Unmuting Connected Device (Beta) -sidebar_label: Muting and Unmuting Connected Device - ---- - -import Tabs from '@theme/Tabs'; -import TabItem from '@theme/TabItem'; - ---- - -You can mute and unmute the connected device by simply calling `symbl.mute()` or `symbl.unmute()`. diff --git a/web-sdk/overview.md b/web-sdk/overview.md deleted file mode 100644 index ce5c0030..00000000 --- a/web-sdk/overview.md +++ /dev/null @@ -1,129 +0,0 @@ ---- -id: web-sdk -title: Symbl Web SDK (Beta) -sidebar_label: Introduction -slug: /web-sdk/overview ---- -import Tabs from '@theme/Tabs'; -import TabItem from '@theme/TabItem'; - ---- - -The Symbl Web SDK provides access to the Symbl APIs for applications in the browser directly. - -We have added the following capabilities in the Web SDK: - -- Connecting, -- Stopping, -- Muting, -- Unmuting and -- Subscribing. - -> **Source Code**
-Find the source code here: [https://github.com/symblai/symbl-web-sdk](https://github.com/symblai/symbl-web-sdk).
- - -## Supported Browsers ---- - - |-- | Chrome | Edge Firefox | Firefox | Safari | - | -------| ---------- | ------- | ----- | ------- | - | macOS | ![icon](/img/tick-mark.png)| ![icon](/img/tick-mark.png)| ![icon](/img/tick-mark.png) | ![icon](/img/tick-mark.png) | - | Windows | ![icon](/img/tick-mark.png) | ![icon](/img/tick-mark.png)| ![icon](/img/tick-mark.png) | | - | Linux | ![icon](/img/tick-mark.png)| | ![icon](/img/tick-mark.png) | - | iOS | ![icon](/img/tick-mark.png)| | ![icon](/img/tick-mark.png) | ![icon](/img/tick-mark.png) | - | Android | ![icon](/img/tick-mark.png)| | ![icon](/img/tick-mark.png) | ![icon](/img/tick-mark.png) | - -## Setup ---- -**To use the Symbl Web SDK,** - -1. Include the following script tags in your HTML file: - -```html - -``` -2. In case of a front-end web application using a framework such as React, import it in the ES2015 style, as given below: - -```bash -import symbl from "@symblai/symbl-web-sdk"; -``` -To view a full example that shows the above, check out the links below: - -- HTML sample -- React app sample - -## Initialization ---- -The `init` authenticates you to use the Symbl API using the provided authentication credentials. To get authentication credentials (App ID and Secret), follow the steps given in the [Authentication](/docs/developer-tools/authentication#step-1-get-your-api-credentials) page. - -You can authenticate: - -- [Using your API Credentials](#authenticate-using-api-credentials) - -                or - -- [Using your Auth Token](#authenticate-using-token) - -### Authenticate using API Credentials - -Use the code given below to authenticate using your App ID and App Secret. - -```js -sdk.init({ - // APP_ID and APP_SECRET come from the Symbl Platform: https://platform.symbl.ai - appId: APP_ID, - appSecret: APP_SECRET, - basePath: 'https://api.symbl.ai' -}) -.then(() => console.log('SDK Initialized.')) -.catch(err => console.error('Error in initialization.', err)); - ``` - -### Authenticate using Token - -Use the code given below to authenticate using the Auth Token. To generate the Auth Token follow the Steps given in the [Authentication](/docs/developer-tools/authentication#step-2-generate-the-access-token) Page. - -```js -sdk.init({ - accessToken: ACCESS_TOKEN_HERE, - basePath: 'https://api.symbl.ai' -}) -.then(() => console.log('SDK Initialized.')) -.catch(err => console.error('Error in initialization.', err)); -``` - - -:::note Web SDK in Labs -The Web SDK is also available as a part of [Symbl Labs](/docs/labs) with select features. You can find the Web SDK Labs documentation here: [https://github.com/symblai/symbl-web-sdk/tree/labs](https://github.com/symblai/symbl-web-sdk/tree/labs) and the source code here: [https://github.com/symblai/symbl-web-sdk/tree/labs](https://github.com/symblai/symbl-web-sdk/tree/labs). - -### Features in Labs - -The following features are available in Labs for Web SDK. For more details, go to the [GitHub Readme]([https://github.com/symblai/symbl-web-sdk/tree/labs](https://github.com/symblai/symbl-web-sdk/tree/labs)) -::: - -## Tutorials ---- -We have prepared a list of tutorials to help you understand how to use the Web SDK. - -* [Transcribing Live Audio Input through Microphone](/docs/web-sdk/transcribing-live-audio-through-microphone) - - -### Code Snippets ---- - -* [Subscribe to real-time Events](/docs/web-sdk/subscribe-real-time) -* [Reconnecting to an Existing Real-time Connection](/docs/web-sdk/reconnecting-real-time) -* [Muting and Unmuting the Connected Device](/docs/web-sdk/muting-and-unmuting-connected-device) -* [Stopping Real-time Connection](/docs/web-sdk/stopping-real-time) - - -### Web SDK Reference ---- -The supported events for the Web SDK are listed below: - -* [Event Handlers](/docs/javascript-sdk/reference#event-handlers-1) - * [onSpeechDetected](/docs/javascript-sdk/reference#onspeechdetected) - * [onMessageResponse](/docs/javascript-sdk/reference#onmessageresponse) - * [onInsightResponse](/docs/javascript-sdk/reference#oninsightresponse) - * [onTopicResponse](/docs/javascript-sdk/reference#ontopicresponse) diff --git a/web-sdk/reconnecting-to-existing-realtime-connection.md b/web-sdk/reconnecting-to-existing-realtime-connection.md deleted file mode 100644 index 020d5d35..00000000 --- a/web-sdk/reconnecting-to-existing-realtime-connection.md +++ /dev/null @@ -1,79 +0,0 @@ ---- -id: reconnecting-real-time -title: Reconnecting to an Existing Real-time Connection (Beta) -sidebar_label: Reconnecting to an Existing Real-time Connection -slug: /web-sdk/reconnecting-real-time ---- -import Tabs from '@theme/Tabs'; -import TabItem from '@theme/TabItem'; - ---- - -In the case that a user closes their browser or has an interruption in their WebSocket connection you can use the `store` object to grab the Connection ID you last used. - -```js -const id = symbl.store.get('connectionID'); - -const connectionConfig = { - id, - insightTypes: ['action_item', 'question'], - config: { - meetingTitle: 'My Test Meeting ' + id, - confidenceThreshold: 0.7, - timezoneOffset: 480, // Offset in minutes from UTC - languageCode: 'en-US', - sampleRateHertz: 44100 - }, - speaker: { - // Optional, if not specified, will simply not send an email in the end. - userId: '', // Update with valid email - name: '' - }, - handlers: { - /** - * This will return live speech-to-text transcription of the call. - */ - onSpeechDetected: (data) => { - if (data) { - const {punctuated} = data - console.log('Live: ', punctuated && punctuated.transcript) - console.log(''); - } - // console.log('onSpeechDetected ', JSON.stringify(data, null, 2)); - }, - /** - * When processed messages are available, this callback will be called. - */ - onMessageResponse: (data) => { - // console.log('onMessageResponse', JSON.stringify(data, null, 2)) - }, - /** - * When Symbl detects an insight, this callback will be called. - */ - onInsightResponse: (data) => { - // console.log('onInsightResponse', JSON.stringify(data, null, 2)) - }, - /** - * When Symbl detects a topic, this callback will be called. - */ - onTopicResponse: (data) => { - // console.log('onTopicResponse', JSON.stringify(data, null, 2)) - } - } -}; - -(async () => { - const connection = await symbl.startRealtimeRequest(connectionConfig, true); -})(); -``` - -The `startRealtimeRequest` connects to a Streaming API Web Socket endpoint using the provided configuration options. Read more about `startRealtimeRequest` [here](/docs/web-sdk/web-sdk-reference#startrealtimerequest). - -Read about the Streaming API parameters for `connectionConfig` [here](/docs/streaming-api/api-reference/#request-parameters). - -Read more about the supported Event Handlers: - -    👉   [onSpeechDetected](/docs/web-sdk/web-sdk-reference#onspeechdetected)
-    👉   [onMessageResponse](/docs/web-sdk/web-sdk-reference#onmessageresponse)
-    👉   [onInsightResponse](/docs/web-sdk/web-sdk-reference#oninsightresponse)
-    👉   [onTopicResponse](/docs/web-sdk/web-sdk-reference#ontopicresponse) \ No newline at end of file diff --git a/web-sdk/stopping-realtime-connection.md b/web-sdk/stopping-realtime-connection.md deleted file mode 100644 index a9418d98..00000000 --- a/web-sdk/stopping-realtime-connection.md +++ /dev/null @@ -1,18 +0,0 @@ ---- -id: stopping-real-time -title: Stopping Real-time Connection (Beta) -sidebar_label: Stopping Real-time Connection -slug: /web-sdk/stopping-real-time ---- -import Tabs from '@theme/Tabs'; -import TabItem from '@theme/TabItem'; - ---- - -In order to end the connection to the realtime WebSocket you'll need to use the following command with your connection object: - -```js -symbl.stopRequest(connection); -``` - -It is recommended to always end the connection programmatically if you do not sever the connection as you could end up using more minutes of time than intended. \ No newline at end of file diff --git a/web-sdk/subscribe-real-time.md b/web-sdk/subscribe-real-time.md deleted file mode 100644 index 75c05887..00000000 --- a/web-sdk/subscribe-real-time.md +++ /dev/null @@ -1,36 +0,0 @@ ---- -id: web-subscribe-real-time -title: Subscribing to an Existing Real-time Connection (Beta) -sidebar_label: Subscribe to an Existing Real-time Connection -slug: /web-sdk/subscribe-real-time ---- -import Tabs from '@theme/Tabs'; -import TabItem from '@theme/TabItem'; - ---- - -The Symbl Web SDK lets you subscribe to real-time events when you connect to one of the Endpoints specified in the above sections. -You must open this example in a different browser while the realtime transcription example is running. - -The below example shows how to achieve this: - -```js -symbl.init({ - appId: '', - appSecret: '', - // accessToken: '', // can be used instead of appId and appSecret - // basePath: '', -}); - -const id = btoa("symbl-ai-is-the-best"); - -symbl.subscribeToStream(id, (data) => { - console.log('data:', data); -}) -``` - -The `subscribeToStream` function allows you to subscribe to existing streaming connection in read-only. It takes the following parameters: - -| Parameters | Type | Example | -| ---------- | ------- | ------- | -| `id` | String | Connection ID created on connection `init`| diff --git a/web-sdk/transcribing-live-audio-through-microphone.md b/web-sdk/transcribing-live-audio-through-microphone.md deleted file mode 100644 index 00618aeb..00000000 --- a/web-sdk/transcribing-live-audio-through-microphone.md +++ /dev/null @@ -1,202 +0,0 @@ ---- -id: transcribing-live-audio-through-microphone -title: Transcribing Live Audio through Microphone (Beta) -sidebar_label: Transcribing Live Audio through Microphone - ---- - -import Tabs from '@theme/Tabs'; -import TabItem from '@theme/TabItem'; - ---- - -To get transcriptions live setup a live microphone and push the audio stream using the browser APIs to access the microphone. - -On this page, the following capabilities are explained: - -- Connecting, -- Stopping, -- Muting, -- Unmuting and -- Subscribing. - -:::note -To see the code for Reconnecting function, go to [Reconnect to existing realtime Connection](/docs/web-sdk/reconnecting-real-time) page. -::: - -### Initialize the SDK - -```js -symbl.init({ - appId: '', - appSecret: '', - // accessToken: '', // can be used instead of appId and appSecret - // basePath: '', -}); -``` - -You can get the `appId` and `appSecret` from the [Symbl Platform](https://platform.symbl.ai). -See the steps to get your API Credentials in the [Authentication](/docs/developer-tools/authentication) section. - -### Start the Connection and pass Configuration Options - -:::note -The `symbl.startRealtimeRequest` function creates a new AudioContext, so the call must be made on user interaction, such as a button click. -::: - -#### HTML Sample - -```html - - - - - -​ - - - - -​ - -​ - -
-

Live transcription:

-

-
-​ - - - -``` - -### Reference - -Read about the Streaming API parameters: - -    👉   [insightTypes](/docs/streaming-api/api-reference#main-message-body)
-    👉   [config](/docs/streaming-api/api-reference#config)
-    👉   [meetingTitle](/docs/streaming-api/api-reference#config)
-    👉   [confidenceThreshold](/docs/streaming-api/api-reference#config)
-    👉   [timezoneOffset](/docs/streaming-api/api-reference#config)
-    👉   [languageCode](/docs/streaming-api/api-reference#config)
-    👉   [sampleRateHertz](/docs/streaming-api/api-reference#speech-recognition)
-    👉   [speaker](/docs/streaming-api/api-reference#speaker)
-    👉   [userId](/docs/streaming-api/api-reference#speaker)
-    👉   [name](/docs/streaming-api/api-reference#speaker)
-    👉   [startRealtimeRequest](/docs/web-sdk/web-sdk-reference#startrealtimerequest)
-    👉   [subscribeToStream](/docs/web-sdk/web-sdk-reference#subscribetostream)
- - -Read more about the supported [Event Handlers](/docs/web-sdk/web-sdk-reference#event-handlers): - -    👉   [onSpeechDetected](/docs/web-sdk/web-sdk-reference#onspeechdetected)
-    👉   [onMessageResponse](/docs/web-sdk/web-sdk-reference#onmessageresponse)
-    👉   [onInsightResponse](/docs/web-sdk/web-sdk-reference#oninsightresponse)
-    👉   [onTopicResponse](/docs/web-sdk/web-sdk-reference#ontopicresponse) - - \ No newline at end of file diff --git a/web-sdk/web-sdk-html.md b/web-sdk/web-sdk-html.md deleted file mode 100644 index 81a3cb56..00000000 --- a/web-sdk/web-sdk-html.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -id: web-sdk-html -title: Web SDK HTML sample -slug: /web-sdk/web-sdk-html ---- -import Tabs from '@theme/Tabs'; -import TabItem from '@theme/TabItem'; - ---- diff --git a/web-sdk/web-sdk-reference.md b/web-sdk/web-sdk-reference.md deleted file mode 100644 index bb409079..00000000 --- a/web-sdk/web-sdk-reference.md +++ /dev/null @@ -1,286 +0,0 @@ ---- -id: web-sdk-reference -title: Web SDK Reference -slug: /web-sdk/web-sdk-reference ---- -import Tabs from '@theme/Tabs'; -import TabItem from '@theme/TabItem'; - ---- - - -## Public Methods - -### init - -```init (String appId, String appSecret)``` - -Authenticates with the Symbl API using the provided authentication credentials. - -#### Parameters - -Name | Description ------|------------ -`appId` | The Symbl Application ID you get from the [Symbl Platform](https://platform.symbl.ai) -`appSecret` | The Symbl Application Secret Token you get from the [Symbl Platform](https://platform.symbl.ai) -`basePath` | The base path of the endpoint. By default it is `https://api.symbl.ai`. -`accessToken` | The Symbl authentication Token you get from your `appId` and `appSecret`. This is an optional parameter you can use to authenticate using auth Token rather than the App ID and App Secret. See sample code [here](/docs/javascript-sdk/introduction#authenticate-using-token). - -#### Returns - -A Promise which is resolved once the API is connected and authenticated with Symbl. - -#### Code Example - -```js -sdk.init({ - // APP_ID and APP_SECRET come from the Symbl Platform: https://platform.symbl.ai - appId: APP_ID, - appSecret: APP_SECRET, - basePath: 'https://api.symbl.ai' -}) -.then(() => console.log('SDK Initialized.')) -.catch(err => console.error('Error in initialization.', err)); -``` -### startRealtimeRequest - -```startRealtimeRequest ( options)``` - -Connects to a [Streaming API](/docs/streamingapi/overview/introduction) Web Socket endpoint using the provided configuration options. - -#### Parameters - -Name | Description ------|------------ -`options` | Options specified for the [Streaming API Configuration Object](https://docs.symbl.ai/docs/streaming-api/api-reference#request-parameters). - -#### Returns - -A Promise which is resolved once real-time request has been established. - -## Event Handlers - -When connecting using [`startRealtimeRequest`](#startRealtimeRequest), you can pass various handlers in the configuration options which be called if the specific event attached to the handler is fired. - -#### Code Example - -```js -handlers: { - /** - * This will return live speech-to-text transcription of the call. - */ - onSpeechDetected: (data) => { - console.log(JSON.stringify(data)) - if (data) { - const {punctuated} = data - console.log('Live: ', punctuated && punctuated.transcript) - } - }, - /** - * When processed messages are available, this callback will be called. - */ - onMessageResponse: (data) => { - console.log('onMessageResponse', JSON.stringify(data, null, 2)) - }, - /** - * When Symbl detects an insight, this callback will be called. - */ - onInsightResponse: (data) => { - console.log('onInsightResponse', JSON.stringify(data, null, 2)) - }, - /** - * When Symbl detects a topic, this callback will be called. - */ - onTopicResponse: (data) => { - console.log('onTopicResponse', JSON.stringify(data, null, 2)) - } -} -``` - -### onSpeechDetected - -To retrieve the real-time transcription results as soon as they are detected. You can use this callback to render live transcription which is specific to the speaker of this audio stream. - -#### onSpeechDetected JSON Response Example - -```js -{ - "type": "recognition_result", - "isFinal": true, - "payload": { - "raw": { - "alternatives": [{ - "words": [{ - "word": "Hello", - "startTime": { - "seconds": "3", - "nanos": "800000000" - }, - "endTime": { - "seconds": "4", - "nanos": "200000000" - } - }, { - "word": "world.", - "startTime": { - "seconds": "4", - "nanos": "200000000" - }, - "endTime": { - "seconds": "4", - "nanos": "800000000" - } - }], - "transcript": "Hello world.", - "confidence": 0.9128385782241821 - }] - } - }, - "punctuated": { - "transcript": "Hello world." - }, - "user": { - "userId": "emailAddress", - "name": "John Doe", - "id": "23681108-355b-4fc3-9d94-ed47dd39fa56" - } -} -``` - -### onMessageResponse - -This callback function contains the "finalized" transcription data for this speaker and if used with multiple streams with other speakers this callback would also provide their messages. - -The "finalized" messages mean that the automatic speech recognition has finalized the state of this part of transcription and has declared it "final". Therefore, this transcription will be more accurate than [`onSpeechDetected`](#onspeechdetected). - -#### onMessageResponse JSON Response Example - -```js -[{ - "from": { - "id": "0a7a36b1-047d-4d8c-8958-910317ed9edc", - "name": "John Doe", - "userId": "emailAddress" - }, - "payload": { - "content": "Hello world.", - "contentType": "text/plain" - }, - "id": "59c224c2-54c5-4762-9582-961bf250b478", - "channel": { - "id": "realtime-api" - }, - "metadata": { - "disablePunctuation": true, - "timezoneOffset": 480, - "originalContent": "Hello world.", - "words": "[{\"word\":\"Hello\",\"startTime\":\"2021-02-04T20:34:59.029Z\",\"endTime\":\"2021-02-04T20:34:59.429Z\"},{\"word\":\"world.\",\"startTime\":\"2021-02-04T20:34:59.429Z\",\"endTime\":\"2021-02-04T20:35:00.029Z\"}]", - "originalMessageId": "59c224c2-54c5-4762-9582-961bf250b478" - }, - "dismissed": false, - "duration": { - "startTime": "2021-02-04T20:34:59.029Z", - "endTime": "2021-02-04T20:35:00.029Z" - } -}] -``` - -### onInsightResponse - -This callback provides you with any of the detected insights in real-time as they are detected. As with the [`onMessageCallback`](#onmessagecallback) this would also return every speaker's insights in case of multiple streams. - -#### onInsightResponse JSON Response Example - -```json -[{ - "id": "94020eb9-b688-4d56-945c-a7e5282258cc", - "confidence": 0.9909798145016999, - "messageReference": { - "id": "94020eb9-b688-4d56-945c-a7e5282258cc" - }, - "hints": [{ - "key": "informationScore", - "value": "0.9782608695652174" - }, { - "key": "confidenceScore", - "value": "0.9999962500210938" - }, { - "key": "comprehensionScore", - "value": "0.9983848333358765" - }], - "type": "action_item", - "assignee": { - "id": "e2c5acf8-b9ed-421a-b3b3-02a5ae9796a0", - "name": "John Doe", - "userId": "emailAddress" - }, - "dueBy": { - "value": "2021-02-05T00:00:00-07:00" - }, - "tags": [{ - "type": "date", - "text": "today", - "beginOffset": 39, - "value": { - "value": { - "datetime": "2021-02-05" - } - } - }, { - "type": "person", - "text": "John Doe", - "beginOffset": 8, - "value": { - "value": { - "name": "John Doe", - "id": "e2c5acf8-b9ed-421a-b3b3-02a5ae9796a0", - "assignee": true, - "userId": "emailAddress" - } - } - }], - "dismissed": false, - "payload": { - "content": "Perhaps John Doe can submit the report today.", - "contentType": "text/plain" - }, - "from": { - "id": "e2c5acf8-b9ed-421a-b3b3-02a5ae9796a0", - "name": "John Doe", - "userId": "emailAddress" - } -}] -``` - -### onTopicResponse - -This callback provides you with any of the detected topics in real-time as they are detected. As with the [`onMessageCallback`](#onmessagecallback) this would also return every topic in case of multiple streams. - -#### onTopicResponse JSON Response Example - -```json -[{ - "id": "e69a5556-6729-11eb-ab14-2aee2deabb1b", - "messageReferences": [{ - "id": "0df44422-0248-47e9-8814-e87f63404f2c", - "relation": "text instance" - }], - "phrases": "auto insurance", - "rootWords": [{ - "text": "auto" - }], - "score": 0.9, - "type": "topic" -}] -``` -### subscribeToStream - -This callback allows you to subcribe - -The `subscribeToStream` function allows you to subscribe to existing streaming connection in read-only. It takes the following parameters: - -| Parameters | Type | Example | -| ---------- | ------- | ------- | -| `id` | String | Connection ID created on connection `init`| - -This is a function of our [Subscribe API](/docs/subscribe-api). \ No newline at end of file