Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge master #341

Merged
merged 11 commits into from
Jan 27, 2022
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
22 changes: 22 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@

# Dependencies
/node_modules

# Production
/build

# Generated files
.docusaurus
.cache-loader
config.json

# Misc
.DS_Store
.env
.env.local
.env.development.local
.env.test.local
.env.production.local

npm-debug.log*
yarn-debug.log*
3 changes: 3 additions & 0 deletions babel.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
module.exports = {
presets: [require.resolve('@docusaurus/core/lib/babel/preset')],
};
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
2 changes: 2 additions & 0 deletions python-sdk/async-audio.md → docs/python-sdk/async-audio.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,10 @@ title: Async Audio API
sidebar_label: Audio API
slug: /python-sdk/async-audio
---

import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

---

The Python SDK allow you to asynchronously send audio conversation data and generate the following:
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
52 changes: 52 additions & 0 deletions docs/web-sdk/muting-and-unmuting-connected-device.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
---
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file should be deleted. It contains Web SDK details which we are not pushing right now (as mentioned earlier).

id: muting-and-unmuting-connected-device
title: Muting and Unmuting Connected Device (Beta)
sidebar_label: Muting and Unmuting Connected Device (Beta)
slug: /web-sdk/muting-and-unmuting-connected-device
---

import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

---

:::note IN BETA PHASE
This feature is in the Beta phase. If you have any questions, ideas or suggestions please reach out to us at [email protected].
:::

You can mute and unmute the connected device by simply calling `symbl.mute()` or `symbl.unmute()`.

### Muting Device
A quick snippet on how to use the mute method is given below:

```js
(async () => {
// Creates the WebSocket in a non-processing state
const stream = await symbl.createStream(connectionConfig);
await symbl.mute(stream);
})();
```
:::note Using createStream to start a realtime request
Creating a stream using `symbl.startRealtimeRequest(config)` has been deprecated in favor of `symbl.createStream(config)`. For `createStream`, the WebSocket is started in a non processing state. You must send the start request before processing any audio.

After the stream is created, you need to call `symbl.mute(stream)` to mute the device.
:::

### Unmuting Device
A quick snippet on how to use the unmute method is given below:

```js
(async () => {
// Creates the WebSocket in a non-processing state
const stream = await symbl.createStream(connectionConfig);
await symbl.unmute(stream);
})();
```

:::note Using createStream to start a realtime request
Creating a stream using `symbl.startRealtimeRequest(config)` has been deprecated in favor of `symbl.createStream(config)`. For `createStream`, the WebSocket is started in a non processing state. You must send the start request before processing any audio.

After the stream is created, you need to call `symbl.unmute(stream)` to unmute the device.
:::


159 changes: 159 additions & 0 deletions docs/web-sdk/overview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,159 @@
---
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file should be deleted. It contains Web SDK details which we are not pushing right now (as mentioned earlier).

id: web-sdk
title: Symbl Web SDK (Beta)
sidebar_label: Introduction
slug: /web-sdk/overview
---

import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

---

:::note IN BETA PHASE
This feature is in the Beta phase. If you have any questions, ideas or suggestions please reach out to us at [email protected].
:::

The Symbl Web SDK provides access to the Symbl APIs for applications directly in the browser.

> **Source Code** <br/>
Find the source code here: [https://github.com/symblai/symbl-web-sdk](https://github.com/symblai/symbl-web-sdk). <br/>


## Supported Browsers
---

|-- | Chrome | Edge Firefox | Firefox | Safari |
| -------| ---------- | ------- | ----- | ------- |
| macOS | ![icon](/img/tick-mark.png)| ![icon](/img/tick-mark.png)| ![icon](/img/tick-mark.png) | ![icon](/img/tick-mark.png) |
| Windows | ![icon](/img/tick-mark.png) | ![icon](/img/tick-mark.png)| ![icon](/img/tick-mark.png) | |
| Linux | ![icon](/img/tick-mark.png)| | ![icon](/img/tick-mark.png) |
| iOS | ![icon](/img/tick-mark.png)| | ![icon](/img/tick-mark.png) | ![icon](/img/tick-mark.png) |
| Android | ![icon](/img/tick-mark.png)| | ![icon](/img/tick-mark.png) | ![icon](/img/tick-mark.png) |

## Setup
---
**To use the Symbl Web SDK,**

Include the following script tag in your HTML file:

```html
<script src="https://sdk.symbl.ai/js/beta/symbl-web-sdk/latest/symbl.min.js"></script>
```
or

```html
<script src="https://sdk.symbl.ai/js/beta/symbl-web-sdk/v0.8.2/symbl.min.js"></script>
```

In case of a front-end web application using a framework such as React, import it in the ES2015 style:

```bash
import symbl from "@symblai/symbl-web-sdk";
```

## Initialization
---
The `init` authenticates you to use the Symbl API using the provided authentication credentials. To get authentication credentials (App ID and Secret), follow the steps given in the [Authentication](/docs/developer-tools/authentication#step-1-get-your-api-credentials) page.

You can authenticate:

- [Using your API Credentials](#authenticate-using-api-credentials)

&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; or

- [Using your Auth Token](#authenticate-using-token)

### Authenticate using API Credentials

Use the code given below to authenticate using your App ID and App Secret.

```js
sdk.init({
// APP_ID and APP_SECRET come from the Symbl Platform: https://platform.symbl.ai
appId: APP_ID,
appSecret: APP_SECRET,
basePath: 'https://api.symbl.ai'
})
.then(() => console.log('SDK Initialized.'))
.catch(err => console.error('Error in initialization.', err));
```

### Authenticate using Token

Use the code given below to authenticate using the Auth Token. To generate the Auth Token follow the Steps given in the [Authentication](/docs/developer-tools/authentication#step-2-generate-the-access-token) Page.

```js
sdk.init({
accessToken: ACCESS_TOKEN_HERE,
basePath: 'https://api.symbl.ai'
})
.then(() => console.log('SDK Initialized.'))
.catch(err => console.error('Error in initialization.', err));
```


:::info Web SDK in Labs
The Web SDK is also available as a part of the [Symbl Labs](/docs/labs) with select features. You can find the Web SDK Labs Readme here: [https://github.com/symblai/symbl-web-sdk/blob/labs/README.md](https://github.com/symblai/symbl-web-sdk/blob/labs/README.md) and the source code here: [https://github.com/symblai/symbl-web-sdk/tree/labs](https://github.com/symblai/symbl-web-sdk/tree/labs).
:::

## Streaming API config options

You can utilize the config options provided for our Streaming API. To read about the Streaming API config options, go to [Streaming API Reference](https://docs.symbl.ai/docs/streaming-api/api-reference/#request-parameters).

### Additional Web SDK configs
You can also pass the following configurations that are available specifically with the Web SDK:

| Name | Default | Description |
| -------| ---------- | ------- |
| `sourceNode` | `null` | For passing in an external [MediaStreamAudioSourceNode](https://developer.mozilla.org/en-US/docs/Web/API/MediaStreamAudioSourceNode/MediaStreamAudioSourceNode) object. Although the Web SDK will handle the audio context and source nodes on its own by default, you can pass this option if you wish to handle it externally. |
| `reconnectOnError` | `true` | If this option is set to `true`, the Web SDK will attempt to reconnect to the WebSocket in case of an error. You can also make use of our [onReconnectFail](/docs/web-sdk/web-sdk-reference#onreconnectfailerr) callback which will fire in case the reconnection attempt fails. |

### Usage Example

```js
const id = btoa("my-first-symbl-ai-code");

const connectionConfig = {
id,
insightTypes: ['action_item', 'question'],
sourceNode: sourceNode,
reconnectOnError: true,
handlers: { // Read the handlers section for more
ondevicechange: () => {
alert('device changed!');
},
...
}
...
}


...

// Creates the WebSocket in a non-processing state
const stream = await symbl.createStream(connectionConfig);

// Send the start request
await symbl.unmute(stream);
```

## Tutorials
---
We have prepared a list of tutorials to help you understand how to use the Web SDK.

* [How to Transcribe a Live Audio Input through Microphone](/docs/web-sdk/transcribing-live-audio-through-microphone)
* [How to pass a custom Node Source](/docs/web-sdk/passing-custom-sourcenode)
* [How to pass a custom Device Change Handler](/docs/web-sdk/passing-custom-ondevicechange-handler)

### Web SDK Reference
---
The supported Handlers and Callbacks for the Web SDK are listed below:

* [Event Handlers](/docs/javascript-sdk/reference#event-handlers-1)
* [onSpeechDetected](/docs/javascript-sdk/reference#onspeechdetected)
* [onMessageResponse](/docs/javascript-sdk/reference#onmessageresponse)
* [onInsightResponse](/docs/javascript-sdk/reference#oninsightresponse)
* [onTopicResponse](/docs/javascript-sdk/reference#ontopicresponse)<br/>

👉 See the complete Web SDK Reference [here](/docs/web-sdk/web-sdk-reference).
106 changes: 106 additions & 0 deletions docs/web-sdk/passing-a-custom-sourceNode.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
---
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file should be deleted. It contains Web SDK details which we are not pushing right now (as mentioned earlier).

id: passing-custom-sourcenode
title: Passing a custom sourceNode (Beta)
sidebar_label: Passing a custom sourceNode (Beta)
slug: /web-sdk/passing-custom-sourcenode
---

import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

---

:::note IN BETA PHASE
This feature is in the Beta phase. If you have any questions, ideas or suggestions please reach out to us at [email protected].
:::

You can pass a custom `MediaStreamAudioSourceNode` object to the Web SDK. By default the Web SDK will create the AudioContext and the `MediaStreamAudioSourceNode` object automatically but using this will give you more control over those.

Once you create the `MediaStreamAudioSourceNode` object you can pass it via the `connectionConfig` as sourceNode.

```js
// create the MediaStreamAudioSourceNode
const AudioContext = window.AudioContext || window.webkitAudioContext;
stream = await navigator.mediaDevices.getUserMedia({
audio: true,
video: false
});
context = new AudioContext();
const sourceNode = context.createMediaStreamSource(stream);

symbl.init({
appId: '<your App ID>',
appSecret: '<your App Secret>',
// accessToken: '<your Access Token>', // can be used instead of appId and appSecret
basePath: 'https://api.symbl.ai',
});

const id = btoa("my-first-symbl-ai-code");
// pass in the MediaStreamAudioSourceNode as sourceNode
const connectionConfig = {
id,
sourceNode,
insightTypes: ['action_item', 'question'],
config: {
meetingTitle: 'My Test Meeting ' + id,
confidenceThreshold: 0.7,
timezoneOffset: 480, // Offset in minutes from UTC
languageCode: 'en-US',
sampleRateHertz: 48000
},
speaker: {
// Optional, if not specified, will simply not send an email in the end.
userId: '', // Update with valid email
name: ''
},
handlers: {
/**
* This will return live speech-to-text transcription of the call.
*/
onSpeechDetected: (data) => {
if (data) {
const {punctuated} = data
console.log('Live: ', punctuated && punctuated.transcript)
console.log('');
}
// console.log('onSpeechDetected ', JSON.stringify(data, null, 2));
},
/**
* When processed messages are available, this callback will be called.
*/
onMessageResponse: (data) => {
// console.log('onMessageResponse', JSON.stringify(data, null, 2))
},
/**
* When Symbl detects an insight, this callback will be called.
*/
onInsightResponse: (data) => {
// console.log('onInsightResponse', JSON.stringify(data, null, 2))
},
/**
* When Symbl detects a topic, this callback will be called.
*/
onTopicResponse: (data) => {
// console.log('onTopicResponse', JSON.stringify(data, null, 2))
}
}
};

(async () => {
// Creates the WebSocket in a non-processing state
const stream = await symbl.createStream(connectionConfig);

// Send the start request
await stream.start(stream);
})();
```

### Updating your external source node

If you wish to update your external source node you can do so by using the `symbl.updateSourceNode` function:

```js
symbl.updateSourceNode(stream, sourceNode);
```


Loading