Bump Speech SDK 1.15.1 to support sovereign clouds

CHANGELOG.md

@@ -31,6 +31,8 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
    -  Set `autoScrollSnapOnActivityOffset` and `autoScrollSnapOnPageOffset` to a number (in pixels) to overscroll/underscroll after the pause
 -  Supports multiple transcripts in a single composition, by [@compulim](https://github.com/compulim) in PR [#3653](https://github.com/microsoft/BotFramework-WebChat/pull/3653)
 -  Resolves [#3368](https://github.com/microsoft/BotFramework-WebChat/issues/3368). Added new `sendBoxButtonAlignment` for button alignment in multi-line text mode, by [@compulim](https://github.com/compulim) in PR [#3668](https://github.com/microsoft/BotFramework-WebChat/pull/3668)
+-  Resolves [#3666](https://github.com/microsoft/BotFramework-WebChat/issues/3666). Added support of sovereign clouds when using Direct Line Speech, by [@compulim](https://github.com/compulim) in PR [#3694](https://github.com/microsoft/BotFramework-WebChat/pull/3694)
+   -  Please refer to [`DIRECT_LINE_SPEECH.md`](https://github.com/microsoft/BotFramework-WebChat/blob/master/docs/DIRECT_LINE_SPEECH.md#directlinespeechcredentials) for instructions
 
 ### Fixed
 
@@ -45,10 +47,13 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
 -  Fixes [#3431](https://github.com/microsoft/BotFramework-WebChat/issues/3431). Race condition between the first bot activity and first user activity, which should not cause the first bot activity to be delayed, by [@compulim](https://github.com/compulim) in PR [#3705](https://github.com/microsoft/BotFramework-WebChat/pull/3705)
 -  Fixes [#3676](https://github.com/microsoft/BotFramework-WebChat/issues/3676). Activities without text should not generate bogus `aria-labelledby`, by [@compulim](https://github.com/compulim) in PR [#3697](https://github.com/microsoft/BotFramework-WebChat/pull/3697)
 -  Fixes [#3625](https://github.com/microsoft/BotFramework-WebChat/issues/3625). Update 'no screen reader for custom activity middleware' warning and add screen reader renderer documentation to `ACCESSIBILITY.md`, by [@corinagum](https://github.com/corinagum) in PR [#3689](https://github.com/microsoft/BotFramework-WebChat/pull/3689)
+-  Fixes [#3411](https://github.com/microsoft/BotFramework-WebChat/issues/3411). With Direct Line Speech, clicking on microphone button during speech recognition should no longer stop working, by [@compulim](https://github.com/compulim) in PR [#3694](https://github.com/microsoft/BotFramework-WebChat/pull/3694)
+   -  Although it no locker lock up microphone, clicking on the microphone button has no effect because Direct Line Speech does not support aborting speech recognition
+-  Fixes [#3421](https://github.com/microsoft/BotFramework-WebChat/issues/3421). With Direct Line Speech, after not able to recognize any speech, it should no longer stop working, by [@compulim](https://github.com/compulim) in PR [#3694](https://github.com/microsoft/BotFramework-WebChat/pull/3694)
 
 ### Changed
 
--  Bumped all dependencies to the latest versions, by [@compulim](https://github.com/compulim) in PR [#3594](https://github.com/microsoft/BotFramework-WebChat/pull/3594)
+-  Bumped all dependencies to the latest versions, by [@compulim](https://github.com/compulim) in PR [#3594](https://github.com/microsoft/BotFramework-WebChat/pull/3594) and PR [#3694](https://github.com/microsoft/BotFramework-WebChat/pull/3694)
    -  Development dependencies
       -  [`@babel/[email protected]`](https://npmjs.com/package/@babel/cli)
       -  [`@babel/[email protected]`](https://npmjs.com/package/@babel/core)
@@ -88,9 +93,11 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
       -  [`@babel/[email protected]`](https://npmjs.com/package/@babel/runtime)
       -  [`[email protected]`](https://npmjs.com/package/globalize)
       -  [`[email protected]`](https://npmjs.com/package/markdown-it)
+      -  [`[email protected]`](https://npmjs.com/package/microsoft-cognitiveservices-speech-sdk)
       -  [`[email protected]`](https://npmjs.com/package/react-redux)
       -  [`[email protected]`](https://npmjs.com/package/redux)
       -  [`[email protected]`](https://npmjs.com/package/sanitize-html)
+      -  [`[email protected]`](https://npmjs.com/package/web-speech-cognitive-services)
       -  [`[email protected]`](https://npmjs.com/package/whatwg-fetch)
 -  [#3392](https://github.com/microsoft/BotFramework-WebChat/issues/3392) Bumped Adaptive Cards to the 2.5.0, by [@corinagum](https://github.com/corinagum) in PR [#3630](https://github.com/microsoft/BotFramework-WebChat/pull/3630)
 -  Fixes [#3618](https://github.com/microsoft/BotFramework-WebChat/issues/3618). Fix AC anchors being disabled when AC is obsolete, by [@corinagum](https://github.com/corinagum) in PR [#3687](https://github.com/microsoft/BotFramework-WebChat/pull/3687)

__tests__/html/speechRecognition.simple.html

@@ -29,8 +29,15 @@
       } = window.WebChatTest;
 
       (async function () {
-        const queryParams = window.WebChatTest.parseURLParams(location.hash);
-        let { sa: speechAuthorizationToken, sr: speechRegion, ss: speechSubscriptionKey, t: channelType } = queryParams;
+        const queryParams = new URLSearchParams(location.hash.replace(/^#/u, ''));
+
+        const channelType = queryParams.get('t');
+        const directLineSpeechHostname = queryParams.get('dlsh');
+        const speechAuthorizationToken = queryParams.get('sa');
+        const speechRegion = queryParams.get('sr');
+        const speechSubscriptionKey = queryParams.get('ss');
+        const speechRecognitionHostname = queryParams.get('srh');
+        const speechSynthesisHostname = queryParams.get('ssh');
 
         if (!channelType) {
           console.warn(
@@ -40,8 +47,13 @@
           channelType = 'dl';
         } else if (channelType !== 'dl' && channelType !== 'dlspeech') {
           throw new Error('Invalid channel type specified, must be either "dl" or "dlspeech".');
-        } else if (speechSubscriptionKey && !speechRegion) {
-          throw new Error('Speech region must be specified when speech subscription key is specified.');
+        } else if (
+          speechSubscriptionKey &&
+          !speechRegion &&
+          ((channelType === 'dl' && (!speechRecognitionHostname || !speechSynthesisHostname)) ||
+            (channelType === 'dlspeech' && !directLineSpeechHostname))
+        ) {
+          throw new Error('Speech region or hostname must be specified when speech subscription key is specified.');
         }
 
         if (!speechAuthorizationToken && !speechSubscriptionKey) {
@@ -58,7 +70,10 @@
 
         const speechCredentials = {
           authorizationToken: speechAuthorizationToken,
+          directLineSpeechHostname,
           region: speechRegion,
+          speechRecognitionHostname,
+          speechSynthesisHostname,
           subscriptionKey: speechSubscriptionKey
         };
 
@@ -175,7 +190,6 @@
           throw detailedError;
         }
 
-        await host.snapshot();
         await host.done();
       })().catch(async err => {
         console.error(err);

__tests__/html/speechRecognition.simple.js

@@ -15,10 +15,17 @@ const {
 
 describe.each([
   ['authorization token with Direct Line protocol', { useAuthorizationToken: true }],
+  ['authorization token with Direct Line protocol using hostname', { useAuthorizationToken: true, useHostname: true }],
   ['subscription key with Direct Line protocol', {}],
+  ['subscription key with Direct Line protocol using hostname', { useHostname: true }],
   ['authorization token with Direct Line Speech protocol', { useAuthorizationToken: true, useDirectLineSpeech: true }],
-  ['subscription key with Direct Line Speech protocol', { useDirectLineSpeech: true }]
-])('speech recognition using %s', (_, { useAuthorizationToken, useDirectLineSpeech }) => {
+  [
+    'authorization token with Direct Line Speech protocol using hostname',
+    { useAuthorizationToken: true, useDirectLineSpeech: true, useHostname: true }
+  ],
+  ['subscription key with Direct Line Speech protocol', { useDirectLineSpeech: true }],
+  ['subscription key with Direct Line Speech protocol using hostname', { useDirectLineSpeech: true, useHostname: true }]
+])('speech recognition using %s', (_, { useAuthorizationToken, useDirectLineSpeech, useHostname }) => {
   test('should recognize "Hello, World!".', async () => {
     let queryParams;
 
@@ -29,18 +36,31 @@ describe.each([
             region: DIRECT_LINE_SPEECH_REGION,
             subscriptionKey: DIRECT_LINE_SPEECH_SUBSCRIPTION_KEY
           }),
-          sr: DIRECT_LINE_SPEECH_REGION,
           t: 'dlspeech'
         };
+
+        if (useHostname) {
+          queryParams.dlsh = `${DIRECT_LINE_SPEECH_REGION}.convai.speech.microsoft.com`;
+          queryParams.srh = `${DIRECT_LINE_SPEECH_REGION}.stt.speech.microsoft.com`;
+          queryParams.ssh = `${DIRECT_LINE_SPEECH_REGION}.tts.speech.microsoft.com`;
+        } else {
+          queryParams.sr = DIRECT_LINE_SPEECH_REGION;
+        }
       } else if (!useDirectLineSpeech && COGNITIVE_SERVICES_SUBSCRIPTION_KEY) {
         queryParams = {
           sa: await fetchSpeechServicesAuthorizationToken({
             region: COGNITIVE_SERVICES_REGION,
             subscriptionKey: COGNITIVE_SERVICES_SUBSCRIPTION_KEY
           }),
-          sr: COGNITIVE_SERVICES_REGION,
           t: 'dl'
         };
+
+        if (useHostname) {
+          queryParams.srh = `${COGNITIVE_SERVICES_REGION}.stt.speech.microsoft.com`;
+          queryParams.ssh = `${COGNITIVE_SERVICES_REGION}.tts.speech.microsoft.com`;
+        } else {
+          queryParams.sr = COGNITIVE_SERVICES_REGION;
+        }
       } else {
         if (useDirectLineSpeech) {
           console.warn(
@@ -65,16 +85,34 @@ describe.each([
 
         const { region, token: authorizationToken } = await res.json();
 
-        queryParams = { sa: authorizationToken, sr: region, t: useDirectLineSpeech ? 'dlspeech' : 'dl' };
+        queryParams = { sa: authorizationToken, t: useDirectLineSpeech ? 'dlspeech' : 'dl' };
+
+        if (useHostname) {
+          if (useDirectLineSpeech) {
+            queryParams.dlsh = `${region}.convai.speech.microsoft.com`;
+          }
+
+          queryParams.srh = `${region}.stt.speech.microsoft.com`;
+          queryParams.ssh = `${region}.tts.speech.microsoft.com`;
+        } else {
+          queryParams.sr = region;
+        }
       }
     } else {
       if (useDirectLineSpeech) {
         if (DIRECT_LINE_SPEECH_SUBSCRIPTION_KEY) {
           queryParams = {
-            sr: DIRECT_LINE_SPEECH_REGION,
             ss: DIRECT_LINE_SPEECH_SUBSCRIPTION_KEY,
             t: 'dlspeech'
           };
+
+          if (useHostname) {
+            queryParams.dlsh = `${DIRECT_LINE_SPEECH_REGION}.convai.speech.microsoft.com`;
+            queryParams.srh = `${DIRECT_LINE_SPEECH_REGION}.stt.speech.microsoft.com`;
+            queryParams.ssh = `${DIRECT_LINE_SPEECH_REGION}.tts.speech.microsoft.com`;
+          } else {
+            queryParams.sr = DIRECT_LINE_SPEECH_REGION;
+          }
         } else {
           return console.warn(
             'No environment variable "DIRECT_LINE_SPEECH_SUBSCRIPTION_KEY" is set, skipping this test.'
@@ -83,10 +121,16 @@ describe.each([
       } else {
         if (COGNITIVE_SERVICES_SUBSCRIPTION_KEY) {
           queryParams = {
-            sr: COGNITIVE_SERVICES_REGION,
             ss: COGNITIVE_SERVICES_SUBSCRIPTION_KEY,
             t: 'dl'
           };
+
+          if (useHostname) {
+            queryParams.srh = `${COGNITIVE_SERVICES_REGION}.stt.speech.microsoft.com`;
+            queryParams.ssh = `${COGNITIVE_SERVICES_REGION}.tts.speech.microsoft.com`;
+          } else {
+            queryParams.sr = COGNITIVE_SERVICES_REGION;
+          }
         } else {
           return console.warn(
             'No environment variable "COGNITIVE_SERVICES_SUBSCRIPTION_KEY" is set, skipping this test.'

__tests__/setup/pageObjects/isDictating.js

@@ -6,5 +6,5 @@ export default async function isDictating(driver) {
   );
   const microphoneButtonClassName = await microphoneButtonContainer.getAttribute('class');
 
-  return microphoneButtonClassName.split(' ').includes('dictating');
+  return microphoneButtonClassName.split(' ').includes('webchat__microphone-button--dictating');
 }

docs/DIRECT_LINE_SPEECH.md

@@ -368,11 +368,13 @@ These are the options to pass when calling `createDirectLineSpeechAdapters`.
         <code>fetchCredentials</code>
       </td>
       <td>
-        <pre>async () => ({<br />&nbsp;&nbsp;authorizationToken: string,<br />&nbsp;&nbsp;region: string<br />}) ||<br /><br />async () => ({<br />&nbsp;&nbsp;region: string,<br />&nbsp;&nbsp;subscriptionKey: string<br />})</pre>
+        <code>
+          <a href="#directlinespeechcredentials">DirectLineSpeechCredentials</a>
+        </code>
       </td>
       <td>(Required)</td>
       <td>
-        An asynchornous function to fetch credentials, including region and either authorization token or subscription key.
+        An asynchronous function to fetch credentials, including either hostname or region, and either authorization token or subscription key.
       </td>
     </tr>
     <tr>
@@ -400,6 +402,28 @@ These are the options to pass when calling `createDirectLineSpeechAdapters`.
   </tbody>
 </table>
 
+### `DirectLineSpeechCredentials`
+
+```js
+type DirectLineSpeechCredentials = {
+  authorizationToken: string,
+  region: string
+} || {
+  authorizationToken: string,
+  directLineSpeechHostname: string
+} || {
+  region: string,
+  subscriptionKey: string
+} || {
+  directLineSpeechHostname: string,
+  subscriptionKey: string
+}
+```
+
+For public clouds, we recommend using the `region` option, such as `"westus2"`.
+
+For sovereign clouds, you should specify the hostname in FQDN through `directLineSpeechHostname` option, such as `"virginia.convai.speech.azure.us"`.
+
 ## Known issues
 
 ### Differences in `conversationUpdate` behaviors