Custom voice (or speech?) is getting picked up in text box but not displayed in the chat history

[prashanthsridhar]

Screenshots

Describe the bug

whatever the user says is being captured in the input text box but it's not being propagated to the chat window.

[corinagum]

Please use our bug report template to provide the information we will need in order to help you.

What, if any, are your console errors?

[prashanthsridhar]

Hi Corina,
steps to reproduce:
The snippet is as folows:

selectVoice: () => ({ voiceURI: 'sample' }),
            webSpeechPonyfillFactory: await this.WebChat.createCognitiveServicesSpeechServicesPonyfillFactory({
                region: 'westus2',
                speechSynthesisDeploymentId: 'sample',
                subscriptionKey: 'sample'
 })

Console output

[corinagum]

@compulim assigning to you (since I don't have a custom voice to test with xD)

[prashanthsridhar]

Additional Information

botConnection:-

const botConnection = new this.WebChat.createDirectLine({
            token:"123123",
            webSocket: true,
            styleOptions:styleOptions
        });

verbatim code without keys:

this.WebChat.renderWebChat({            
            styleOptions: {
                bubbleFromUserBackground: '#F16236',
                bubbleFromUserTextColor: 'White',
            },
            directLine: botConnection,
            language: 'en-GB',
            chatTitle: false,
            userID: "user",
            name: "user",
            sender: "user",    
            receiver: "user",
            selectVoice: () => ({ voiceURI: 'name' }),
            webSpeechPonyfillFactory: await this.WebChat.createCognitiveServicesSpeechServicesPonyfillFactory({
                region: 'westus2',
                speechRecognitionEndpointId: '12123123123123213',
                speechSynthesisDeploymentId: '12312312312312312',
                subscriptionKey: '123123123123'
            })            
        },
            this.botWindowElement.nativeElement
        );

Description:

We are using custom speech and whatever we say is getting picked up and displayed in the text area, but it is not going to the chat window. If we type the text and hit enter it moves to the chat window .

[mountju]

have the same issue

[prashanthsridhar]

Hi @compulim, this is a major blocker for us. It will be really helpful if this is picked up ASAP.

[corinagum]

@prashanthsridhar, sorry for the delay. I've managed to test both custom voice and custom speech and do not encounter any errors. Could you provide more detail about your Cognitive Services set up so that we can try to reproduce the problem? First it would be great if you could clarify which is causing you issues, since your bug reports custom voice originally, but you are using both in your example code.

Do you have a deployed source that I could take a look at?

[corinagum]

@prashanthsridhar it would be great if you could check out the latest bits. Please check out our Contributing.md guidelines, which will tell you have to build the project. After you run npm build, you can take out the webchat file from packages/bundle/dist/ and link that in your app to test latest bits.

Since you are both using Angular, we'll add that to the list of possible reasons that this bug is occurring. Please let me know the results of testing on latest bits, and we will go from there. :)

[prashanthsridhar]

Hi @corinagum ,i compiled the js file from the repo and used it in my app. The issue still exists.

[TechieWithBeard]

I'm also facing the same issue, compiled the js file from the repo and used it in my app.

[corinagum]

Thanks for reporting. @vishnuramero, are you also using an Angular App?

[TechieWithBeard]

@corinagum yes, the same issue, pure HTML file works but when I'm trying to integrate with angular speech to text is not working, it's recognizing the voice but not sending anything. is there any fix??

[manish-95]

Hi, I am attempting the same in my Angular App using the sample provided in 03.b.host-with-angular. To this sample, I have tried to use my Azure Cognitive Service Speech subscription key to add Speech and voice capabilities following the instructions provided here.

Although the speech to text service is activated and I am able to see the recognized text in my message window, the same is not sent to the Bot. Attaching a screenshot with the error that I see in the console window, with this setup.

[manish-95]

Update: I have tried the same setup, but with Browser Speech instead of Cognitive Services, and I am able to see that the message picked up by STT is sent to the Bot successfully.

[TechieWithBeard]

did a patchwork, works fine for me changed "webchat.js" file a bit.

created a new array and pushed the data from speech to the array.

[prashanthsridhar]

@corinagum will this be added in a release or patch?

[corinagum]

@prashanthsridhar at this time there's no plan for a patch. Unfortunately we still have to investigate the fix :/

[corinagum]

Investigating this now! Hope to have updates soon.

[corinagum]

Update: I have a repro of this problem and am hunting around looking for the source code. :) thanks for your patience!

[corinagum]

I'm reassigning this to @compulim. I believe the fix is going to be in his package: https://github.com/compulim/web-speech-cognitive-services

[compulim]

This is still scheduled for R8. Please stay tuned.

[compulim]

Working on this one now. Repro on 4.7.1 and latest [email protected].

[compulim]

This bug is same as the one mentioned in our FAQs (#1961), i.e. speech will fail on Web Chat 4.6 when running under Angular, due to an unresolved bug in Angular that opened since 2016.

I have create a new bug with workaround steps, along with background and "how to fix".

Please follow the workaround steps in #2957.