Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Direct Line Speech support #2621

Merged
merged 47 commits into from
Dec 4, 2019
Merged

Conversation

compulim
Copy link
Contributor

@compulim compulim commented Nov 21, 2019

Fixes #2571. Fixes #2572. Fixes #2574.

Changelog Entry

Added

Description

  • Add Direct Line Speech SDK package
  • Bundle Direct Line Speech SDK with Web Chat CDN bundle and NPM package
  • Add new sample 06.i.directline-speech-sample

Specific Changes

  • Added a new package botframework-directlinespeech-sdk under the folder /packages/directlinespeech-sdk
    • This is the Direct Line Speech SDK adapter set, including both DirectLineJS-alt, and Web Speech API ponyfill
  • Support unabortable speech recognition engine
    • If the speech recognition engine does not allow abort, we will disable microphone button while recognition is ongoing
  • We will start sending detailed speech recognition result to the bot, including confidence and actual transcript
  • We will peacefully ignore OAuth if the DLJS adapter does not support it
  • New 06.i.direct-line-speech sample

  • Add documentation to SPEECH.md
  • Testing Added
    • We need manual testing

Copy link
Member

@stevengum stevengum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@compulim, do we need the commits before adbc6e to get DLS integration, or are those separate changes?

I'd like to take a look through this PR, but I'm not familiar with the codebase and what the other commits are needed for 😔

(I'd like to lessen my cognitive load in this PR if possible 😄)

@compulim compulim changed the title Add Direct Line Speech support [DRAFT] Add Direct Line Speech support Nov 21, 2019
@compulim compulim marked this pull request as ready for review December 3, 2019 10:34
@compulim compulim changed the title [DRAFT] Add Direct Line Speech support Add Direct Line Speech support Dec 3, 2019
@cwhitten
Copy link
Member

cwhitten commented Dec 3, 2019

The CI error is coming from https://registry.npmjs.org/botframework-directlinespeech-sdk not being found

@corinagum corinagum added the R7 label Dec 3, 2019
Copy link
Contributor

@corinagum corinagum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

finished the 06.i readme

serve.json Outdated Show resolved Hide resolved
samples/06.i.direct-line-speech/README.md Outdated Show resolved Hide resolved
samples/06.i.direct-line-speech/README.md Show resolved Hide resolved
samples/06.i.direct-line-speech/README.md Outdated Show resolved Hide resolved
samples/06.i.direct-line-speech/README.md Show resolved Hide resolved
samples/06.i.direct-line-speech/README.md Outdated Show resolved Hide resolved
samples/06.i.direct-line-speech/README.md Outdated Show resolved Hide resolved
samples/06.i.direct-line-speech/README.md Outdated Show resolved Hide resolved
samples/06.i.direct-line-speech/README.md Outdated Show resolved Hide resolved
samples/06.i.direct-line-speech/README.md Outdated Show resolved Hide resolved
Comment on lines +4 to +5
const EMPTY_MP3_BASE64 =
'SUQzBAAAAAAAI1RTU0UAAAAPAAADTGF2ZjU3LjU2LjEwMQAAAAAAAAAAAAAA//tAwAAAAAAAAAAAAAAAAAAAAAAASW5mbwAAAA8AAAACAAABhgC7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7//////////////////////////////////////////////////////////////////8AAAAATGF2YzU3LjY0AAAAAAAAAAAAAAAAJAUHAAAAAAAAAYYoRBqpAAAAAAD/+xDEAAPAAAGkAAAAIAAANIAAAARMQU1FMy45OS41VVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVf/7EMQpg8AAAaQAAAAgAAA0gAAABFVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😯

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've never seen this in an application before but it makes sense!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😅 I found a better way to prepare white noise programmatically. But need time to prove whether it works. You know, Safari is quite picky on Web Audio and this code already works in our adapter.

Not to mention the new code could be just about same length as this one.

BTW, this one is generated by ffmpeg -f lavfi -i anullsrc=r=44100:cl=mono -t 0.01 -q:a 9 -acodec libmp3lame out.mp3.

@coveralls
Copy link

Coverage Status

Coverage decreased (-2.8%) to 60.848% when pulling fb4ad40 on compulim:feat-dlspeech into 0fcd7fe on microsoft:master.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants