Motivation

Web Speech API does not support SSML input to the speech synthesis engine WICG/speech-api#10, or the ability to capture the output of speechSynthesis.speak() as aMedaiStreamTrack or raw audio https://lists.w3.org/Archives/Public/public-speech-api/2017Jun/0000.html.

See Issue 1115640: [FUGU] NativeTransferableStream.

Synopsis

"externally_connectable" => Native Messaging => eSpeak NG => MediaStreamTrack.

Use local espeak-ng with -m option set in the browser.

Output speech sythesis audio as a live MediaStreamTrack.

Use Native Messaging, Bash with GNU Core Utiltities to input text and Speech Synthesis Markup Language as STDIN to espeak-ng, stream STDOUT in "real-time" as live MediaStreamTrack.

Install

Dependencies

eSpeak NG Building eSpeak NG.

git clone https://github.com/guest271314/native-messaging-espeak-ng.git
cd native-messaging-espeak-ng
chmod u+x nm_espeak_ng.sh

Navigate to chrome://extensions, set Developer mode to on, click Load unpacked, select downloaded git directory.

Note the generated extension ID, substitute that value for <id> in nm_epseakng.json and AudioStream.js.

Substitute full local path to nm_espeakng.sh for /path/to in nm_espeakng.json.

"allowed_origins": [ "chrome-extension://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/" ]

Copy nm_espeakng.json to NativeMessagingHosts directory in Chromium or Chrome configuration folder, on Linux, i.e., ~/.config/chromium; ~/.config/google-chrome-unstable.

cp nm_espeakng.json ~/.config/chromium/NativeMessagingHosts

Reload extension.

Usage

On origins listed in "matches" array in "web_accessible_resources" and "externally_connectable" object in manifest.json, e.g., at console

let text = `So we need people to have weird new
ideas ... we need more ideas to break it
and make it better ...

Use it. Break it. File bugs. Request features.

- Soledad Penadés, Real time front-end alchemy, or: capturing, playing,
  altering and encoding video and audio streams, without
  servers or plugins!
<br>  
von Braun believed in testing. I cannot
emphasize that term enough – test, test,
test. Test to the point it breaks.

- Ed Buckbee, NASA Public Affairs Officer, Chasing the Moon
<br>
Now watch. ..., this how science works.
One researcher comes up with a result.
And that is not the truth. No, no.
A scientific emergent truth is not the
result of one experiment. What has to
happen is somebody else has to verify
it. Preferably a competitor. Preferably
someone who doesn't want you to be correct.

- Neil deGrasse Tyson, May 3, 2017 at 92nd Street Y
<br>
It’s like they say, if the system fails you, you create your own system.

- Michael K. Williams, Black Market
<br>
1. If a (logical or axiomatic formal) system is consistent, it cannot be complete.
2. The consistency of axioms cannot be proved within their own system.

- Kurt Gödel, Incompleteness Theorem, On Formally Undecidable Propositions of Principia Mathematica and Related Systems`;
let {AudioStream} = await import(`chrome-extension://<id>/AudioStream.js?${new Date().getTime()}`)
let audioStream = new AudioStream({stdin: `espeak-ng -m --stdout "${text}"`});
await audioStream.start();

To record MediaStreamTrack pass recorder: true (set to false by default) to second parameter

let audioStream = var audioStream = new AudioStream({
   stdin: `espeak-ng -m --stdout '<voice name="Storm">Hello world.<br></voice>'`, 
   recorder: true
});
let ab = await audioStream.start(); // ArrayBuffer
let blobURL = URL.createObjectURL(new Blob([ab], {type: 'audio/wav'}));

Abort the request and audio output.

await audioStream.abort();

Dynamically set "externally_connectable" and "web_accessible_resources"

manifest.json is set by default to one match pattern, "https://*.github.com/*". To write the current active Tab URL match pattern in "matches" array of "externally_connectable" and "web_accessible_resources", click the extension icon, select native-messaging-espeak-ng extension directory, and grant permission to edit files in the folder using File System Access API.

References

• Include test for setting an SSML document at SpeechSynthesisUtterance .text property within speech-api
• This is again recording from microphone, not from audiooutput device
• Support SpeechSynthesis to a MediaStreamTrack
• Clarify getUserMedia({audio:{deviceId:{exact:<audiooutput_device>}}}) in this specification mandates capability to capture of audio output device - not exclusively microphone input device
• How to modify existing code or build with -m option set for default SSML parsing?
• Issue 795371: Implement SSML parsing at SpeechSynthesisUtterance
• Implement SSML parsing at SpeechSynthesisUtterance
• How is a complete SSML document expected to be parsed when set once at .text property of SpeechSynthesisUtterance instance?
• How to programmatically send a unix socket command to a system server autospawned by browser or convert JavaScript to C++ souce code for Chromium?
• <script type="shell"> to execute arbitrary shell commands, and import stdout or result written to local file as a JavaScript module
• Add execute() to FileSystemDirectoryHandle
• Issue 795371: Implement SSML parsing at SpeechSynthesisUtterance
• Implement SSML parsing at SpeechSynthesisUtterance
• How is a complete SSML document expected to be parsed when set once at .text property of SpeechSynthesisUtterance instance?
• How to programmatically send a unix socket command to a system server autospawned by browser or convert JavaScript to C++ souce code for Chromium?
• <script type="shell"> to execute arbitrary shell commands, and import stdout or result written to local file as a JavaScript module
• Add execute() to FileSystemDirectoryHandle
• SpeechSynthesis to a MediaStreamTrack or: How to execute arbitrary shell commands using inotify-tools and DevTools Snippets