SDK with Cloudflare workers and Fetch API #645

rodbs · 2023-03-09T17:01:03Z

When using this sdk with cloudflare workers I get this error:
[XMLHttpRequest is not defined](https://community.cloudflare.com/t/xmlhttprequest-is-not-defined/133866)

I understand it's due to the fact the sdk doesn't suport the Fetch API.
Is it planned to support this API to use speech services sdk with Cloudflare workers?
Is there any way to make it work now with workers?

The text was updated successfully, but these errors were encountered:

glharper · 2023-03-10T00:11:42Z

@rodbs Thanks for opening this issue, and using Speech SDK. I've merged a fix for this into our repo removing the reference to XMLHttpRequest, is there a way you could pull and build the latest commit? (If not, shoot me a email at <my_username>(at)microsoft(dot)com and I'll send you binaries.)

If you could check whether this unblocks you, it'd be much appreciated (and you'd get a shoutout in our next release notes :-) ).

rodbs · 2023-03-10T17:41:58Z

@glharper It's not working. I've buil it, and copied the new distrib folder to the node_modules/microsoft-cognitiveservices-speech-sdk/distrib. I still have the error:
(I'm using Remix on Cloudflare Pages)

[mf:err] TypeError: globalThis.XMLHttpRequest is not a constructor
    at checkTypeSupport (C:\Users\sources\igls-remix-cf\node_modules\rollup-plugin-node-polyfills\polyfills\http-lib\capability.js:20:11)
    at node_modules/rollup-plugin-node-polyfills/polyfills/http-lib/capability.js (C:\Users\sources\igls-remix-cf\node_modules\rollup-plugin-node-polyfills\polyfills\http-lib\capability.js:39:45)
    at C:\Users\rodri\AppData\Local\Temp\tmp-53100-fk9AiL2BqHmS\bundledWorker-0.5935916311359655.js:48:56
    at node_modules/rollup-plugin-node-polyfills/polyfills/http-lib/request.js (C:\Users\sources\igls-remix-cf\node_modules\rollup-plugin-node-polyfills\polyfills\http-lib\request.js:1:1)
    at C:\Users\rodri\AppData\Local\Temp\tmp-53100-fk9AiL2BqHmS\bundledWorker-0.5935916311359655.js:48:56
    at node-modules-polyfills:http (C:\Users\sources\igls-remix-cf\public\node-modules-polyfills:http:30:1)
    at C:\Users\rodri\AppData\Local\Temp\tmp-53100-fk9AiL2BqHmS\bundledWorker-0.5935916311359655.js:48:56
    at node-modules-polyfills-commonjs:http (C:\Users\sources\igls-remix-cf\public\node-modules-polyfills-commonjs:http:2:18)
    at C:\Users\rodri\AppData\Local\Temp\tmp-53100-fk9AiL2BqHmS\bundledWorker-0.5935916311359655.js:51:50
    at node_modules/microsoft-cognitiveservices-speech-sdk/distrib/es2015/external/ocsp/ocsp/agent.js (C:\Users\sources\igls-remix-cf\node_modules\microsoft-cognitiveservices-speech-sdk\distrib\es2015\external\ocsp\ocsp\agent.js:6:12)
    ```

glharper · 2023-03-13T15:41:48Z

@rodbs I've created a branch removing the offending code, could you build and test again?
Test branch

rodbs · 2023-03-13T22:07:02Z

@glharper No, I'm getting the same error (I've tried with remove-ocsp branch)

 C:\Users\sources\igls-remix-cf\node_modules\rsvp\dist\lib\rsvp\-internal.js:75
      var error = TRY_CATCH_ERROR.error;
          ^
TypeError: globalThis.XMLHttpRequest is not a constructor
    at originalResolve (C:\Users\sources\igls-remix-cf\node_modules\rsvp\dist\lib\rsvp\-internal.js:75:11)
    at node_modules/rollup-plugin-node-polyfills/polyfills/http-lib/capability.js (C:\Users\sources\igls-remix-cf\node_modules\rsvp\dist\lib\rsvp\-internal.js:102:133)
    at C:\Users\rodri\AppData\Local\Temp\tmp-20132-nDcsnj4wPzj5\bundledWorker-0.43920512800175326.js:48:56
    at node_modules/rollup-plugin-node-polyfills/polyfills/http-lib/request.js (C:\Users\sources\igls-remix-cf\public\node-modules-polyfills:assert:179:3)
    at C:\Users\rodri\AppData\Local\Temp\tmp-20132-nDcsnj4wPzj5\bundledWorker-0.43920512800175326.js:48:56
    at node-modules-polyfills:http (C:\Users\sources\igls-remix-cf\node_modules\minimatch\minimatch.js:54:41)
    at C:\Users\rodri\AppData\Local\Temp\tmp-20132-nDcsnj4wPzj5\bundledWorker-0.43920512800175326.js:48:56
    at node-modules-polyfills-commonjs:http (C:\Users\sources\igls-remix-cf\node_modules\minimatch\minimatch.js:304:14)    at C:\Users\rodri\AppData\Local\Temp\tmp-20132-nDcsnj4wPzj5\bundledWorker-0.43920512800175326.js:51:50

glharper · 2023-03-14T15:10:52Z

@rodbs I've updated the branch to remove the dependency on rsvp, please give it another shot, and thanks for testing this for us. :-)

rodbs · 2023-03-14T23:32:41Z

@glharper I don't get the error anymore, but I cannot make it work. I don't know why but when I run this code I don't get the promise resolved. There's no error message at all.
I'm running this code in Remix+Cloudflare. I think it worked when I ran it in Remix+Node (and for sure in NextJS).
So not sure if it's something related with Cloudflare workers ....

  const synthesizer = new sdk.SpeechSynthesizer(speechConfig);

  const prom = new Promise((resolve, reject) => {
    const ssml = `<speak version="1.0" xmlns="http://www.w3.org/2001/10/synthesis" xmlns:mstts="https://www.w3.org/2001/mstts" xml:lang="en-US">
          <voice name="${voiParam.name}">
          <prosody rate="${voiceSpeedNumber}">            
              ${text}            
            </prosody>
          </voice>
          </speak>
       `;
 
    synthesizer.speakSsmlAsync(
      ssml,
      async (result) => {
        if (result.errorDetails) {       
          console.error(result.errorDetails);
          synthesizer.close();
        } else {
    
          const rdm = parseInt(Math.random() * 10000);       
          const fileName = `${tablename}-${rowindex}-${key}-${cloud}-${voiParam.gender}-${voiceSpeedName}-${rdm}.mp3`;

           const file = new File([result.audioData], fileName, {
           type: 'audio/mp3',
           lastModified: Date.now()
            });
 
           const url = await saveVoiceToStg(fileName, file, folder);
         
          synthesizer.close();
          resolve([url, fileName]);
        }
      },
      (error) => {
        console.log('err', error);
        synthesizer.close();
      }
    );
  });
 
  return prom;

 Promise {
  <pending>,
  [Symbol(async_id_symbol)]: 11403,
  [Symbol(trigger_async_id_symbol)]: 10994,
  [Symbol(kResourceStore)]: undefined,
  [Symbol(kResourceStore)]: RequestContext {
    requestDepth: 1,
    pipelineDepth: 1,
    durableObject: false,
    externalSubrequestLimit: 1000,
    internalSubrequestLimit: 1000
  }
}

glharper · 2023-03-15T00:44:37Z

@rodbs Try using const synthesizer = new sdk.SpeechSynthesizer(speechConfig, null);
Not sure what the DefaultSpeakerOutput on a Cloudflare worker would be.

rodbs · 2023-03-15T16:49:40Z

@glharper
I think the problems lies in the way CF works deals with async tasks and this method waitUntil

https://developers.cloudflare.com/workers/runtime-apis/fetch-event/#waituntil

Here it's an explanation:
https://community.cloudflare.com/t/should-i-use-event-waituntil-or-await-fetch-or-just-fetch/56183/2

event.waitUntil() is only useful if you have some asynchronous task that you want to keep running after returning a response to the visitor. Usually the Workers runtime will cancel any background tasks spawned from a particular FetchEvent once it detects that the response has been sent. event.waitUntil() tells the runtime to wait for another promise in addition to the visitor response before canceling any tasks. This probably doesn’t apply to your immediate needs. Further, event.waitUntil() doesn’t return anything, so treating its return value as anything other than undefined will lead to an exception.

I think I need to enclose with a waitUntil the promise running the synthesizer.speakSsmlAsync. The only issue I see it's that waitUntil doesn't return anything, so I cannot return the url of the bucket where uploading audio file....

glharper · 2023-03-15T17:27:37Z

@rodbs Couldn't you declare the url variable outside the waitUntil, like:

   let url = null;
   await event.waitUntil(
    synthesizer.speakSsmlAsync(
      ssml,
      async (result) => {
         [...]
         url = await saveVoiceToStg(fileName, file, folder);
         [...]
    });
   );

rodbs · 2023-03-15T23:48:17Z

@glharper
The synthesizer.speakSsmlAsync code inside the waitUntil is not being executed. I don't why because I don't get any message error, not even doing a try/catch. It's like it's being skipped ... Might it be because it's not doing inside a Fetch to Azure API?

glharper · 2023-03-16T02:01:11Z

@rodbs
If you have access to console output, could you add the following code to the beginning:
sdk.Diagnostics.SetLoggingLevel(sdk.LogLevel.Debug);
and see what console output you get?

Also, you're using the sdk.SpeechSynthesizer(speechConfig, null) syntax, correct?

The SDK connects to a websocket service, so there should be no Fetch happening.

rodbs · 2023-03-16T11:34:18Z

@glharper
Yes, I set this config sdk.SpeechSynthesizer(speechConfig, null)

I've run two tests (I deleted the subscription key in the output) : first one with waitUntil,

2023-03-16T11:21:23.364Z | ConnectionStartEvent | privName: ConnectionStartEvent | privEventId: AA35EECE3F914F6F884080943F4F630A | privEventTime: 2023-03-16T11:21:23.364Z | privEventType: 1 | privMetadata: {} | privConnectionId: 90C0817D3F8841E4B4BED8900737FFC5 | privUri: wss://westeurope.tts.speech.microsoft.com/cognitiveservices/websocket/v1?Ocp-Apim-Subscription-Key=...................&X-ConnectionId=90C0817D3F8841E4B4BED8900737FFC5 | privHeaders: <NULL>

and this one with a Promise/resolve
2023-03-16T11:28:53.087Z | SynthesisTriggeredEvent | privName: SynthesisTriggeredEvent | privEventId: BE9B72E1A2004F7BBE07C66B2D9EB546 | privEventTime: 2023-03-16T11:28:53.087Z | privEventType: 1 | privMetadata: {} | privRequestId: 915C7FD74B674E45A291EE8586B7281E | privSessionAudioDestinationId: <NULL> | privTurnAudioDestinationId: <NULL> 2023-03-16T11:28:53.087Z | ConnectingToSynthesisServiceEvent | privName: ConnectingToSynthesisServiceEvent | privEventId: E966B260E51A4067A05E22416C5CE413 | privEventTime: 2023-03-16T11:28:53.087Z | privEventType: 1 | privMetadata: {} | privRequestId: 915C7FD74B674E45A291EE8586B7281E | privAuthFetchEventId: 8B8C3EEC77DB42CCB1FDBDEF985826FB 2023-03-16T11:28:53.087Z | ConnectionStartEvent | privName: ConnectionStartEvent | privEventId: D0FE081004E649E8ADD28F34C531CED6 | privEventTime: 2023-03-16T11:28:53.087Z | privEventType: 1 | privMetadata: {} | privConnectionId: 94181D344C014D73A91060FB44D300A7 | privUri: wss://westeurope.tts.speech.microsoft.com/cognitiveservices/websocket/v1?Ocp-Apim-Subscription-Key=.....................&X-ConnectionId=94181D344C014D73A91060FB44D300A7 | privHeaders: <NULL>

glharper · 2023-03-16T13:12:56Z

@rodbs The sdk is trying to connect to the websocket endpoint, but cannot. If the ws library weren't available, I'd expect similar "can't resolve"/"not a constructor" error messages as the ones you originally posted.

My best guess is a network configuration issue between your Cloudflare instance and the wss://westeurope.tts.speech.microsoft.com endpoint. One thing to check is if westeurope.tts.speech.microsoft.com resolves a DNS address when you ping it from Cloudflare. Another is that the 443 port (for wss traffic) is not blocked on your side.

Beyond that, you'll need to engage Cloudflare support and figure out why that connection is not happening.

rodbs · 2023-03-16T15:55:19Z

@glharper
Ok, understood. I going to check in Cloudflare discord channel because I've just read the wss might no work in local mode (only ws) . wss seems to work only when deployed to prod (but if that's the case I can not tested until you deploy this version!)

Btw, is there a way to ping over wss/ws? Is this code valid ?
https://stackoverflow.com/questions/67327292/how-to-ping-a-websocket-server-every-1-second-in-js

glharper · 2023-03-17T13:38:36Z

@rodbs To ping over wss, you have to establish a connection first, and it doesn't look like you're able to establish a connection with the wss://westeurope.tts.speech.microsoft.com endpoint yet.

rodbs · 2023-03-17T16:46:05Z

@glharper Yeap, I think you're right. I've posted the issue in Cloudflare but no reply yet.

In the meantime, are you planning to publish these changes to avoid the XMLHttpRequest in the next release?

glharper · 2023-03-17T17:15:18Z

@rodbs Our tentative timeline for the OCSP removal is 1.28.0 (expected beginning of May). If you need this in a versioned branch sooner, I can talk to our team about a 1.28-prerelease when we release 1.27 in a few weeks.

rodbs · 2023-03-17T19:11:36Z

@glharper Yeap, if it could be in a prerelease it'd be highly appreciated so that I can do more testing. Thanks!

rodbs · 2023-04-06T11:08:18Z

@glharper
I've seen v1.27 has been released. Are you planning to release 1.28-prerelease as you commented a few ago?. Thx!

glharper · 2023-04-09T20:17:08Z

@rodbs
I've been OOF this past week, I'll bend the ear of our release manager and see if we can get that out this week.

glharper · 2023-04-11T14:02:32Z

@rodbs This change has been merged in master, if you're able to take a commit specific dependency.

rodbs · 2023-04-12T16:17:34Z

@glharper
I'm installing it by adding this to package.json

"microsoft-cognitiveservices-speech-sdk": "https://github.com/microsoft/cognitive-services-speech-sdk-js#a79b92f5bd774e03113a196402cedbee82f8cf90",

But I'm getting this error when install (using yarn):

...
$ npm run build

> [email protected] build
> gulp compress && gulp build2015

[18:13:34] Using gulpfile ~\AppData\Local\Yarn\Cache\v6\.tmp\21c2f99d30ecb4b33a45d8db9ea14fae.a79b92f5bd774e03113a196402cedbee82f8cf90.prepare\gulpfile.js
[18:13:34] Starting 'compress'...
[18:13:34] Starting 'bundle'...
[18:13:34] Starting 'build'...
[18:13:34] Starting 'build'...
C:/Users/rodri/AppData/Local/Yarn/Cache/v6/.tmp/21c2f99d30ecb4b33a45d8db9ea14fae.a79b92f5bd774e03113a196402cedbee82f8cf90.prepare/node_modules/@types/babel__traverse/index.d.ts(68,34): error TS7039: Mapped object type implicitly has an 'any' template type.
C:/Users/rodri/AppData/Local/Yarn/Cache/v6/.tmp/21c2f99d30ecb4b33a45d8db9ea14fae.a79b92f5bd774e03113a196402cedbee82f8cf90.prepare/node_modules/@types/babel__traverse/index.d.ts(68,50): error TS1005: ']' expected.
C:/Users/rodri/AppData/Local/Yarn/Cache/v6/.tmp/21c2f99d30ecb4b33a45d8db9ea14fae.a79b92f5bd774e03113a196402cedbee82f8cf90.prepare/node_modules/@types/babel__traverse/index.d.ts(68,50): error TS2304: Cannot find name 'as'.
C:/Users/rodri/AppData/Local/Yarn/Cache/v6/.tmp/21c2f99d30ecb4b33a45d8db9ea14fae.a79b92f5bd774e03113a196402cedbee82f8cf90.prepare/node_modules/@types/babel__traverse/index.d.ts(68,53): error TS1005: ';' expected.
C:/Users/rodri/AppData/Local/Yarn/Cache/v6/.tmp/21c2f99d30ecb4b33a45d8db9ea14fae.a79b92f5bd774e03113a196402cedbee82f8cf90.prepare/node_modules/@types/babel__traverse/index.d.ts(68,53): error TS2552: Cannot find name 'T'. Did you mean 't'?
C:/Users/rodri/AppData/Local/Yarn/Cache/v6/.tmp/21c2f99d30ecb4b33a45d8db9ea14fae.a79b92f5bd774e03113a196402cedbee82f8cf90.prepare/node_modules/@types/babel__traverse/index.d.ts(68,55): error TS2304: Cannot find name 'P'.
C:/Users/rodri/AppData/Local/Yarn/Cache/v6/.tmp/21c2f99d30ecb4b33a45d8db9ea14fae.a79b92f5bd774e03113a196402cedbee82f8cf90.prepare/node_modules/@types/babel__traverse/index.d.ts(68,58): error TS1005: ';' expected.
C:/Users/rodri/AppData/Local/Yarn/Cache/v6/.tmp/21c2f99d30ecb4b33a45d8db9ea14fae.a79b92f5bd774e03113a196402cedbee82f8cf90.prepare/node_modules/@types/babel__traverse/index.d.ts(68,66): error TS2693: 'any' only refers to a type, but is being used as a value here.
C:/Users/rodri/AppData/Local/Yarn/Cache/v6/.tmp/21c2f99d30ecb4b33a45d8db9ea14fae.a79b92f5bd774e03113a196402cedbee82f8cf90.prepare/node_modules/@types/babel__traverse/index.d.ts(68,70): error TS1011: An element access expression should take an argument.
C:/Users/rodri/AppData/Local/Yarn/Cache/v6/.tmp/21c2f99d30ecb4b33a45d8db9ea14fae.a79b92f5bd774e03113a196402cedbee82f8cf90.prepare/node_modules/@types/babel__traverse/index.d.ts(68,74): error TS2304: Cannot find name 'P'.
C:/Users/rodri/AppData/Local/Yarn/Cache/v6/.tmp/21c2f99d30ecb4b33a45d8db9ea14fae.a79b92f5bd774e03113a196402cedbee82f8cf90.prepare/node_modules/@types/babel__traverse/index.d.ts(68,78): error TS2693: 'never' only refers to a type, but is being used as a value here.
C:/Users/rodri/AppData/Local/Yarn/Cache/v6/.tmp/21c2f99d30ecb4b33a45d8db9ea14fae.a79b92f5bd774e03113a196402cedbee82f8cf90.prepare/node_modules/@types/babel__traverse/index.d.ts(68,83): error TS1005: ';' expected.
C:/Users/rodri/AppData/Local/Yarn/Cache/v6/.tmp/21c2f99d30ecb4b33a45d8db9ea14fae.a79b92f5bd774e03113a196402cedbee82f8cf90.prepare/node_modules/@types/babel__traverse/index.d.ts(68,84): error TS1128: Declaration or statement expected.
C:/Users/rodri/AppData/Local/Yarn/Cache/v6/.tmp/21c2f99d30ecb4b33a45d8db9ea14fae.a79b92f5bd774e03113a196402cedbee82f8cf90.prepare/node_modules/@types/babel__traverse/index.d.ts(68,86): error TS2304: Cannot find name 'P'.
C:/Users/rodri/AppData/Local/Yarn/Cache/v6/.tmp/21c2f99d30ecb4b33a45d8db9ea14fae.a79b92f5bd774e03113a196402cedbee82f8cf90.prepare/node_modules/@types/babel__traverse/index.d.ts(68,88): error TS1128: Declaration or statement expected.
TypeScript: 7 syntax errors
TypeScript: 8 semantic errors
TypeScript: emit succeeded (with errors)
[18:13:58] 'build' errored after 25 s
[18:13:58] Error: TypeScript: Compilation failed
    at Output.mightFinish (C:\Users\rodri\AppData\Local\Yarn\Cache\v6\.tmp\21c2f99d30ecb4b33a45d8db9ea14fae.a79b92f5bd774e03113a196402cedbee82f8cf90.prepare\node_modules\gulp-typescript\release\output.js:130:43)
    at C:\Users\rodri\AppData\Local\Yarn\Cache\v6\.tmp\21c2f99d30ecb4b33a45d8db9ea14fae.a79b92f5bd774e03113a196402cedbee82f8cf90.prepare\node_modules\gulp-typescript\release\output.js:65:22
[18:13:58] 'build' errored after 25 s
[18:13:58] 'bundle' errored after 25 s
[18:13:58] 'compress' errored after 25 s

rodbs · 2023-04-12T18:31:43Z

@glharper
I've managed to install it locally using npm instead of yarn.
I've got this error when running it; I think it's the same one I got weeks ago about no connecting using wss:

2023-04-12T18:14:30.103Z | SynthesisTriggeredEvent | privName: SynthesisTriggeredEvent | privEventId: 5AA0CE8D864B4D57B1B19E82B6E9221A | privEventTime: 2023-04-12T18nId: <NULL>
2023-04-12T18:14:30.103Z | ConnectingToSynthesisServiceEvent | privName: ConnectingToSynthesisServiceEvent | privEventId: 7737B2386C7345DF818FFD787FD32C05 | privEventTime: 2023-04-12T18:14:30.103Z | privEventType: 1 | privMetadata: {} | privRequestId: 2724BB03AA6441848676AD8057B6D0C0 | privAuthFetchEventId: 184280BD999D47FF8024030E371C4BEC
2023-04-12T18:14:30.103Z | ConnectionStartEvent | privName: ConnectionStartEvent | privEventId: 10C8B02F22DB464583E18A5DA011BCBA | privEventTime: 2023-04-12T18:14:30.103Z | privEventType: 1 | privMetadata: {} | privConnectionId: 6F7BFCA02DA24845A56B779B30143A0D | privUri: wss://westeurope.tts.speech.microsoft.com/cogniti

But when deploying to Cloudflare I got this other error; it seems it doesn't like to download from git


20:19:54.445 | npm WARN tarball tarball data for microsoft-cognitiveservices-speech-sdk@git+ssh://[email protected]/microsoft/cognitive-services-speech-sdk-js.git#a79b92f5bd774e03113a196402cedbee82f8cf90 (sha512-TKf5bid2fhEUxQGmtRMGT0qSocQc1qIhqkM+czbWyZqRv/nFiTT8wPYuv2rfo/5iC8y3F1/5dWUXwpULDFDOZQ==) seems to be corrupted. Trying again.
-- | --
20:22:15.348 | npm WARN tarball tarball data for microsoft-cognitiveservices-speech-sdk@git+ssh://[email protected]/microsoft/cognitive-services-speech-sdk-js.git#a79b92f5bd774e03113a196402cedbee82f8cf90 (sha512-TKf5bid2fhEUxQGmtRMGT0qSocQc1qIhqkM+czbWyZqRv/nFiTT8wPYuv2rfo/5iC8y3F1/5dWUXwpULDFDOZQ==) seems to be corrupted. Trying again.
20:22:26.517 | npm ERR! code EINTEGRITY
20:22:26.518 | npm ERR! sha512-TKf5bid2fhEUxQGmtRMGT0qSocQc1qIhqkM+czbWyZqRv/nFiTT8wPYuv2rfo/5iC8y3F1/5dWUXwpULDFDOZQ== integrity checksum failed when using sha512: wanted sha512-TKf5bid2fhEUxQGmtRMGT0qSocQc1qIhqkM+czbWyZqRv/nFiTT8wPYuv2rfo/5iC8y3F1/5dWUXwpULDFDOZQ== but got sha512-Irijnuz2COG7l3IHAORMADaGbR7f25fEnQioUBOGHgvaLjJI0Eh1qRdnh2uGZOPScP0Q7w4MD1JgfhHTcTB7EQ==. (1601293 bytes)
20:22:26.564 |  
20:22:26.564 | npm ERR! A complete log of this run can be found in:
20:22:26.565 | npm ERR!     /opt/buildhome/.npm/_logs/2023-04-12T18_22_26_529Z-debug.log

Is there any workaround to deploy it? otherwise I guess I'd need to wait to the final release ....

glharper · 2023-04-14T20:21:45Z

@rodbs We now have a 1.28-beta branch here.

rodbs · 2023-04-15T09:37:41Z

@glharper
I've seen the release and it works when deploying on cloudflare.
I haven't been able to test it thoroughly (the wss connection), though. Next week I'll be on a business trip so not sure if I'll be be able to test it. I'll let you know. Thx!

rodbs · 2023-04-24T16:14:00Z

@glharper

I've got this log when running the sdk in prod

(log) _action create_voice_full
  (log) fileName bad-a#bad-0-eng-AZR-female-low.mp3
  (log) null
  (info) 2023-04-24T16:07:21.586Z | SynthesisTriggeredEvent | privName: SynthesisTriggeredEvent | privEventId: 702C31FEAE1C40E083583AC835D6A0D5 | privEventTime: 2023-04-24T16:07:21.586Z | privEventType: 1 | privMetadata: {} | privRequestId: 7A2297F67B124C33876CAF496B7DABF6 | privSessionAudioDestinationId: <NULL> | privTurnAudioDestinationId: <NULL>
  (info) 2023-04-24T16:07:21.586Z | ConnectingToSynthesisServiceEvent | privName: ConnectingToSynthesisServiceEvent | privEventId: 504539E5E6024B3592DA0F1CD48879F8 | privEventTime: 2023-04-24T16:07:21.586Z | privEventType: 1 | privMetadata: {} | privRequestId: 7A2297F67B124C33876CAF496B7DABF6 | privAuthFetchEventId: 236341C884E147EB8C4FE8D675F6E904
  (log) {
  file: 'words/bad-a#bad-0-eng-AZR-female-low.mp3',
  voiceName: 'en-US-AriaNeural',
  voiceGender: 'female',
  voiceType: 'AZR',
  voiceSpeed: 'low'
}
  (info) 2023-04-24T16:07:21.586Z | ConnectionStartEvent | privName: ConnectionStartEvent | privEventId: FB2591A765C54753BBEBF4B3D14667B4 | privEventTime: 2023-04-24T16:07:21.586Z | privEventType: 1 | privMetadata: {} | privConnectionId: 7E9767F5499D4D44AB1AEADF2328152C | privUri: wss://westeurope.tts.speech.microsoft.com/cognitiveservices/websocket/v1?Ocp-Apim-Subscription-Key=f3686836b3e44035bd03972ff7d847e2&X-ConnectionId=7E9767F5499D4D44AB1AEADF2328152C | privHeaders: <NULL>
  (log) res batch {
  file: 'words/bad-a#bad-0-eng-AZR-female-low.mp3',
  voiceName: 'en-US-AriaNeural',
  voiceGender: 'female',
  voiceType: 'AZR',
  voiceSpeed: 'low'
}
  (info) 2023-04-24T16:07:22.084Z | ConnectionEstablishedEvent | privName: ConnectionEstablishedEvent | privEventId: B2D78D0CB0624B9A9E73ED93A9156416 | privEventTime: 2023-04-24T16:07:22.084Z | privEventType: 1 | privMetadata: {} | privConnectionId: 7E9767F5499D4D44AB1AEADF2328152C
  (info) 2023-04-24T16:07:22.084Z | SynthesisStartedEvent | privName: SynthesisStartedEvent | privEventId: 78F4D76C44B54761AE57FBAF38BAFB19 | privEventTime: 2023-04-24T16:07:22.084Z | privEventType: 1 | privMetadata: {} | privRequestId: 7A2297F67B124C33876CAF496B7DABF6 | privAuthFetchEventId: 236341C884E147EB8C4FE8D675F6E904
  (info) 2023-04-24T16:07:22.084Z | ConnectionMessageSentEvent | privName: ConnectionMessageSentEvent | privEventId: 4FD385050C4247F8ABE3FA7155041A6C | privEventTime: 2023-04-24T16:07:22.084Z | privEventType: 1 | privMetadata: {} | privConnectionId: 7E9767F5499D4D44AB1AEADF2328152C | privNetworkSentTime: 2023-04-24T16:07:22.084Z | privMessage: {"privBody":"{\"context\":{\"system\":{\"name\":\"SpeechSDK\",\"version\":\"1.28.0-beta.1\",\"build\":\"JavaScript\",\"lang\":\"JavaScript\"},\"os\":{\"platform\":\"Node/undefined\",\"name\":\"Cloudflare-Workers\"}}}","privMessageType":0,"privHeaders":{"Path":"speech.config","X-RequestId":"7A2297F67B124C33876CAF496B7DABF6","X-Timestamp":"2023-04-24T16:07:22.084Z","Content-Type":"application/json"},"privId":"DA30C36F329148D58231F50472556959","privSize":173,"privPath":"speech.config","privRequestId":"7A2297F67B124C33876CAF496B7DABF6","privContentType":"application/json"}    
  (log) RESULT [object Object]
  (log) 1 if
  (log) ERROR websocket send error: Websocket not ready 7E9767F5499D4D44AB1AEADF2328152C DA30C36F329148D58231F50472556959 Error
    at $e.sendRawMessage (functionsWorker-0.04974505925468953.js:33452:129)
    at $e.<anonymous> (functionsWorker-0.04974505925468953.js:33474:22)
    at Generator.next (<anonymous>)
    at c (functionsWorker-0.04974505925468953.js:33336:13)

I've tried several times and always get the same final error. (Websocket not ready)

I'm using this code:

  console.log(sdk.Diagnostics.SetLoggingLevel(sdk.LogLevel.Debug));

  const prom = new Promise((resolve, reject) => {
      const synthesizer = new sdk.SpeechSynthesizer(speechConfig, null);

    synthesizer.speakSsmlAsync(
      ssml,
      async (result) => {
        console.log('RESULT', result);
        if (result.errorDetails) {
          console.log('1 if ');
          console.log('ERROR', result.errorDetails);
          synthesizer.close();
        } else {
          console.log('2 if');
          const file = new File([result.audioData], fileName, {
            type: 'audio/mp3',
            lastModified: Date.now()
          });
          const res = await context.env.R2_IGLSBKT.put(fileName, file);
          console.log('res R2', res);
          synthesizer.close();  
          resolve(fileName);
        }
      },
      (error) => {
        console.log('err', error);
        synthesizer.close();
      }
    );
  });
  
  await context.waitUntil(prom);

glharper · 2023-04-25T12:30:58Z

@rodbs This sounds a lot like a networking issue between Cloudflare and the endpoint.
What's happening is a connection is established, then the websocket is abruptly closed before the first message can be sent. (That "websocket not ready" message can be found here.)There's not much I can do on the SDK side, other than ask random configuration questions like: Is there a max https connection time enforced?

rodbs · 2023-04-27T15:25:07Z

@glharper
I'm consulting Cloudflare's guys on Discord. Not success so far to make it work ...

glharper · 2023-05-02T18:03:23Z

@rodbs FYI 1.28.0 is now available via npm

rodbs · 2023-05-09T21:40:15Z

@glharper
I've tried version 1.28 but I get the same error. I have contacted Cloudflare on Discord, but no luck!

glharper · 2023-05-11T13:59:49Z

@rodbs I'm closing this issue, as it's morphed beyond anything that the SDK itself can address. If you come up with any code-resolvable issues, feel free to open a new issue. Thanks again.

nmfisher · 2023-12-18T02:31:02Z

If anyone else is coming up against the same problem, this seems to be an issue with Cloudflare workerd (see my comment here).

There's an open PR which looks like it should fix this.

TowhidKashem · 2023-12-30T07:07:47Z

hey @nmfisher, I tried your temp fix from the other thread:

WebSocket.prototype.CONNECTING = 0;
WebSocket.prototype.OPEN = 1;
WebSocket.prototype.CLOSING = 2;
WebSocket.prototype.CLOSED = 3;

which did make the errors go away fortunately but I still can't get the speakTextAsync method to work. When running it outside cloudflare workers it works fine but on cloudflare the response looks like this:

{
    "privResultId": "E9EC170FAD994582BA01D7C185D8B41F",
    "privReason": 10,
    "privAudioData": {},
    "privAudioDuration": 29125000
  }

vs how it usually looks when it's successful:

{
    privResultId: '4BD0845CB23E4DE1B35B3C255FA074B5',
    privReason: 10,
    privErrorDetails: undefined,
    privProperties: undefined,
    privAudioData: ArrayBuffer {
      [Uint8Contents]: <ff f3 c8 c4 00 79 5c 15 90 00 d6 b2 bc 3f f9 ce 2a d3 3e 4c 48 1a 5b 98 d2 65 03 95 09 8d 2e 69 91 19 80 50 c1 82 30 6a 9b 1b a6 86 a0 d1 92 24 67 11 1b 53 2a 49 88 3b 2c dd 04 89 10 60 07 19 20 ca 22 a3 e6 24 b9 99 26 64 49 98 f0 2f b2 a9 98 30 a0 e2 91 56 c8 61 8e 1a a4 c6 70 c1 9e 48 69 12 1a a5 ... 58652 more bytes>,
      byteLength: 58752
    },
    privAudioDuration: 35750000
}

were you able to actually get text to convert to audio fine?

nmfisher · 2023-12-31T07:05:31Z

@TowhidKashem yes this is working fine for me in production using speakSsmlAsync in Cloudflare Pages Functions.

I wouldn't expect any material difference between Functions and Pages Functions, so perhaps it's just some underlying issue with your use of the Azure SDK. Are you sure you are calling await at the correct place and resolving the async Promise correctly?

TowhidKashem · 2023-12-31T10:14:32Z

speakSsmlAsync

Ooof you know what I got it working, the issue was I forgot to add the credentials via the dashboard, I thought just adding them to .dev.vars and deploying would be enough. Anyway thanks its working now with ur temp fix!

glharper mentioned this issue Mar 9, 2023

remove blob method from IAudioSource and implementing classes #646

Merged

glharper closed this as completed in #646 Mar 10, 2023

glharper reopened this Mar 13, 2023

glharper closed this as completed May 11, 2023

nmfisher mentioned this issue Dec 18, 2023

🐛 Bug Report — Runtime APIs: names of WebSocket ready state constants differ from the Living Standard cloudflare/workerd#1370

Closed

SDK with Cloudflare workers and Fetch API #645

SDK with Cloudflare workers and Fetch API #645

Comments

rodbs commented Mar 9, 2023

glharper commented Mar 10, 2023

rodbs commented Mar 10, 2023

glharper commented Mar 13, 2023

rodbs commented Mar 13, 2023

glharper commented Mar 14, 2023

rodbs commented Mar 14, 2023

glharper commented Mar 15, 2023

rodbs commented Mar 15, 2023

glharper commented Mar 15, 2023 • edited Loading

rodbs commented Mar 15, 2023

glharper commented Mar 16, 2023 • edited Loading

rodbs commented Mar 16, 2023

glharper commented Mar 16, 2023

rodbs commented Mar 16, 2023

glharper commented Mar 17, 2023

rodbs commented Mar 17, 2023

glharper commented Mar 17, 2023

rodbs commented Mar 17, 2023

rodbs commented Apr 6, 2023

glharper commented Apr 9, 2023

glharper commented Apr 11, 2023

rodbs commented Apr 12, 2023

rodbs commented Apr 12, 2023 • edited Loading

glharper commented Apr 14, 2023 • edited Loading

rodbs commented Apr 15, 2023

rodbs commented Apr 24, 2023

glharper commented Apr 25, 2023

rodbs commented Apr 27, 2023

glharper commented May 2, 2023

rodbs commented May 9, 2023

glharper commented May 11, 2023

nmfisher commented Dec 18, 2023

TowhidKashem commented Dec 30, 2023 • edited Loading

nmfisher commented Dec 31, 2023

TowhidKashem commented Dec 31, 2023

glharper commented Mar 15, 2023 •

edited

Loading

glharper commented Mar 16, 2023 •

edited

Loading

rodbs commented Apr 12, 2023 •

edited

Loading

glharper commented Apr 14, 2023 •

edited

Loading

TowhidKashem commented Dec 30, 2023 •

edited

Loading