Add a method to get raw audio sample data

[4321ba]

Describe the project you are working on

An interactive wave to midi editor / music transcription software called MeloMIDI

Describe the problem or limitation you are having in your project

I needed to get the raw audio data (samples and sample rate) at runtime from the user's audio file, which is not possible with GDScript. I not only need it for playback (for which I used GDScriptAudioImport), but to process the audio non-realtime.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

It would help if the decoding could be done with the editor's built-in decoders (wav, ogg and mp3).

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

AudioStream (or AudioStreamMP3, AudioStreamOGGVorbis and AudioStreamSample) could have a method to get the samples as a PoolByteArray (and the sample rate as an int) (in case the decoders already have the functionality and it's just not exposed). Channels are a question.

If this enhancement will not be used often, can it be worked around with a few lines of script?

Well kindof, you can use a 3rd party c++ library, like I used libnyquist, it just seems a little bit wasteful to have 2 decoders for all the formats. I'm alright with it though, I need gdnative for an fft library anyway (the built-in spectrum analyzer is wayyy too imprecise for my niche use-case, and it's realtime too so the user would need to listen to the whole thing before anything else).
I don't know though if anyone needed this who doesn't use gdnative, I just thought I'd share my use case. It's fine to close this too if there's no way any other people found this useful :D.

Is there a reason why this should be core and not an add-on in the asset library?

It would affect core classes.

[awltux]

I'm interested in this too. I wondered if there were any thoughts on how i could implement it? I get the impression that a specialised AudioStreamPlay module might do the trick, but that might also integrate it with the AudioServer. I'm also not clear whether it could be forced to play in real time rather than sample time. (Noob to code base).

[Eclextic]

No, honestly I think anyone could easily just expose the byte array of these AudioStream classes…

Shouldn’t be really hard if we already store them for use…