Soundfont Sustain

[jonahdlin]

Hey there, love this library! I'm working on an application which needs a tidy library for playing back soundfonts with sustain. This library checks all the boxes of what I'm looking for except that critical one. Is soundfont sustain going to be worked on anytime soon?

[danigb]

There's a partial implementation https://github.com/danigb/smplr#soundfont-sustained-notes, you can track some of the progress and problems here #23

Summary is: some instruments works, some others doesn't. If you give it a try, let me know how it goes.

[jonahdlin]

I've been playing around with that feature. I found some bugs with the following instruments using MusyngKite:

Trumpet

• When looping C4 or D4, the note changes pitch to C#4 after a second or so
• Many other notes change pitch more drastically when held and sound robotic

Trombone, Tuba

• The loop has a noticeable gap on any note

Violin

• The loop repeats often, and has a noticeable gap on any note

Viola

• A clicking sound begins after the initial attack on any note

Rock organ

• Clicking sound beings after initial attack on any note

Honestly, all soundfont instruments I tested have some bug with loop like the above.

[danigb]

Hi,

thanks a lot for testing this out. I think most of the problems (if not all) comes from the looping data points. So first question: where did you get that looping data?

I don't have time (and honestly, motivation) to create that data by myself. But I'd be more than happy to accept PRs if someone want to do it (I know @goldst made one available, but from my tests, data is not ready or accurate yet).

One tedious way to do it is by opening original Soundfont files, trying to find those values and then tweaking by ear.

Anyway, one problem looks suspicious to me: the pitch change. I'll try to reproduce and fix it (if is a code problem). Can you give me the steps to reproduce your results?

Thanks! 🙌

[goldst]

Hello, I'm the one who tried to extract the loop points previously. I (until now) unsuccessfully tried to optimize this and because I have been mentioned and there still seems to be demand for this, think it would be helpful to share some more info on why the remaining work is unfortunately more challenging than expected. This is also related to the topic of tweaking by ear.

I took the loop points that I linked in the issue from the original sound fonts. However, the audio samples contained in the original soundfont are different from the one we are using:

The original soundfonts only contain audio samples for a few pitches per instrument. When sending a midi signal, a normal sampler would convert a sample to the correct pitch, loop it, add attack/release, and some effects like LFO and reverb that are all defined in the sound font.

The MIDI.js soundfonts we are using contain the rendered samples. This means, pitch adjustment, looping (they all have the same length), LFO, reverb, etc are already rendered into the audio file. This audio file also doesn't always have the same sample rate as the original one.

So what we are trying to do, is to loop on audio that looks completely different than the one the loop points are meant for. Even if we take into consideration the different sample rate, the additional effects like LFO and reverb would lead to unclean loops.

Under the given circumstances, I doubt that even searching manually for loop points will lead to very good results. I could imagine that re-exporting the original soundfont into a format like the one used in the smpldsnds repos would make more sense because then you can work with the raw samples. I might continue experimenting with that in the future, I can't guarantee that I will though.

[jonahdlin]

No problem! The pitch change one is strange. You can reproduce by

1. Navigate to https://danigb.github.io/smplr/
2. Load the Soundfont keyboard
3. Select "Trumpet" from the dropdown
4. Ensure "Loop" is checked, but it should be by default
5. Press the first key on the leftmost side of the keyboard

It should play a consistent C, but instead it plays a C for a moment, then switches to a C# and holds that.

Note this also happens when you press D, it switches to C# after a while.

[jonahdlin]

@goldst That's very helpful! I'm new to all this audio stuff. So to confirm, the original soundfont contains:

1. A few samples per instrument
2. Instructions for how to modify the those few samples to reproduce any pitch

But this library contains the output of playing a soundfont on an actual MIDI device. So rather than load a few samples it loads one sample for each note, rendering the original loop points useless.

Perhaps it's something I can work on in the future then to use the original soundfont files. For my purposes I need to play many instruments at once, which to my understanding using this library to do that would use an untenable amount of memory.