Music crackling/popping

[Novicek]

Preliminary checks

• I've checked that there aren't other open issues on the same topic.
• I've checked that this issue is reproducible on the latest snapshot build.

Platform

Windows

Describe the bug

Most of the music in this version of the game has a very noticeable crackle, which can be noticed only few seconds into the main menu music for example, but happens during all the music in game I heard so far.

Of course I've tried first if the issue is on my side, but was not able to resolve it: My audio drivers are up to date. I have no audio enhancements on my audio device enabled. My format is set to 48000Hz 24Bits and I've tried almost every possible setting too (reducing channels to 16Bits, changing sample rate to 44100Hz or even 96000Hz).

However, switching from my headphones to my speakers, I didn't hear any crackle, so it is most likely on my side. I am still putting this here in case of a suggestion and also because the same music does not have crackle in the other versions of the game (GOG version for DOS/Windows are both fine), so it still possibly has something to do with the way the playback works in this project.

Save file

Save file not required to reproduce the issue.

Additional info

Playing the track (homm2_41.ogg) itself via a media player in Windows, I do not hear the crackle either, even with volume set to much higher than in game.

[zenseii]

Hey, @Novicek. So if I'm not mistaken your files are in .ogg format. I haven't heard about this issue with .ogg files before. You could try to download the higher quality .flac pack from the Heroes 2 download page on GOG among the extras and use that with fheroes2.

There's a script that comes with fheroes2 that automatically renames all the music files for you and imports them from the .zip archive.

[Novicek]

Hey @zenseii, thanks for the reply. I tried what you suggested and the result is still the same, I get crackle when the music plays through fheroes2.

Just to confirm I did it right, I downloaded the zip with flac files, ran the resource_extraction_toolset.bat, dropped the zip in there, which created a music.flac folder. I then backed the original music folder and renamed music.flac to music.

[zenseii]

Just to confirm I did it right

Yep, that's the right procedure.

Are you testing on the latest snapshot or version 1.0? Not that it should matter afaik.

What are the specs of your PC? I've had intermittent crackling because my PC was really old, but only when moving a hero on the adventure map, while you said this happens to you regularly on any screen?

[Novicek]

I tested on the snapshot before I posted, but then reverted to 1.0.0. I also experienced with older builds and it's the same everywhere.

I built my PC January 2021, with a Ryzen9 5900x, RTX 3070, 32GB RAM on the Gigabyte B550 AORUS ELITE motherboard. I don't get performance issues, it's just the music. Most prominent are the town and main menu music, I hear it a lot less on the adventure map and combat screen. Maybe I could record a video to demonstrate.

[zenseii]

In other words the PC specs is not the problem. Have you tried different headsets?

[Novicek]

I just did on my old earbuds. It's still there, though I hear it a bit less.

[Districh-ru]

The issue is in SDL2 (or maybe in the way we use it).
When I put the SDL2.dll file from new version (1.0 or latest commit) I get the cracks and also the frequency range of music playback is reduced to 11 kHz (as the game uses 22050 sampling frequency):

And when I put SDL2.dll file from v.0.9.21 all is OK:

[Districh-ru]

In audio.cpp we set: freq = 22050;.
I've tried to change it to 44100 and this issue has gone...
But checking the original sounds I've found that they have some, maybe mirrored, frequencies. The original sounds sample rate is 22050 Hz, so its frequency range is up to 11025 Hz, and we have frequencies up to 16000 Hz:

But these 11025-16000 Hz frequencies are about -20 dB, so it may be ignored for now.
Also when I used freq 96000 or 192000 Hz I sometimes had cracks in the game sounds, but the music was fine. So the SDL2 re-sampler might have some issues.

@oleg-derevenetz, @ihhub, could you please check this sound playback issue when you have time. Maybe we have to downgrade/upgrade the SDL2 or change the freq to 44100. But this may be not good for all platforms.

[oleg-derevenetz]

Maybe we have to downgrade/upgrade the SDL2 or change the freq to 44100. But this may be not good for all platforms.

We can't upgrade it because we already build the latest available release (2.26.1) for Windows. There were serious issues with their resampler in previous version(s), so I have to downgrade the SDL to 2.0.22 (IIRC), but we can't do this forever. In theory, we can change the frequency to 44100 for Windows only - at least temporarily.

[Districh-ru]

In theory, we can change the frequency to 44100 for Windows only - at least temporarily.

This could be a temporary solution. Taking into account that music in OGG files has 48000 Hz sampling rate, IMHO it is better to have this sampling rate in the game. But with current SDL2.dll this sampling rate causes cracks in the game sounds. :(
With 44100 Hz I did not notice any cracks in the sound. This frequency is also good for music files, as the highest frequency in them is about 20.5 kHz.
With SDL 2.0.22 the game always gives me the warning: Audio::Init: Audio frequency is initialized as 192000 instead of 22050. So downgrading the SDL will make the game to use the system sample rate.
Let's hope that in future versions of SDL the resampler will be fixed.

[oleg-derevenetz]

Support for the libsamplerate was added recently in the sdl2 vcpkg port, and building sdl2 with samplerate feature may resolve this issue, because libsamplerate will be used for resampling instead of the SDL built-in resampler implementation. However vcpkg team (or maybe I will if I succeed) will need to resolve this issue first, because glib is mandatory to build the FluidSynth plugin that is used by the SDL_mixer to play MIDI files.

[ihhub]

I have a feeling we are trying to fix issues in SDL library itself :( If this issue is Windows specific should we (possibly not now) use Windows OS libraries to play midi and music files, like DirectMusic?

[oleg-derevenetz]

@ihhub

use Windows OS libraries to play midi and music files, like DirectMusic?

We already tried to use Windows native MIDI (SDL_mixer uses it if no FluidSynth plugin or soundfont is present) and we all know that there is a ton of problems with it - it is slow and have issues with volume, that's why we moved to the FluidSynth at the time. Also SDL uses its own resampler implementation (and not the Windows resampler, although it can be used in the SDL code) because native resampler is even worse and brings even more distortion :)

I think it's better to spend relatively little time choosing the right plugins/libs for SDL instead of wasting time on some home-grown non-potable implementation that still will need to use the same libs at the end :)

P.S. Ahaha, they turned native Windows resampler on back in November, so now you hear the results of native Windows resampler :) It seems that "just use Windows native libs" is not an option in any case.

[oleg-derevenetz]

Although I was able to build SDL2 with libsamplerate support for Windows, I'm not sure how it will work and if libsamplerate resampler will be involved at all, because it seems that currently SDL2 uses Windows resampler instead of its own (and maybe instead of libsamplerate too) and it is hardcoded. So let's begin from a simpler approach: just set the audio frequency to 44100 for Windows only.

@Novicek @Districh-ru could you please test the build from #6455 with all supported music types (including OGG, MP3, FLAC and MIDI) as well as in-game sounds for these distortions?

[Novicek]

@oleg-derevenetz Tested all sound formats (though MIDI didn't give me crackle even beforehand far as I could tell) and all seems fine.

[Districh-ru]

@oleg-derevenetz, I was busy yesterday, so could not test the build.
I've tested current build and found that the sound clicks are almost inaudible, but they are still present in music (OGG, MP3, FLAC) - in spectrogram they are vertical lines:

But this is better, than with 22050 Hz sample rate:

And knowing this I'll prefer to use SDL2.dll v. 2.0.22. :(

For long time I've used WinAmp plugin 'YASAPI/NT' (https://sourceforge.net/projects/out-yasapi/), which uses WASAPI. And this plugin with default configuration also had some distortions, including mirrored frequencies. But in plugin configuration there is a setting "SRC Default Quality", enabling which made WinAmp sound perfect - without distortions and mirrored frequencies.

Maybe the authors of SDL should use the same setting to avoid sound distortions of resampler.

[oleg-derevenetz]

Hi @Districh-ru since you already have the infrastructure in place, could you please try the DLLs from this PR:

fheroes2/fheroes2-prebuilt-deps#28

and tell is it makes any difference? Please note that there is one new DLL (samplerate.dll) goes in addition.

[Districh-ru]

Hi @oleg-derevenetz, I tried these DLLs and found that there is no difference with the latest master build. Small clicks still present in music.

[oleg-derevenetz]

@Districh-ru OK, thanks! I also suspected that there will be no difference, because the use of native WASAPI resampler is currently hardcoded in SDL.

[oleg-derevenetz]

@Districh-ru

Maybe the authors of SDL should use the same setting to avoid sound distortions of resampler.

What this setting does is enable the use of the AUDCLNT_STREAMFLAGS_SRC_DEFAULT_QUALITY flag. This flag is already used in SDL:

    /* we've gotten reports that WASAPI's resampler introduces distortions, but in the short term
       it fixes some other WASAPI-specific quirks we haven't quite tracked down.
       Refer to bug #6326 for the immediate concern. */
#if 0
    this->spec.freq = waveformat->nSamplesPerSec;  /* force sampling rate so our resampler kicks in, if necessary. */
#else
    /* favor WASAPI's resampler over our own */
    if (this->spec.freq != waveformat->nSamplesPerSec) {
        streamflags |= (AUDCLNT_STREAMFLAGS_AUTOCONVERTPCM | AUDCLNT_STREAMFLAGS_SRC_DEFAULT_QUALITY);
        waveformat->nSamplesPerSec = this->spec.freq;
        waveformat->nAvgBytesPerSec = waveformat->nSamplesPerSec * waveformat->nChannels * (waveformat->wBitsPerSample / 8);
    }
#endif

And still the results are not good.

[Districh-ru]

So the issue is in some other place. Let's hope that the developers of SDL could fix it (libsdl-org/SDL#6326, libsdl-org/SDL#5538)

And maybe we should leave this issue opened as the bug still exists.

[oleg-derevenetz]

And maybe we should leave this issue opened as the bug still exists.

This bug exists in SDL indeed, but we applied a workaround which works good enough for an ordinary user. However, so be it.

[Districh-ru]

Hello @autoexecbatman, I unpacked the samples from fheroes2.sf3 (using RIFF and SF format descriptions) and checked them using Audacity and found no clicks in the samples, except the Applause.ogg which contains claps - so I can conclude that fheroes2.sf3 it is OK.

Here are the unpacked samples if you want to check them: unpacked.zip

[lothar-cell]

So, after reading through the thread, I still don't know how to fix the cracking issue (MacOS/Wine). Tried placing the FLAC files from GOG in the MUSIC folder, and the latest .ogg in unpacked.zip.

Can someone explain to a user how to fix the cracking audio?

[oleg-derevenetz]

Hi @lothar-cell

So, after reading through the thread, I still don't know how to fix the cracking issue (MacOS/Wine).

Is there any specific reason why you use such an exotic configuration instead of running fheroes2 directly on macOS?

[lothar-cell]

I couldn't make the MacOS version work with the .sh script and I don't know how to/don't want to build from source, and Wine with the .exe works out of the box.

[oleg-derevenetz]

I couldn't make the MacOS version work with the .sh script

What exactly do you mean? Where did you install fheroes2 from (MacPorts, Homebrew?) and what exactly is not working?

[lothar-cell]

1. downloaded the MacOS version from here: https://github.com/ihhub/fheroes2/releases/download/1.0.12/fheroes2_macos_x86-64_SDL2.zip
2. double click on install_sdl2.sh - Terminal launches in the background, nothing happens
3. dragged the script onto the Terminal window:

4. gave up and installed the win64 exe version with Wine straight away without problems: https://github.com/ihhub/fheroes2/releases/download/1.0.12/fheroes2_windows_x64_SDL2_installer.exe

[lothar-cell]

However, does this have anything to do with the sound issue? As I read it's a problem on all platforms.

[oleg-derevenetz]

1. downloaded the MacOS version from here

Have you tried installing fheroes2 using Homebrew or MacPorts? They will install all the necessary dependencies.

However, does this have anything to do with the sound issue? As I read it's a problem on all platforms.

I do not know where you got this from. There are several different issues observed by different people in different conditions - some issues are caused by the operation of the resampler on a certain platform (sampling rate was tuned at the time because of this), some occur only at maximum volume and only with MIDI music. Nobody knows what's going on with the audio "drivers" and the resampler in Wine, this configuration is a bit exotic.

[lothar-cell]

Have you tried installing fheroes2 using Homebrew or MacPorts? They will install all the necessary dependencies.

I installed it with Crossover, not using MacPorts, where can I find instructions for Homebrew?

[oleg-derevenetz]

where can I find instructions for Homebrew?

In our installation guide:

https://ihhub.github.io/fheroes2/INSTALL.html#homebrew-mac

[lothar-cell]

Thank you, I managed to install it and the audio issue is gone:)
But man, this was complicated, installs in a hidden folder to /usr/local/Cellar/fheroes2/1.0.12/

Can I move this whole Cellar folder from there, as it is 3.43GB?

[oleg-derevenetz]

But man, this was complicated, installs in a hidden folder to /usr/local/Cellar/fheroes2/1.0.12/
Can I move this whole Cellar folder from there, as it is 3.43GB?

The path to the installed apps is determined by package manager (Homebrew). I personally don't use Homebrew, I use MacPorts, but I suppose that you can try to move it to other place and create a symlink to the new location (sort of sudo ln -s /some/new/location/Cellar /usr/local/Cellar). Try to ask this question in the Homebrew support forum.

[lothar-cell]

Thanks @oleg-derevenetz !

[oleg-derevenetz]

I am closing this issue because it has, in fact, turned into a set of unrelated issues and only confuses people.

Original issue with SDL resampler on Windows should have been mitigated by #6455, and then there were additional fixes in the SDL itself related to resampling.

SDL_mixer issue with sound artifacts when playing MIDI music with max volume should have been fixed by libsdl-org/SDL_mixer#508 (credits to @Districh-ru) and this fix should be included into SDL_mixer 2.8.0, which is now used by our Windows and Android dependencies starting from #8381.

For new issues of a similar nature (if they appear), I recommend opening separate issues so as not to mix everything into one pile.