Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Achieving lowest possible latency on both Linux and Windows #13

Closed
jameshball opened this issue Jun 11, 2021 · 4 comments
Closed

Achieving lowest possible latency on both Linux and Windows #13

jameshball opened this issue Jun 11, 2021 · 4 comments

Comments

@jameshball
Copy link
Contributor

In my program, I'm currently attempting to set up the XtService in order of lowest->highest latency.

This code shows what I've tried to do: https://github.com/jameshball/osci-render/blob/06b0fdf6dfa8f21bfac2d26b1349d2e5c4a8d61b/src/main/java/sh/ball/audio/engine/XtAudioEngine.java#L98

Essentially, I'm doing the following, using the Core docs to guide my reasonings:

  • First trying to setup to SYSTEM_AUDIO, as on Windows with WASAPI this is by far the lowest latency in my experience.
  • If that fails, try to setup PRO_AUDIO, as this is also listed as being 'focused on low latency'
  • Finally, if it still fails, setup CONSUMER_AUDIO as low latency is less important

This achieves the intended result on Windows, where the latency from WASAPI is significantly lower than with DirectSound, however, when trying this on multiple Linux machines, if ALSA is used (SYSTEM_AUDIO) there is a very significant delay. I would estimate delays from ALSA to be more than a second. If I instead switch to PulseAudio (CONSUMER_AUDIO) then the latency is massively improved and is nearer the latency DirectSound on Windows (approx. 0.2 seconds?).

I am confused as to why this is the case, as I believe PulseAudio is simply a wrapper around ALSA? I have tried my program on three different Linux machines on different distros with the same result.

Do you have any advice or clarity on what APIs you think I should be using and when, or why you think the latency for ALSA is so large?

Thank you so much!

@sjoerdvankreel
Copy link
Owner

Hey, the alsa default output device on many linux distros is a virtual device that feeds back into the pulseaudio soundserver which in turn rewires the audio back to alsa to an actual hardware device. Try selecting one of the hw:x,y devices instead, on my dev box i can get easily below 6ms using plain alsa. Beware of the plughw: devices, these may add additional latency. Also keep in mind that pulseaudio "hijacks" the would-be-default-output-device (i.e. actual hardware device) to make its virtual device the default output device. Look for pasuspender module or if you have the luxury, set up a machine with PA completely removed from it if low latency is your primary concern.

@jameshball
Copy link
Contributor Author

Ah, that makes sense!

How can I get the default ALSA hardware device, i.e. hw:0,0 using XtAudio? Currently I've looped through all available devices and there are some hw: devices listed but all in the following forms:

hw:CARD=HDMI,DEV=7,TYPE=2
hw:CARD=PCH,DEV=0,TYPE=2
etc.

@sjoerdvankreel
Copy link
Owner

sjoerdvankreel commented Jun 11, 2021

You can't. Xt-Audio searches for the first device called "default", otherwise selects index 0. That's whats get reported by XtService.GetDefaultDevice. So in your case that might just be the PulseAudio device. You'll have to manually select the desired device based on the output of XtDeviceList.GetName, in your case i'd say you want the "PCH" device, hdmi is probably your monitor. If XtDeviceList.GetName doesnt provide enough info, you can match its output against the "aplay -l" command which provides a bit more details. Heres some example output from my dev box:

XT-Audio:
ALSA: surround21 (Input R/W)
ALSA: surround21 (Input MMap)
ALSA: surround21 (Output R/W)
ALSA: surround21 (Output MMap)
ALSA: surround40 (Input R/W)
ALSA: surround40 (Input MMap)
ALSA: surround40 (Output R/W)
ALSA: surround40 (Output MMap)
ALSA: surround41 (Input R/W)
ALSA: surround41 (Input MMap)
ALSA: surround41 (Output R/W)
ALSA: surround41 (Output MMap)
ALSA: surround50 (Input R/W)
ALSA: surround50 (Input MMap)
ALSA: surround50 (Output R/W)
ALSA: surround50 (Output MMap)
ALSA: surround51 (Input R/W)
ALSA: surround51 (Input MMap)
ALSA: surround51 (Output R/W)
ALSA: surround51 (Output MMap)
ALSA: surround71 (Input R/W)
ALSA: surround71 (Input MMap)
ALSA: surround71 (Output R/W)
ALSA: surround71 (Output MMap)
ALSA: null (Input R/W)
ALSA: null (Input MMap)
ALSA: null (Output R/W)
ALSA: null (Output MMap)
ALSA: samplerate (Input R/W)
ALSA: samplerate (Input MMap)
ALSA: samplerate (Output R/W)
ALSA: samplerate (Output MMap)
ALSA: speexrate (Input R/W)
ALSA: speexrate (Input MMap)
ALSA: speexrate (Output R/W)
ALSA: speexrate (Output MMap)
ALSA: jack (Input R/W)
ALSA: jack (Input MMap)
ALSA: jack (Output R/W)
ALSA: jack (Output MMap)
ALSA: oss (Input R/W)
ALSA: oss (Input MMap)
ALSA: oss (Output R/W)
ALSA: oss (Output MMap)
ALSA: pulse (Input R/W)
ALSA: pulse (Input MMap)
ALSA: pulse (Output R/W)
ALSA: pulse (Output MMap)
ALSA: upmix (Input R/W)
ALSA: upmix (Input MMap)
ALSA: upmix (Output R/W)
ALSA: upmix (Output MMap)
ALSA: vdownmix (Input R/W)
ALSA: vdownmix (Input MMap)
ALSA: vdownmix (Output R/W)
ALSA: vdownmix (Output MMap)
ALSA: hdmi:CARD=HDMI,DEV=0 (Output R/W)
ALSA: hdmi:CARD=HDMI,DEV=0 (Output MMap)
ALSA: hdmi:CARD=HDMI,DEV=1 (Output R/W)
ALSA: hdmi:CARD=HDMI,DEV=1 (Output MMap)
ALSA: hdmi:CARD=HDMI,DEV=2 (Output R/W)
ALSA: hdmi:CARD=HDMI,DEV=2 (Output MMap)
ALSA: hdmi:CARD=HDMI,DEV=3 (Output R/W)
ALSA: hdmi:CARD=HDMI,DEV=3 (Output MMap)
ALSA: hdmi:CARD=HDMI,DEV=4 (Output R/W)
ALSA: hdmi:CARD=HDMI,DEV=4 (Output MMap)
ALSA: dmix:CARD=HDMI,DEV=3 (Output R/W)
ALSA: dmix:CARD=HDMI,DEV=3 (Output MMap)
ALSA: dmix:CARD=HDMI,DEV=7 (Output R/W)
ALSA: dmix:CARD=HDMI,DEV=7 (Output MMap)
ALSA: dmix:CARD=HDMI,DEV=8 (Output R/W)
ALSA: dmix:CARD=HDMI,DEV=8 (Output MMap)
ALSA: dmix:CARD=HDMI,DEV=9 (Output R/W)
ALSA: dmix:CARD=HDMI,DEV=9 (Output MMap)
ALSA: dmix:CARD=HDMI,DEV=10 (Output R/W)
ALSA: dmix:CARD=HDMI,DEV=10 (Output MMap)
ALSA: dsnoop:CARD=HDMI,DEV=3 (Output R/W)
ALSA: dsnoop:CARD=HDMI,DEV=3 (Output MMap)
ALSA: dsnoop:CARD=HDMI,DEV=7 (Output R/W)
ALSA: dsnoop:CARD=HDMI,DEV=7 (Output MMap)
ALSA: dsnoop:CARD=HDMI,DEV=8 (Output R/W)
ALSA: dsnoop:CARD=HDMI,DEV=8 (Output MMap)
ALSA: dsnoop:CARD=HDMI,DEV=9 (Output R/W)
ALSA: dsnoop:CARD=HDMI,DEV=9 (Output MMap)
ALSA: dsnoop:CARD=HDMI,DEV=10 (Output R/W)
ALSA: dsnoop:CARD=HDMI,DEV=10 (Output MMap)
ALSA: hw:CARD=HDMI,DEV=3 (Output R/W)
ALSA: hw:CARD=HDMI,DEV=3 (Output MMap)
ALSA: hw:CARD=HDMI,DEV=7 (Output R/W)
ALSA: hw:CARD=HDMI,DEV=7 (Output MMap)
ALSA: hw:CARD=HDMI,DEV=8 (Output R/W)
ALSA: hw:CARD=HDMI,DEV=8 (Output MMap)
ALSA: hw:CARD=HDMI,DEV=9 (Output R/W)
ALSA: hw:CARD=HDMI,DEV=9 (Output MMap)
ALSA: hw:CARD=HDMI,DEV=10 (Output R/W)
ALSA: hw:CARD=HDMI,DEV=10 (Output MMap)
ALSA: plughw:CARD=HDMI,DEV=3 (Output R/W)
ALSA: plughw:CARD=HDMI,DEV=3 (Output MMap)
ALSA: plughw:CARD=HDMI,DEV=7 (Output R/W)
ALSA: plughw:CARD=HDMI,DEV=7 (Output MMap)
ALSA: plughw:CARD=HDMI,DEV=8 (Output R/W)
ALSA: plughw:CARD=HDMI,DEV=8 (Output MMap)
ALSA: plughw:CARD=HDMI,DEV=9 (Output R/W)
ALSA: plughw:CARD=HDMI,DEV=9 (Output MMap)
ALSA: plughw:CARD=HDMI,DEV=10 (Output R/W)
ALSA: plughw:CARD=HDMI,DEV=10 (Output MMap)
ALSA: usbstream:CARD=HDMI (Input R/W)
ALSA: usbstream:CARD=HDMI (Input MMap)
ALSA: usbstream:CARD=HDMI (Output R/W)
ALSA: usbstream:CARD=HDMI (Output MMap)
ALSA: default:CARD=PCH (Input R/W)
ALSA: default:CARD=PCH (Input MMap)
ALSA: default:CARD=PCH (Output R/W)
ALSA: default:CARD=PCH (Output MMap)
ALSA: sysdefault:CARD=PCH (Input R/W)
ALSA: sysdefault:CARD=PCH (Input MMap)
ALSA: sysdefault:CARD=PCH (Output R/W)
ALSA: sysdefault:CARD=PCH (Output MMap)
ALSA: front:CARD=PCH,DEV=0 (Input R/W)
ALSA: front:CARD=PCH,DEV=0 (Input MMap)
ALSA: front:CARD=PCH,DEV=0 (Output R/W)
ALSA: front:CARD=PCH,DEV=0 (Output MMap)
ALSA: surround21:CARD=PCH,DEV=0 (Output R/W)
ALSA: surround21:CARD=PCH,DEV=0 (Output MMap)
ALSA: surround40:CARD=PCH,DEV=0 (Output R/W)
ALSA: surround40:CARD=PCH,DEV=0 (Output MMap)
ALSA: surround41:CARD=PCH,DEV=0 (Output R/W)
ALSA: surround41:CARD=PCH,DEV=0 (Output MMap)
ALSA: surround50:CARD=PCH,DEV=0 (Output R/W)
ALSA: surround50:CARD=PCH,DEV=0 (Output MMap)
ALSA: surround51:CARD=PCH,DEV=0 (Output R/W)
ALSA: surround51:CARD=PCH,DEV=0 (Output MMap)
ALSA: surround71:CARD=PCH,DEV=0 (Output R/W)
ALSA: surround71:CARD=PCH,DEV=0 (Output MMap)
ALSA: iec958:CARD=PCH,DEV=0 (Output R/W)
ALSA: iec958:CARD=PCH,DEV=0 (Output MMap)
ALSA: dmix:CARD=PCH,DEV=0 (Input R/W)
ALSA: dmix:CARD=PCH,DEV=0 (Input MMap)
ALSA: dmix:CARD=PCH,DEV=0 (Output R/W)
ALSA: dmix:CARD=PCH,DEV=0 (Output MMap)
ALSA: dmix:CARD=PCH,DEV=1 (Output R/W)
ALSA: dmix:CARD=PCH,DEV=1 (Output MMap)
ALSA: dmix:CARD=PCH,DEV=2 (Input R/W)
ALSA: dmix:CARD=PCH,DEV=2 (Input MMap)
ALSA: dsnoop:CARD=PCH,DEV=0 (Input R/W)
ALSA: dsnoop:CARD=PCH,DEV=0 (Input MMap)
ALSA: dsnoop:CARD=PCH,DEV=0 (Output R/W)
ALSA: dsnoop:CARD=PCH,DEV=0 (Output MMap)
ALSA: dsnoop:CARD=PCH,DEV=1 (Output R/W)
ALSA: dsnoop:CARD=PCH,DEV=1 (Output MMap)
ALSA: dsnoop:CARD=PCH,DEV=2 (Input R/W)
ALSA: dsnoop:CARD=PCH,DEV=2 (Input MMap)
ALSA: hw:CARD=PCH,DEV=0 (Input R/W)
ALSA: hw:CARD=PCH,DEV=0 (Input MMap)
ALSA: hw:CARD=PCH,DEV=0 (Output R/W)
ALSA: hw:CARD=PCH,DEV=0 (Output MMap)
ALSA: hw:CARD=PCH,DEV=1 (Output R/W)
ALSA: hw:CARD=PCH,DEV=1 (Output MMap)
ALSA: hw:CARD=PCH,DEV=2 (Input R/W)
ALSA: hw:CARD=PCH,DEV=2 (Input MMap)
ALSA: plughw:CARD=PCH,DEV=0 (Input R/W)
ALSA: plughw:CARD=PCH,DEV=0 (Input MMap)
ALSA: plughw:CARD=PCH,DEV=0 (Output R/W)
ALSA: plughw:CARD=PCH,DEV=0 (Output MMap)
ALSA: plughw:CARD=PCH,DEV=1 (Output R/W)
ALSA: plughw:CARD=PCH,DEV=1 (Output MMap)
ALSA: plughw:CARD=PCH,DEV=2 (Input R/W)
ALSA: plughw:CARD=PCH,DEV=2 (Input MMap)
ALSA: usbstream:CARD=PCH (Input R/W)
ALSA: usbstream:CARD=PCH (Input MMap)
ALSA: usbstream:CARD=PCH (Output R/W)
ALSA: usbstream:CARD=PCH (Output MMap)

aplay -l (also check aplay -L):
**** List of PLAYBACK Hardware Devices ****
card 0: HDMI [HDA Intel HDMI], device 3: HDMI 0 [HDMI 0]
Subdevices: 1/1
Subdevice #0: subdevice #0
card 0: HDMI [HDA Intel HDMI], device 7: HDMI 1 [HDMI 1]
Subdevices: 1/1
Subdevice #0: subdevice #0
card 0: HDMI [HDA Intel HDMI], device 8: HDMI 2 [HDMI 2]
Subdevices: 1/1
Subdevice #0: subdevice #0
card 0: HDMI [HDA Intel HDMI], device 9: HDMI 3 [HDMI 3]
Subdevices: 1/1
Subdevice #0: subdevice #0
card 0: HDMI [HDA Intel HDMI], device 10: HDMI 4 [HDMI 4]
Subdevices: 1/1
Subdevice #0: subdevice #0
card 1: PCH [HDA Intel PCH], device 0: ALC892 Analog [ALC892 Analog]
Subdevices: 1/1
Subdevice #0: subdevice #0
card 1: PCH [HDA Intel PCH], device 1: ALC892 Digital [ALC892 Digital]
Subdevices: 1/1
Subdevice #0: subdevice #0

In this case PCH is the on-board audio device.

@jameshball
Copy link
Contributor Author

That makes more sense, thanks a lot for the clarification!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants