Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Android won't adhere to fixed buffer size #902

Open
zhpixel517 opened this issue Jul 27, 2024 · 0 comments
Open

Android won't adhere to fixed buffer size #902

zhpixel517 opened this issue Jul 27, 2024 · 0 comments

Comments

@zhpixel517
Copy link

zhpixel517 commented Jul 27, 2024 •
edited
Loading

Hi all. I've created this pretty simple program that just listens to audio from the microphone and plays it back out to the speakers. All it does is record a sample, pass it to a ring buffer, and the output callback will copy that to the output.

Take a look:

pub fn loopback_audio() {
    const BUFFER_SIZE: usize = 480;
    const RINGBUFFER_SIZE: usize = BUFFER_SIZE * 2;

    let (mut prod, mut cons) = RingBuffer::<f32>::new(RINGBUFFER_SIZE);
    let host = cpal::default_host();

    let input_device = host.default_input_device().unwrap();
    let output_device = host.default_output_device().unwrap();

    let input_config = cpal::StreamConfig {
        channels: 1,
        sample_rate: cpal::SampleRate(48000),
        buffer_size: cpal::BufferSize::Default
    };
    let output_config: cpal::StreamConfig = input_config.clone();

    let input_stream = input_device
        .build_input_stream(
            &input_config,
            move |data: &[f32], _: &_| {
                if let Ok(chunk) = prod.write_chunk_uninit(BUFFER_SIZE) {
                    chunk.fill_from_iter(data.to_owned()); // push to ring buffer
                }
            },
            move |err| {
                eprintln!("There was an input error: {:?}", err);
            },
            None,
        )
        .unwrap();

    let output_stream = output_device
        .build_output_stream(
            &output_config,
            move |data: &mut [f32], _: &cpal::OutputCallbackInfo| {
                debug!("Entered callback");
                let available = cons.slots();
                let to_read = data.len().min(available);

                match cons.read_chunk(to_read) {
                    Ok(chunk) => {
                        debug!("Read a chunk from the ringbuffer");
                        let (first, second) = chunk.as_slices(); //sort of like first half, second half, because rb is circular
                        let mid = first.len();  
                        data[..mid].copy_from_slice(first);
                        if second.len() != 0 {
                            data[mid..].copy_from_slice(second);
                        }
                        debug!("Copied all");
                        chunk.commit_all();
                        debug!("Committed all");
                    }
                    Err(error) => debug!("{} ", error),
                }
            },
            move |err| {
                debug!("There was an output error: {:?}", err);
            },
            None,
        )
        .unwrap();

    input_stream.play().unwrap();
    output_stream.play().unwrap();

    std::thread::sleep(Duration::from_secs(150));
}

However, when I run this, I can see that it runs a few times, but eventually crashes with this error message:

Launching lib/main.dart on Pixel 6 in debug mode...
Resolving dependencies...
Downloading packages...
  args 2.4.2 (2.5.0 available)
  collection 1.18.0 (1.19.0 available)
  github 9.17.0 (9.24.0 available)
  http 1.1.0 (1.2.2 available)
  http_parser 4.0.2 (4.1.0 available)
  path 1.8.0 (1.9.0 available)
  petitparser 5.4.0 (6.0.2 available)
  toml 0.14.0 (0.16.0 available)
  version 3.0.0 (3.0.2 available)
Got dependencies!
9 packages have newer versions incompatible with dependency constraints.
Try `dart pub outdated` for more information.
Compiling bin/build_tool_runner.dart to kernel file bin/build_tool_runner.dill.
INFO: Precompiled binaries are disabled
INFO: Building rust_lib_flutter_rust_test for aarch64-linux-android
INFO: Building rust_lib_flutter_rust_test for i686-linux-android
INFO: Building rust_lib_flutter_rust_test for x86_64-linux-android
✓ Built build/app/outputs/flutter-apk/app-debug.apk
Connecting to VM Service at ws://127.0.0.1:57042/zUXjnBVwEb0=/ws
I/OboeAudio(18953): openStreamInternal() INPUT -------- OboeVersion1.8.1 --------
D/OboeAudio(18953): AAudioLoader():  dlopen(libaaudio.so) returned 0x3c43ad50f5ab366f
I/AAudio  (18953): AAudioStreamBuilder_openStream() called ----------------------------------------
I/AudioStreamBuilder(18953): rate   =  48000, channels  = 1, channelMask = 0x80000001, format   = 5, sharing = SH, dir = INPUT
I/AudioStreamBuilder(18953): device =      0, sessionId = -1, perfMode = 10, callback: ON with frames = 0
I/AudioStreamBuilder(18953): usage  =      1, contentType = 2, inputPreset = 6, allowedCapturePolicy = 0
I/AudioStreamBuilder(18953): privacy sensitive = false, opPackageName = (null), attributionTag = (null)
D/AudioStreamBuilder(18953): build() MMAP not used because AAUDIO_PERFORMANCE_MODE_LOW_LATENCY not requested.
D/utter_rust_test(18953): PlayerBase::PlayerBase()
D/AAudioStream(18953): setState(s#1) from 0 to 2
I/AAudio  (18953): AAudioStreamBuilder_openStream() returns 0 = AAUDIO_OK for s#1 ----------------
D/OboeAudio(18953): AudioStreamAAudio.open() format=2, sampleRate=48000, capacity = 2880
D/OboeAudio(18953): calculateDefaultDelayBeforeCloseMillis() default = 21
D/OboeAudio(18953): AudioStreamAAudio.open: AAudioStream_Open() returned AAUDIO_OK = 0
I/OboeAudio(18953): openStreamInternal() OUTPUT -------- OboeVersion1.8.1 --------
I/AAudio  (18953): AAudioStreamBuilder_openStream() called ----------------------------------------
I/AudioStreamBuilder(18953): rate   =  48000, channels  = 1, channelMask = 0x80000001, format   = 5, sharing = SH, dir = OUTPUT
I/AudioStreamBuilder(18953): device =      0, sessionId = -1, perfMode = 10, callback: ON with frames = 0
I/AudioStreamBuilder(18953): usage  =      1, contentType = 2, inputPreset = 6, allowedCapturePolicy = 0
I/AudioStreamBuilder(18953): privacy sensitive = false, opPackageName = (null), attributionTag = (null)
D/AudioStreamBuilder(18953): build() MMAP not used because AAUDIO_PERFORMANCE_MODE_LOW_LATENCY not requested.
D/utter_rust_test(18953): PlayerBase::PlayerBase()
D/AudioStreamTrack(18953): open(), request notificationFrames = 0, frameCount = 0
D/AAudioStream(18953): setState(s#2) from 0 to 2
I/AAudio  (18953): AAudioStreamBuilder_openStream() returns 0 = AAUDIO_OK for s#2 ----------------
D/OboeAudio(18953): AudioStreamAAudio.open() format=2, sampleRate=48000, capacity = 3848
D/OboeAudio(18953): calculateDefaultDelayBeforeCloseMillis() default = 41
D/OboeAudio(18953): AudioStreamAAudio.open: AAudioStream_Open() returned AAUDIO_OK = 0
D/AAudio  (18953): AAudioStream_requestStart(s#1) called --------------
D/AAudioStream(18953): setState(s#1) from 2 to 3
D/AAudio  (18953): AAudioStream_requestStart(s#1) returned 0 ---------
D/AAudio  (18953): AAudioStream_requestStart(s#2) called --------------
D/AAudioStream(18953): setState(s#2) from 2 to 3
D/AudioStreamLegacy(18953): onAudioDeviceUpdate(deviceId = 22)
D/AudioStreamLegacy(18953): onAudioDeviceUpdate(deviceId = 22)
D/AAudio  (18953): AAudioStream_requestStart(s#2) returned 0 ---------
D/rust_lib_flutter_rust..(18953): Entered callback
D/rust_lib_flutter_rust..(18953): Read a chunk
D/rust_lib_flutter_rust..(18953): Copied all
D/rust_lib_flutter_rust..(18953): Committed all
D/rust_lib_flutter_rust..(18953): Entered callback
D/rust_lib_flutter_rust..(18953): Read a chunk
D/rust_lib_flutter_rust..(18953): Copied all
D/rust_lib_flutter_rust..(18953): Committed all
D/AudioStreamLegacy(18953): onAudioDeviceUpdate(deviceId = 3)
D/AAudioStream(18953): setState(s#1) from 3 to 4
D/rust_lib_flutter_rust..(18953): Entered callback
D/rust_lib_flutter_rust..(18953): Read a chunk
D/rust_lib_flutter_rust..(18953): Copied all
D/rust_lib_flutter_rust..(18953): Committed all
D/AAudioStream(18953): setState(s#2) from 3 to 4
D/rust_lib_flutter_rust..(18953): Entered callback
D/rust_lib_flutter_rust..(18953): Read a chunk
D/rust_lib_flutter_rust..(18953): Copied all
D/rust_lib_flutter_rust..(18953): Committed all
D/rust_lib_flutter_rust..(18953): Entered callback
D/rust_lib_flutter_rust..(18953): Read a chunk
D/rust_lib_flutter_rust..(18953): Copied all
D/rust_lib_flutter_rust..(18953): Committed all
D/rust_lib_flutter_rust..(18953): Entered callback
D/rust_lib_flutter_rust..(18953): Read a chunk
F/libc    (18953): Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 19092 (AudioTrack), pid 18953 (utter_rust_test)
*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
Build fingerprint: 'google/oriole/oriole:14/AP2A.240605.024/11860263:user/release-keys'
Revision: 'MP1.0'
ABI: 'arm64'
Timestamp: 2024-07-27 11:45:32.841996546-0600
Process uptime: 4s
Cmdline: com.example.flutter_rust_test
pid: 18953, tid: 19092, name: AudioTrack  >>> com.example.flutter_rust_test <<<
uid: 10341
tagged_addr_ctrl: 0000000000000001 (PR_TAGGED_ADDR_ENABLE)
signal 6 (SIGABRT), code -1 (SI_QUEUE), fault addr --------
    x0  0000000000000000  x1  0000000000004a94  x2  0000000000000006  x3  0000007556b749d0
    x4  0000007440dc95a8  x5  0000007440dc95a8  x6  0000007440dc95a8  x7  0000000000000000
    x8  00000000000000f0  x9  0000007820e25350  x10 0000000000000001  x11 0000007820e76170
    x12 0000000000000003  x13 0000000000000012  x14 0000000000000000  x15 000000000000000c
    x16 0000007820edcfd0  x17 0000007820ec8560  x18 000000742d5e4000  x19 0000000000004a09
    x20 0000000000004a94  x21 00000000ffffffff  x22 0000000000000000  x23 0000007440dfed68
    x24 0000007556b74c70  x25 0000007440c626d8  x26 0000007828ceb780  x27 0000000000000000
    x28 0000000000000b40  x29 0000007556b74a50
    lr  0000007820e5f8b8  sp  0000007556b749b0  pc  0000007820e5f8e4  pst 0000000000001000
28 total frames
backtrace:
      #00 pc 000000000005d8e4  /apex/com.android.runtime/lib64/bionic/libc.so (abort+164) (BuildId: 1d36f8ae6e0af6158793abea7d4f4f2b)
      #01 pc 00000000001b4e98  /data/app/~~t_t-ZausPkFT7OqxtUcQ4A==/com.example.flutter_rust_test-6UUUaPpOY4r5WiJSCaJqNg==/lib/arm64/librust_lib_flutter_rust_test.so
      #02 pc 00000000001b29a0  /data/app/~~t_t-ZausPkFT7OqxtUcQ4A==/com.example.flutter_rust_test-6UUUaPpOY4r5WiJSCaJqNg==/lib/arm64/librust_lib_flutter_rust_test.so
      #03 pc 00000000001b2888  /data/app/~~t_t-ZausPkFT7OqxtUcQ4A==/com.example.flutter_rust_test-6UUUaPpOY4r5WiJSCaJqNg==/lib/arm64/librust_lib_flutter_rust_test.so
      #04 pc 00000000001b25b4  /data/app/~~t_t-ZausPkFT7OqxtUcQ4A==/com.example.flutter_rust_test-6UUUaPpOY4r5WiJSCaJqNg==/lib/arm64/librust_lib_flutter_rust_test.so
      #05 pc 00000000001b105c  /data/app/~~t_t-ZausPkFT7OqxtUcQ4A==/com.example.flutter_rust_test-6UUUaPpOY4r5WiJSCaJqNg==/lib/arm64/librust_lib_flutter_rust_test.so
      #06 pc 00000000001b2340  /data/app/~~t_t-ZausPkFT7OqxtUcQ4A==/com.example.flutter_rust_test-6UUUaPpOY4r5WiJSCaJqNg==/lib/arm64/librust_lib_flutter_rust_test.so
      #07 pc 00000000001cde48  /data/app/~~t_t-ZausPkFT7OqxtUcQ4A==/com.example.flutter_rust_test-6UUUaPpOY4r5WiJSCaJqNg==/lib/arm64/librust_lib_flutter_rust_test.so
      #08 pc 00000000001d04c4  /data/app/~~t_t-ZausPkFT7OqxtUcQ4A==/com.example.flutter_rust_test-6UUUaPpOY4r5WiJSCaJqNg==/lib/arm64/librust_lib_flutter_rust_test.so
      #09 pc 00000000000c5018  /data/app/~~t_t-ZausPkFT7OqxtUcQ4A==/com.example.flutter_rust_test-6UUUaPpOY4r5WiJSCaJqNg==/lib/arm64/librust_lib_flutter_rust_test.so
      #10 pc 00000000000bda7c  /data/app/~~t_t-ZausPkFT7OqxtUcQ4A==/com.example.flutter_rust_test-6UUUaPpOY4r5WiJSCaJqNg==/lib/arm64/librust_lib_flutter_rust_test.so
      #11 pc 00000000000b1ff0  /data/app/~~t_t-ZausPkFT7OqxtUcQ4A==/com.example.flutter_rust_test-6UUUaPpOY4r5WiJSCaJqNg==/lib/arm64/librust_lib_flutter_rust_test.so
      #12 pc 00000000000a7df4  /data/app/~~t_t-ZausPkFT7OqxtUcQ4A==/com.example.flutter_rust_test-6UUUaPpOY4r5WiJSCaJqNg==/lib/arm64/librust_lib_flutter_rust_test.so
      #13 pc 00000000000b4880  /data/app/~~t_t-ZausPkFT7OqxtUcQ4A==/com.example.flutter_rust_test-6UUUaPpOY4r5WiJSCaJqNg==/lib/arm64/librust_lib_flutter_rust_test.so
      #14 pc 00000000000b324c  /data/app/~~t_t-ZausPkFT7OqxtUcQ4A==/com.example.flutter_rust_test-6UUUaPpOY4r5WiJSCaJqNg==/lib/arm64/librust_lib_flutter_rust_test.so
      #15 pc 0000000000196d8c  /data/app/~~t_t-ZausPkFT7OqxtUcQ4A==/com.example.flutter_rust_test-6UUUaPpOY4r5WiJSCaJqNg==/lib/arm64/librust_lib_flutter_rust_test.so
      #16 pc 00000000001731a4  /data/app/~~t_t-ZausPkFT7OqxtUcQ4A==/com.example.flutter_rust_test-6UUUaPpOY4r5WiJSCaJqNg==/lib/arm64/librust_lib_flutter_rust_test.so
      #17 pc 00000000001799fc  /data/app/~~t_t-ZausPkFT7OqxtUcQ4A==/com.example.flutter_rust_test-6UUUaPpOY4r5WiJSCaJqNg==/lib/arm64/librust_lib_flutter_rust_test.so
      #18 pc 0000000000179400  /data/app/~~t_t-ZausPkFT7OqxtUcQ4A==/com.example.flutter_rust_test-6UUUaPpOY4r5WiJSCaJqNg==/lib/arm64/librust_lib_flutter_rust_test.so
      #19 pc 000000000002de10  /system/lib64/libaaudio_internal.so (aaudio::AudioStream::maybeCallDataCallback(void*, int)+192) (BuildId: 0cf8e802c015dcff49dfa99f8e9b4983)
      #20 pc 00000000000321d0  /system/lib64/libaaudio_internal.so (aaudio::AudioStreamLegacy::callDataCallbackFrames(unsigned char*, int)+304) (BuildId: 0cf8e802c015dcff49dfa99f8e9b4983)
      #21 pc 00000000000310cc  /system/lib64/libaaudio_internal.so (aaudio::AudioStreamLegacy::onMoreData(android::AudioTrack::Buffer const&)+636) (BuildId: 0cf8e802c015dcff49dfa99f8e9b4983)
      #22 pc 00000000000a84f0  /system/lib64/libaudioclient.so (android::AudioTrack::processAudioBuffer()+2832) (BuildId: 3cbde63a227ac7432c8e29bb4c1fa488)
      #23 pc 00000000000a76e0  /system/lib64/libaudioclient.so (android::AudioTrack::AudioTrackThread::threadLoop()+272) (BuildId: 3cbde63a227ac7432c8e29bb4c1fa488)
      #24 pc 00000000000115d4  /system/lib64/libutils.so (android::Thread::_threadLoop(void*)+244) (BuildId: c07f08c7e5a964a8f8c6bc5c820fb795)
      #25 pc 00000000000edf3c  /system/lib64/libandroid_runtime.so (android::AndroidRuntime::javaThreadShell(void*)+140) (BuildId: 07fe69a1909e86b0aa90b83a17bd2e07)
      #26 pc 000000000006efbc  /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void*)+204) (BuildId: 1d36f8ae6e0af6158793abea7d4f4f2b)
      #27 pc 0000000000060d60  /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+64) (BuildId: 1d36f8ae6e0af6158793abea7d4f4f2b)
Lost connection to device.

Exited.

I honestly don't have any clue why this is happening. I did notice that the error only happened if the chunk.commit_all() was there, but that is a critical line because it frees the ring buffer back up. Without it, the error goes away but no sound is played. I don't think this is an issue with the ring buffer crate, because it seems to work multiple times before the whole program crashes. I took a look at this issue and this one as well, with no luck. I also tried this on an iOS device and it worked perfectly, so I'm not entirely sure why this isn't working on Android. I wish I had more information to share, but I honestly don't know anything else. Any ideas?

Edit: I found that this issue stemmed from this program not taking into account fluctuations of the length of the input data. At certain times the program would try and copy some empty data into the output buffer, and I think this caused the crash. This leads me to try and diagnose the next issue - if I were to print data.len() inside the data callback of input_stream:
debug!("input_config: len of input data is {}", data.len());
I get this output:

D/rust_lib_flutter_rust..(25504): input_config: len of input data is 960
D/rust_lib_flutter_rust..(25504): input_config: len of input data is 960
D/rust_lib_flutter_rust..(25504): input_config: len of input data is 960
D/rust_lib_flutter_rust..(25504): input_config: len of input data is 640
D/rust_lib_flutter_rust..(25504): input_config: len of input data is 320
D/rust_lib_flutter_rust..(25504): input_config: len of input data is 960
D/rust_lib_flutter_rust..(25504): input_config: len of input data is 960
D/rust_lib_flutter_rust..(25504): input_config: len of input data is 960
D/rust_lib_flutter_rust..(25504): input_config: len of input data is 896
D/rust_lib_flutter_rust..(25504): input_config: len of input data is 64

notice that it fluctuates a lot but tends to hang around 960. I did ask a similar question about this in another comment a few months back, but what really confuses me here in particular is the fact that I set the input config's buffer size to be fixed to 480:

const BUFFER_SIZE: u32 = 480;
let input_config = cpal::StreamConfig {
        channels: 1,
        sample_rate: cpal::SampleRate(48000),
        buffer_size: cpal::BufferSize::Fixed(BUFFER_SIZE as u32)
    };

yet, it seems like this isn't being adhered to at all in the input. is this behavior something that I would just have to make my program take into account and work around, or is something wrong here?
@zhpixel517 zhpixel517 changed the title Android AudioTrack crash Android won't adhere to fixed buffer size Jul 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@zhpixel517