UI <> AirWindows swap race condition crash

[mhetrick]

Bug Description:
I received a hard crash while flipping through Data Broth's Sequences presets. It looks like it triggered an access error in AirWindowsEffect::AWFxSelectorMapper::nameAtStreamedIndex(int).

Surge Synthesizer
Plugin Host: Ableton Live 11.0.2
System: macOS 64-bit VST3 on Apple M1
Build Info: Built on 2021-04-21 at 11:26:22, using pipeline host 'Mac-1619004058536.local' with 'AppleClang-12.0.0.12000032'
Version: 1.9.0.91069f8

Process:               Live [15320]
Path:                  /Applications/Ableton Live 11 Suite.app/Contents/MacOS/Live
Identifier:            com.ableton.live
Version:               11.0.2 [2021-03-26_5b7f2a59a5] (11.0.2 [2021-03-26_5b7f2a59a5])
Code Type:             X86-64 (Translated)
Parent Process:        ??? [1]
Responsible:           Live [15320]
User ID:               501

Date/Time:             2021-05-07 22:42:19.119 -0400
OS Version:            macOS 11.3 (20E232)
Report Version:        12
Anonymous UUID:        19FF7FB5-AF55-3176-931A-3213FCC8A025

Sleep/Wake UUID:       0AF2EC0B-0564-4A59-8EEF-B125B76DDF85

Time Awake Since Boot: 440000 seconds
Time Since Wake:       5900 seconds

System Integrity Protection: enabled

Crashed Thread:        0  MainThread  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
Exception Codes:       KERN_INVALID_ADDRESS at 0x00007fb772991028
Exception Note:        EXC_CORPSE_NOTIFY

Termination Signal:    Segmentation fault: 11
Termination Reason:    Namespace SIGNAL, Code 0xb
Terminating Process:   exc handler [15320]

VM Regions Near 0x7fb772991028:
    MALLOC_NANO (reserved)   600018000000-600020000000 [128.0M] rw-/rwx SM=NUL  reserved VM address space (unallocated)
--> 
    MALLOC_LARGE             7fc725c00000-7fc7268c0000 [ 12.8M] rw-/rwx SM=PRV  

Thread 0 Crashed:: MainThread  Dispatch queue: com.apple.main-thread
0   libc++.1.dylib                	0x00007fff2036f0d6 std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::basic_string(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 4
1   com.vemberaudio.vst3.surge    	0x00000001ac85bc15 AirWindowsEffect::AWFxSelectorMapper::nameAtStreamedIndex(int) + 37
2   com.vemberaudio.vst3.surge    	0x00000001ac6f9e39 Parameter::get_display(char*, bool, float) + 2569
3   com.vemberaudio.vst3.surge    	0x00000001ac594463 SurgeGUIEditor::getDisplayForTag(long) + 211
4   com.vemberaudio.vst3.surge    	0x00000001ac4d81e8 CMenuAsSlider::draw(VSTGUI::CDrawContext*) + 536
5   com.vemberaudio.vst3.surge    	0x00000001ac602a0c VSTGUI::CViewContainer::drawRect(VSTGUI::CDrawContext*, VSTGUI::CRect const&) + 2508
6   com.vemberaudio.vst3.surge    	0x00000001ac602013 VSTGUI::CFrame::drawRect(VSTGUI::CDrawContext*, VSTGUI::CRect const&) + 371
7   com.vemberaudio.vst3.surge    	0x00000001ac60a90d non-virtual thunk to VSTGUI::CFrame::platformDrawRect(VSTGUI::CDrawContext*, VSTGUI::CRect const&) + 13
8   com.vemberaudio.vst3.surge    	0x00000001ac66afbc VSTGUI::NSViewFrame::drawRect(CGRect*) + 252
9   com.vemberaudio.vst3.surge    	0x00000001ac66882a VSTGUI_NSView_drawRect(objc_object*, objc_selector*, CGRect) + 74
10  com.apple.AppKit              	0x00007fff22d63d1b _NSViewDrawRect + 139
11  com.apple.AppKit              	0x00007fff2349445c -[NSView _recursive:displayRectIgnoringOpacity:inContext:stopAtLayerBackedViews:] + 2115
12  com.apple.AppKit              	0x00007fff22d63335 -[NSView(NSLayerKitGlue) _drawViewBackingLayer:inContext:drawingHandler:] + 967
13  com.apple.QuartzCore          	0x00007fff26ea0de1 CABackingStoreUpdate_ + 219
14  com.apple.QuartzCore          	0x00007fff26f080f9 invocation function for block in CA::Layer::display_() + 53
15  com.apple.QuartzCore          	0x00007fff26ea02f3 -[CALayer _display] + 2247
16  com.apple.AppKit              	0x00007fff22d62ea5 -[_NSBackingLayer display] + 475
17  com.apple.AppKit              	0x00007fff22cce00f -[_NSViewBackingLayer display] + 555
18  com.apple.QuartzCore          	0x00007fff26e9f25f CA::Layer::display_if_needed(CA::Transaction*) + 863
19  com.apple.QuartzCore          	0x00007fff26fea44a CA::Context::commit_transaction(CA::Transaction*, double, double*) + 666
20  com.apple.QuartzCore          	0x00007fff26e8074f CA::Transaction::commit() + 713
21  com.apple.AppKit              	0x00007fff22d781dc __62+[CATransaction(NSCATransaction) NS_setFlushesWithDisplayLink]_block_invoke + 285
22  com.apple.AppKit              	0x00007fff234cc443 ___NSRunLoopObserverCreateWithHandler_block_invoke + 41
23  com.apple.CoreFoundation      	0x00007fff204ceb7d __CFRUNLOOP_IS_CALLING_OUT_TO_AN_OBSERVER_CALLBACK_FUNCTION__ + 23
24  com.apple.CoreFoundation      	0x00007fff204cea0d __CFRunLoopDoObservers + 543
25  com.apple.CoreFoundation      	0x00007fff204cde94 __CFRunLoopRun + 845
26  com.apple.CoreFoundation      	0x00007fff204cd480 CFRunLoopRunSpecific + 567
27  com.apple.HIToolbox           	0x00007fff28951203 RunCurrentEventLoopInMode + 292
28  com.apple.HIToolbox           	0x00007fff28950f65 ReceiveNextEventCommon + 587
29  com.apple.HIToolbox           	0x00007fff28950d03 _BlockUntilNextEventMatchingListInModeWithFilter + 70
30  com.apple.AppKit              	0x00007fff22bf3b32 _DPSNextEvent + 864
31  com.apple.AppKit              	0x00007fff22bf2305 -[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 1364
32  com.apple.AppKit              	0x00007fff22be4679 -[NSApplication run] + 586
33  com.ableton.live              	0x00000001016d49d0 0x100621000 + 17512912
34  libdyld.dylib                 	0x00007fff203f1f3d start + 1

[baconpaul]

First of all, thank you for providing a stack trace. It is very useful and appreciated.

I hate to ask such a pedestrian question but: were you changing presets quickly? I think 1.9 still has a race in live vst3 patch changes somewhere we could never find.

But let me add this to our list of issues to try and reproduce and address in our cycle. Thank you again!

[baconpaul]

huh so i just loaded xt into mac live 11 demo and went to data broth sequence patches and started pressing the 'next' arrow quickly

and it went blammo!

[baconpaul]

Thread 0 Crashed:: MainThread  Dispatch queue: com.apple.main-thread
0   libc++.1.dylib                	0x00007fff204400d6 std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::basic_string(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 4
1   org.surge-synth-team.surge-xt 	0x0000000229f64b0e AirWindowsEffect::AWFxSelectorMapper::nameAtStreamedIndex(int) const + 78 (AirWindowsEffect.h:65)
2   org.surge-synth-team.surge-xt 	0x000000022a1567d0 Parameter::get_display(char*, bool, float) const + 9984 (Parameter.cpp:3462)
3   org.surge-synth-team.surge-xt 	0x0000000229b4d110 SurgeGUIEditor::getDisplayForTag(long, bool, float) + 352 (SurgeGUIEditor.cpp:3636)
4   org.surge-synth-team.surge-xt 	0x0000000229e281bb Surge::Widgets::MenuForDiscreteParams::paint(juce::Graphics&) + 1035 (MenuForDiscreteParams.cpp:70)
5   org.surge-synth-team.surge-xt 	0x000000022944e001 juce::Component::paintComponentAndChildren(juce::Graphics&) + 289 (juce_Component.cpp:1963)
6   org.surge-synth-team.surge-xt 	0x000000022944dea4 juce::Component::paintEntireComponent(juce::Graphics&, bool) + 1028 (juce_Component.cpp:2060)
7   org.surge-synth-team.surge-xt 	0x000000022944da5f juce::Component::paintWithinParentContext(juce::Graphics&) + 143 (juce_Component.cpp:1947)
8   org.surge-synth-team.surge-xt 	0x000000022944e451 juce::Component::paintComponentAndChildren(juce::Graphics&) + 1393 (juce_Component.cpp:2005)
9   org.surge-synth-team.surge-xt 	0x000000022944dea4 juce::Component::paintEntireComponent(juce::Graphics&, bool) + 1028 (juce_Component.cpp:2060)
10  org.surge-synth-team.surge-xt 	0x000000022944da5f juce::Component::paintWithinParentContext(juce::Graphics&) + 143 (juce_Component.cpp:1947)
11  org.surge-synth-team.surge-xt 	0x000000022944e451 juce::Component::paintComponentAndChildren(juce::Graphics&) + 1393 (juce_Component.cpp:2005)
12  org.surge-synth-team.surge-xt 	0x000000022944dea4 juce::Component::paintEntireComponent(juce::Graphics&, bool) + 1028 (juce_Component.cpp:2060)
13  org.surge-synth-team.surge-xt 	0x000000022944da5f juce::Component::paintWithinParentContext(juce::Graphics&) + 143 (juce_Component.cpp:1947)
14  org.surge-synth-team.surge-xt 	0x000000022944e451 juce::Component::paintComponentAndChildren(juce::Graphics&) + 1393 (juce_Component.cpp:2005)
15  org.surge-synth-team.surge-xt 	0x000000022944dea4 juce::Component::paintEntireComponent(juce::Graphics&, bool) + 1028 (juce_Component.cpp:2060)
16  org.surge-synth-team.surge-xt 	0x000000022944da5f juce::Component::paintWithinParentContext(juce::Graphics&) + 143 (juce_Component.cpp:1947)
17  org.surge-synth-team.surge-xt 	0x000000022944e1b9 juce::Component::paintComponentAndChildren(juce::Graphics&) + 729 (juce_Component.cpp:1979)
18  org.surge-synth-team.surge-xt 	0x000000022944dea4 juce::Component::paintEntireComponent(juce::Graphics&, bool) + 1028 (juce_Component.cpp:2060)
19  org.surge-synth-team.surge-xt 	0x0000000229521a64 juce::ComponentPeer::handlePaint(juce::LowLevelGraphicsContext&) + 596 (juce_ComponentPeer.cpp:132)
20  org.surge-synth-team.surge-xt 	0x00000002295ed739 juce::NSViewComponentPeer::drawRect(CGContext*, CGRect, float) + 265 (juce_mac_NSViewComponentPeer.mm:924)
21  org.surge-synth-team.surge-xt 	0x00000002295ed613 juce::NSViewComponentPeer::drawRect(CGRect) + 435 (juce_mac_NSViewComponentPeer.mm:912)
22  org.surge-synth-team.surge-xt 	0x00000002295eaa62 juce::JuceNSViewClass::drawRect(objc_object*, objc_selector*, CGRect) + 130 (juce_mac_NSViewComponentPeer.mm:1858)

[baconpaul]

and of course no crash in reaper.

[baconpaul]

Oh cool I made reaper crash doing the same!

[baconpaul]

Same place. OK so this is nothing to do with live. I have a race condition between the UI and Audio thread at AirWindows reset, Lemme see if Standalone does it.

[baconpaul]

Yeah so it will crash if you are 'looking' at an fx slot which gets an Airwindows in and out of it as you run

  static int swap = 0;
    if (swap == 3)
    {
        synth->incrementPatch(true, true);
        swap = 0;
    }
    swap++;

pub that in idle, sit on slot A1, run standalone, and it will crash eventually. Cool.

Well not cool. This is a problem with direct memory access of the synth. But it gives me an idea.

[baconpaul]

oh i know why. has to do with how we set up parameters and the way airwindows has its mapper owned. basically we have a reference to a dangling pointer when we swap quickly.

i bet i get this one this week. but it is subtle. great bug report anyway. no more noise until i get a fix though.

[baconpaul]

got it