Spike - investigate React Native + keyboard

[TKDickson]

In fixing and testing #9288, we found further issues with RN and keyboard on both platforms, but Android seemed to have more issues.

Generally, it looks like there have been reports of keyboard navigation not playing nicely with React Native, which then have gone unfixed and ended up getting auto-closed:

• Keyboard focus (from a physical keyboard) won't enter TextInput fields on Tab/Shift+Tab in Android
• Keyboard Focus Stuck in Hidden View - Unable to Navigate Using Tab/Shift+Arrow Keys
• [TextInput] Android/iOS Keyboard next/prev Buttons & tab Support

^ Very quick google search results, NOT a comprehensive list.

In parallel with better understanding what assistive tech people are using (#9469), we should spend time looking into what our options are to support keyboard in RN.

[alexandec]

Talked with Binny on this, will focus on tracking down the exact cause of the React Native keyboard navigation focus issue we are seeing. The focus gets stuck on an element and can't reach other elements on the screen. Also increasing the estimate a bit to cover more time to track this down.

[alexandec]

Reproducing the problem

I was able to reproduce the problem by checking out the a471fce commit, then running yarn install and yarn pods. The commit is prior to the workarounds we implemented. Then I ran an iPhone 15 simulator with iOS 17.5 and turned on Full Keyboard Access. I confirmed that claims in the claims list are not focusable using the arrow keys on the keyboard.

For comparison I downloaded iOS 16.4 in XCode and ran that on an iPhone 14 simulator with the a471fce commit. With Full Keyboard Access turned on, keyboard navigation worked great. No focus issues. So the problem definitely does not exist in iOS 16.4, but does exist in iOS 17.5, even with the same React Native version running on both.

Potential Cause

ARIA roles describe an element's function on the web for accessibility purposes. UIAccessibilityTraits do the same thing on iOS. React Native maps ARIA roles (accessibilityRole) to UIAccessibilityTraits. However, there are many more ARIA roles than UIAccessibilityTraits, so many of the ARIA roles get mapped to UIAccessibilityTraitNone.

Here's an excerpt from React Native's RCTViewManager to show how the mapping works:

...
@"link" : @(UIAccessibilityTraitLink),
@"list" : @(UIAccessibilityTraitNone),
@"listitem" : @(UIAccessibilityTraitNone),
...

These mappings have been in place for 15+ months so if the sole cause were a mapping change, we would have seen issues well before iOS 17.

In the above example you can see that the "link" ARIA role maps cleanly to UIAccessibilityTraitLink. However the "list" and "listitem" ARIA roles do not have equivalents, so they get mapped to UIAccessibilityTraitNone. Some other ARIA roles mapped to UIAccessibilityTraitNone include "list", "listitem", "menu", "menuitem", "radio", "radiogroup", "spinbutton", "tab", "tabbar", and "tablist".

It appears that all elements mapped to UIAccessibilityTraitNone are, as of iOS 17, unable to receive keyboard focus when "Full Keyboard Access" is enabled. I did a bunch of searching to try and see what keyboard navigation changes were implemented in iOS 17 but was unable to find anything definitive.

UIAccessibilityTraitNone vs UIAccessibilityTraitLink

Since changing the accessibilityRole from "menuitem" to "link" in our BaseListItem component fixes keyboard navigation on the Claims screen, I guessed that a similar change in RCTViewManager would have the same effect. I tried changing

@"menuitem" : @(UIAccessibilityTraitNone),

to

@"menuitem" : @(UIAccessibilityTraitLink),

in RCTViewManager, then restarted my iPhone simulator. This change allowed keyboard navigation to work properly on the Claims List screen. I was able to use the arrow keys to move up and down between the different claims in the list. However, the change introduced a VoiceOver issue which is that the elements are announced with the word "link" at the end. Not ideal. But this test does confirm that UIAccessibilityTraitNone disallows keyboard focus, while UIAccessibilityTraitLink allows it.

I also tried removing the @"menuitem" : @(UIAccessibilityTraitNone), line completely, but this change had no effect. Keyboard navigation was still broken.

Modifying assignment of accessibilityRoleTraits

I tried a few variations on the changes suggested in this React Native issue. The relevant code is:

RCT_CUSTOM_VIEW_PROPERTY(accessibilityRole, UIAccessibilityTraits, RCTView)
{
  UIAccessibilityTraits accessibilityRoleTraits =
      json ? [RCTConvert UIAccessibilityTraits:json] : UIAccessibilityTraitNone;
  if (view.reactAccessibilityElement.accessibilityRoleTraits != accessibilityRoleTraits) {
    view.accessibilityRoleTraits = accessibilityRoleTraits;
    view.reactAccessibilityElement.accessibilityRole = json ? [RCTConvert NSString:json] : nil;
    [self updateAccessibilityTraitsForRole:view withDefaultView:defaultView];
  }
}

I tried things like changing the if statement to if (true) or assigning nil to the accessibilityRoleTraits. Separately, I also commented out a few sections in an attempt to avoid modifying accessibilityTraits entirely. None of these changes fixed the problem.

Focusable

I found a react-native-macos PR with a discussion of the focusable prop. There is some useful explanation there, including:

In React Native, the View prop focusable is a cross platform prop that determines whether a View receives keyboard focus. We implemented it natively by having acceptsFirstResponder (AKA, can a View receive focus at all) only return true if both focusable is true and [NSApp isFullKeyboardAccessEnabled] is true. This behavior approximately matches the equivalent SwiftUI Modifier.

However, based on React Native's own docs, the focusable prop is only supported on Android. It therefore doesn't seem relevant for iOS.

Potential workaround

We could change the mapping so that everything which React Native currently maps to UIAccessibilityTraitNone would instead be mapped to UIAccessibilityTraitButton or UIAccessibilityTraitLink. This would require patching React Native but it should be doable.

The advantage to this approach is that no changes in the app would be required. The disadvantage is that VoiceOver would announce many things as buttons or links that are not actually buttons or links. I'm not convinced that this change is better than the workaround we've already implemented, changing the a11yRoles of various elements to buttons and links in the app itself.

[alexandec]

Update: iOS 18 was released on 9/16/24. I was able to update my iPhone to iOS 18 and test Full Keyboard Access there. I tried reversing our keyboard navigation workarounds locally, then navigated throughout the app using only the keyboard. The issues preventing navigation were gone. I think the issues were caused by an iOS 17 bug that is fixed in iOS 18.

[alexandec]

@TKDickson from my perspective this one is good to close. Feel free to do so if you agree.

[TKDickson]

Agreed; very thorough. And also very glad iOS 18 resolved this on their end. Closing.