TalkBack support for ScrollView accessibility announcements (list and…

… grid) - Javascript Only Changes (facebook#33180)

Summary:
This is the Javascript-only changes from D34518929 (facebook@dd6325b), split out for push safety. Original summary and test plan below:

This issue fixes [30977][17] . The Pull Request was previously published by [intergalacticspacehighway][13] with [31666][19].
The solution consists of:
1. Adding Javascript logic in the [FlatList][14], SectionList, VirtualizedList components to provide accessibility information (row and column position) for each cell in the method [renderItem][20] as a fourth parameter [accessibilityCollectionItem][21]. The information is saved on the native side in the AccessibilityNodeInfo and announced by TalkBack when changing row, column, or page ([video example][12]). The prop accessibilityCollectionItem is available in the View component which wraps each FlatList cell.
2. Adding Java logic in [ReactScrollView.java][16] and HorizontalScrollView to announce pages with TalkBack when scrolling up/down. The missing AOSP logic in [ScrollView.java][10] (see also the [GridView][11] example) is responsible for announcing Page Scrolling with TalkBack.

Relevant Links:
x [Additional notes on this PR][18]
x [discussion on the additional container View around each FlatList cell][22]
x [commit adding prop getCellsInItemCount to VirtualizedList][23]

## Changelog

[Android] [Added] - Accessibility announcement for list and grid in FlatList

Pull Request resolved: facebook#33180

Test Plan:
[1]. TalkBack announces pages and cells with Horizontal Flatlist in the Paper Renderer ([link][1])
[2]. TalkBack announces pages and cells with Vertical Flatlist in the Paper Renderer ([link][2])
[3]. `FlatList numColumns={undefined}` Should not trigger Runtime Error NoSuchKey exception columnCount when enabling TalkBack. ([link][3])
[4]. TalkBack announces pages and cells with Nested Horizontal Flatlist in the rn-tester app ([link][4])

[1]: fabOnReact/react-native-notes#6 (comment)
[2]: fabOnReact/react-native-notes#6 (comment)
[3]: fabOnReact/react-native-notes#6 (comment)
[4]: fabOnReact/react-native-notes#6 (comment)
[10]:https://github.com/aosp-mirror/platform_frameworks_base/blob/1ac46f932ef88a8f96d652580d8105e361ffc842/core/java/android/widget/AdapterView.java#L1027-L1029 "GridView.java method responsible for calling setFromIndex and setToIndex"
[11]:fabOnReact/react-native-notes#6 (comment) "test case on Android GridView"
[12]:fabOnReact/react-native-notes#6 (comment) "TalkBack announces pages and cells with Horizontal Flatlist in the Paper Renderer"
[13]:https://github.com/intergalacticspacehighway "github intergalacticspacehighway"
[14]:https://github.com/fabriziobertoglio1987/react-native/blob/80acf523a4410adac8005d5c9472fb87f78e12ee/Libraries/Lists/FlatList.js#L617-L636 "FlatList accessibilityCollectionItem"
[16]:https://github.com/fabriziobertoglio1987/react-native/blob/5706bd7d3ee35dca48f85322a2bdcaec0bce2c85/ReactAndroid/src/main/java/com/facebook/react/views/scroll/ReactScrollView.java#L183-L184 "logic added to ReactScrollView.java"
[17]: facebook#30977
[18]: fabOnReact/react-native-notes#6
[19]: facebook#31666
[20]: https://reactnative.dev/docs/next/flatlist#required-renderitem "FlatList renderItem documentation"
[21]: fabOnReact@7514735 "commit that introduces fourth param accessibilityCollectionItem in callback renderItem"
[22]: facebook#33180 (comment) "discussion on the additional container View around each FlatList cell"
[23]: fabOnReact@d50fd1a "commit adding prop getCellsInItemCount to VirtualizedList"

Reviewed By: lunaleaps

Differential Revision: D37668064

Pulled By: blavalla

fbshipit-source-id: 7ba4068405fdcb9823d0daed2d8c36f0a56dbf0f

Libraries/Components/View/ViewPropTypes.js

@@ -461,6 +461,23 @@ export type ViewProps = $ReadOnly<{|
    */
   accessibilityActions?: ?$ReadOnlyArray<AccessibilityActionInfo>,
 
+  /**
+   *
+   * Node Information of a FlatList, VirtualizedList or SectionList collection item.
+   * A collection item starts at a given row and column in the collection, and spans one or more rows and columns.
+   *
+   * @platform android
+   *
+   */
+  accessibilityCollectionItem?: ?{
+    rowIndex: number,
+    rowSpan: number,
+    columnIndex: number,
+    columnSpan: number,
+    heading: boolean,
+    itemIndex: number,
+  },
+
   /**
    * Specifies the nativeID of the associated label text. When the assistive technology focuses on the component with this props, the text is read aloud.
    *

Libraries/Lists/FlatList.js

@@ -625,11 +625,18 @@ class FlatList<ItemT> extends React.PureComponent<Props<ItemT>, void> {
           return (
             <View style={StyleSheet.compose(styles.row, columnWrapperStyle)}>
               {item.map((it, kk) => {
+                const itemIndex = index * cols + kk;
+                const accessibilityCollectionItem = {
+                  ...info.accessibilityCollectionItem,
+                  columnIndex: itemIndex % cols,
+                  itemIndex: itemIndex,
+                };
                 const element = renderer({
                   // $FlowFixMe[incompatible-call]
                   item: it,
-                  index: index * cols + kk,
+                  index: itemIndex,
                   separators: info.separators,
+                  accessibilityCollectionItem,
                 });
                 return element != null ? (
                   <React.Fragment key={kk}>{element}</React.Fragment>
@@ -660,6 +667,7 @@ class FlatList<ItemT> extends React.PureComponent<Props<ItemT>, void> {
     return (
       <VirtualizedList
         {...restProps}
+        numColumns={numColumns}
         getItem={this._getItem}
         getItemCount={this._getItemCount}
         keyExtractor={this._keyExtractor}

Libraries/Lists/VirtualizedList.js

@@ -42,6 +42,7 @@ const ScrollView = require('../Components/ScrollView/ScrollView');
 const View = require('../Components/View/View');
 const Batchinator = require('../Interaction/Batchinator');
 const ReactNative = require('../Renderer/shims/ReactNative');
+const Platform = require('../Utilities/Platform');
 const flattenStyle = require('../StyleSheet/flattenStyle');
 const StyleSheet = require('../StyleSheet/StyleSheet');
 const infoLog = require('../Utilities/infoLog');
@@ -74,6 +75,11 @@ type State = {
  * Use the following helper functions for default values
  */
 
+// numColumnsOrDefault(this.props.numColumns)
+function numColumnsOrDefault(numColumns: ?number) {
+  return numColumns ?? 1;
+}
+
 // horizontalOrDefault(this.props.horizontal)
 function horizontalOrDefault(horizontal: ?boolean) {
   return horizontal ?? false;
@@ -1010,10 +1016,35 @@ class VirtualizedList extends React.PureComponent<Props, State> {
     );
   }
 
+  _getCellsInItemCount = (props: Props) => {
+    const {getCellsInItemCount, data} = props;
+    if (getCellsInItemCount) {
+      return getCellsInItemCount(data);
+    }
+    if (Array.isArray(data)) {
+      return data.length;
+    }
+    return 0;
+  };
+
   /* $FlowFixMe[missing-local-annot] The type annotation(s) required by Flow's
    * LTI update could not be added via codemod */
   _defaultRenderScrollComponent = props => {
+    const {getItemCount, data} = props;
     const onRefresh = props.onRefresh;
+    const numColumns = numColumnsOrDefault(props.numColumns);
+    const accessibilityRole = Platform.select({
+      android: numColumns > 1 ? 'grid' : 'list',
+    });
+    const rowCount = getItemCount(data);
+    const accessibilityCollection = {
+      // over-ride _getCellsInItemCount to handle Objects or other data formats
+      // see https://bit.ly/35RKX7H
+      itemCount: this._getCellsInItemCount(props),
+      rowCount,
+      columnCount: numColumns,
+      hierarchical: false,
+    };
     if (this._isNestedWithSameOrientation()) {
       // $FlowFixMe[prop-missing] - Typing ReactNativeComponent revealed errors
       return <View {...props} />;
@@ -1028,6 +1059,8 @@ class VirtualizedList extends React.PureComponent<Props, State> {
         // $FlowFixMe[prop-missing] Invalid prop usage
         <ScrollView
           {...props}
+          accessibilityRole={accessibilityRole}
+          accessibilityCollection={accessibilityCollection}
           refreshControl={
             props.refreshControl == null ? (
               <RefreshControl
@@ -1043,8 +1076,14 @@ class VirtualizedList extends React.PureComponent<Props, State> {
         />
       );
     } else {
-      // $FlowFixMe[prop-missing] Invalid prop usage
-      return <ScrollView {...props} />;
+      return (
+        // $FlowFixMe[prop-missing] Invalid prop usage
+        <ScrollView
+          {...props}
+          accessibilityRole={accessibilityRole}
+          accessibilityCollection={accessibilityCollection}
+        />
+      );
     }
   };
 
@@ -1821,10 +1860,19 @@ class CellRenderer extends React.Component<
     }
 
     if (renderItem) {
+      const accessibilityCollectionItem = {
+        itemIndex: index,
+        rowIndex: index,
+        rowSpan: 1,
+        columnIndex: 0,
+        columnSpan: 1,
+        heading: false,
+      };
       return renderItem({
         item,
         index,
         separators: this._separators,
+        accessibilityCollectionItem,
       });
     }
 

Libraries/Lists/VirtualizedListProps.js

@@ -27,10 +27,20 @@ export type Separators = {
   ...
 };
 
+export type AccessibilityCollectionItem = {
+  itemIndex: number,
+  rowIndex: number,
+  rowSpan: number,
+  columnIndex: number,
+  columnSpan: number,
+  heading: boolean,
+};
+
 export type RenderItemProps<ItemT> = {
   item: ItemT,
   index: number,
   separators: Separators,
+  accessibilityCollectionItem: AccessibilityCollectionItem,
   ...
 };
 
@@ -49,9 +59,19 @@ type RequiredProps = {|
    */
   getItem: (data: any, index: number) => ?Item,
   /**
-   * Determines how many items are in the data blob.
+   * Determines how many items (rows) are in the data blob.
    */
   getItemCount: (data: any) => number,
+  /**
+   * Determines how many cells are in the data blob
+   * see https://bit.ly/35RKX7H
+   */
+  getCellsInItemCount?: (data: any) => number,
+  /**
+   * The number of columns used in FlatList.
+   * The default of 1 is used in other components to calculate the accessibilityCollection prop.
+   */
+  numColumns?: ?number,
 |};
 type OptionalProps = {|
   renderItem?: ?RenderItemType<Item>,

Libraries/Lists/VirtualizedList_EXPERIMENTAL.js

@@ -49,6 +49,7 @@ const ScrollView = require('../Components/ScrollView/ScrollView');
 const View = require('../Components/View/View');
 const Batchinator = require('../Interaction/Batchinator');
 const ReactNative = require('../Renderer/shims/ReactNative');
+const Platform = require('../Utilities/Platform');
 const flattenStyle = require('../StyleSheet/flattenStyle');
 const StyleSheet = require('../StyleSheet/StyleSheet');
 const infoLog = require('../Utilities/infoLog');
@@ -81,6 +82,11 @@ type State = {
  * Use the following helper functions for default values
  */
 
+// numColumnsOrDefault(this.props.numColumns)
+function numColumnsOrDefault(numColumns: ?number) {
+  return numColumns ?? 1;
+}
+
 // horizontalOrDefault(this.props.horizontal)
 function horizontalOrDefault(horizontal: ?boolean) {
   return horizontal ?? false;
@@ -1205,10 +1211,35 @@ class VirtualizedList extends React.PureComponent<Props, State> {
     );
   }
 
+  _getCellsInItemCount = (props: Props) => {
+    const {getCellsInItemCount, data} = props;
+    if (getCellsInItemCount) {
+      return getCellsInItemCount(data);
+    }
+    if (Array.isArray(data)) {
+      return data.length;
+    }
+    return 0;
+  };
+
   /* $FlowFixMe[missing-local-annot] The type annotation(s) required by Flow's
    * LTI update could not be added via codemod */
   _defaultRenderScrollComponent = props => {
+    const {getItemCount, data} = props;
     const onRefresh = props.onRefresh;
+    const numColumns = numColumnsOrDefault(props.numColumns);
+    const accessibilityRole = Platform.select({
+      android: numColumns > 1 ? 'grid' : 'list',
+    });
+    const rowCount = getItemCount(data);
+    const accessibilityCollection = {
+      // over-ride _getCellsInItemCount to handle Objects or other data formats
+      // see https://bit.ly/35RKX7H
+      itemCount: this._getCellsInItemCount(props),
+      rowCount,
+      columnCount: numColumns,
+      hierarchical: false,
+    };
     if (this._isNestedWithSameOrientation()) {
       // $FlowFixMe[prop-missing] - Typing ReactNativeComponent revealed errors
       return <View {...props} />;
@@ -1223,6 +1254,8 @@ class VirtualizedList extends React.PureComponent<Props, State> {
         // $FlowFixMe[prop-missing] Invalid prop usage
         <ScrollView
           {...props}
+          accessibilityRole={accessibilityRole}
+          accessibilityCollection={accessibilityCollection}
           refreshControl={
             props.refreshControl == null ? (
               <RefreshControl
@@ -1238,8 +1271,14 @@ class VirtualizedList extends React.PureComponent<Props, State> {
         />
       );
     } else {
-      // $FlowFixMe[prop-missing] Invalid prop usage
-      return <ScrollView {...props} />;
+      return (
+        // $FlowFixMe[prop-missing] Invalid prop usage
+        <ScrollView
+          {...props}
+          accessibilityRole={accessibilityRole}
+          accessibilityCollection={accessibilityCollection}
+        />
+      );
     }
   };
 
@@ -2037,10 +2076,19 @@ class CellRenderer extends React.Component<
     }
 
     if (renderItem) {
+      const accessibilityCollectionItem = {
+        itemIndex: index,
+        rowIndex: index,
+        rowSpan: 1,
+        columnIndex: 0,
+        columnSpan: 1,
+        heading: false,
+      };
       return renderItem({
         item,
         index,
         separators: this._separators,
+        accessibilityCollectionItem,
       });
     }
 

Libraries/Lists/VirtualizedSectionList.js

@@ -9,7 +9,7 @@
  */
 
 import type {ViewToken} from './ViewabilityHelper';
-
+import type {AccessibilityCollectionItem} from './VirtualizedListProps';
 import {keyExtractor as defaultKeyExtractor} from './VirtualizeUtils';
 import invariant from 'invariant';
 import * as React from 'react';
@@ -342,7 +342,16 @@ class VirtualizedSectionList<
   _renderItem =
     (listItemCount: number) =>
     // eslint-disable-next-line react/no-unstable-nested-components
-    ({item, index}: {item: Item, index: number, ...}) => {
+    ({
+      item,
+      index,
+      accessibilityCollectionItem,
+    }: {
+      item: Item,
+      index: number,
+      accessibilityCollectionItem: AccessibilityCollectionItem,
+      ...
+    }) => {
       const info = this._subExtractor(index);
       if (!info) {
         return null;
@@ -371,6 +380,7 @@ class VirtualizedSectionList<
             LeadingSeparatorComponent={
               infoIndex === 0 ? this.props.SectionSeparatorComponent : undefined
             }
+            accessibilityCollectionItem={accessibilityCollectionItem}
             cellKey={info.key}
             index={infoIndex}
             item={item}
@@ -486,6 +496,7 @@ type ItemWithSeparatorProps = $ReadOnly<{|
   updatePropsFor: (prevCellKey: string, value: Object) => void,
   renderItem: Function,
   inverted: boolean,
+  accessibilityCollectionItem: AccessibilityCollectionItem,
 |}>;
 
 function ItemWithSeparator(props: ItemWithSeparatorProps): React.Node {
@@ -503,6 +514,7 @@ function ItemWithSeparator(props: ItemWithSeparatorProps): React.Node {
     index,
     section,
     inverted,
+    accessibilityCollectionItem,
   } = props;
 
   const [leadingSeparatorHiglighted, setLeadingSeparatorHighlighted] =
@@ -576,6 +588,7 @@ function ItemWithSeparator(props: ItemWithSeparatorProps): React.Node {
     index,
     section,
     separators,
+    accessibilityCollectionItem,
   });
   const leadingSeparator = LeadingSeparatorComponent != null && (
     <LeadingSeparatorComponent