While using accessibility(TalkBack) on popups on Android, Focus also goes to background view.[Bug]

[HirakCTS]

🐛 Bug Report

I am using popups in Android. While i turned on accessibility, focus of TalkBack also goes to dimmed background view. It supposed to stay on the popup only.
Actual Behavior - Focus goes to background view as well. Breaking functionality of accessibility.

Focus supposed to stay on popup view only

Reproduction steps

Configuration

Version: 1.x

Platform:

• 🤖 Android
• 🐒 Xamarin.Forms

[LuckyDucko]

hey @HirakCTS could you provide a minimal project, and steps to reproduce the Accessibility talk back for Android.

if you can give me this, I can go about debugging your issue.

Also, are you able to check if the issue remains after you have updated to latest, as well as latest xamarin forms

[HirakCTS]

@LuckyDucko I am using prism for popups. I have also raised issue with them and xamarin as well, but unable to find solution. Also, I am using latest version of xamarin and all the related libraries.

Please refer this

[LuckyDucko]

I appreciate you going through the effort with this.

In the sample app, it seems that focus isn't moving onto the popup page, but instead remaining on the page behind.

However, if you are able to tap on a UI element within the popup page that has no interact-able UI elements underneath, then the talkback correctly focuses on the popup view only.

i'll investigate more when I can, however, it seems that the error is due to the talkback getting confused on what page 'Context' it should be reading from.

[johnshardman]

I am seeing this same issue, using Rg.Plugins.Popup 1.2.0.223 and Xamarin.Forms 4.8.0.1451.
Problem occurs on Android with TalkBack, and on iOS with VoiceOver. I haven't seen the same problem on UWP.

[WilliamWatterson86]

Hi @LuckyDucko - also seeing this issue. Is there any workaround?

[LuckyDucko]

@WilliamWatterson86 I haven't looked at this in a while, and I'm unsure how to go about it fixing it to be honest.

To be clear, do you get it with the latest version?

[WilliamWatterson86]

Thanks @LuckyDucko. Yes I updated today and it still happens. I only see it on android, iOS seems ok.

I made a bit of a hack today and check if voice over is on and if it is I set the page that would be in the background to IsVisible false when navigating to the popup page, then back to true when I navigate back. So this hides the background page and so the elements aren’t selectable anymore. Looks a bit odd when the page in the background disappears but I think it’s fine as a temp workaround.

[LuckyDucko]

@WilliamWatterson86

Were you having issues with iOS in anything below say, version 2.0.0.5?
Recently we have been doing some fixes around that for iOS, and if it seems to have 'fixed' itself on iOS now, then perhaps we can look into doing some sort of similar touch ups on android

[WilliamWatterson86]

@LuckyDucko - I have just tried it with 2.0.0.4, and it seems to work fine with that version too. So unless the fix was before that?

[LuckyDucko]

@WilliamWatterson86 it was just a shot in the dark, it could be one before that aswell, or it could be fixed through xamarin/new version of iOS.

Still have android to work out though

[McFlemchSoda]

Hi, I'm experiencing the same issue and I don't think Xamarin Forms will fix this since it's specific to popups. We're using your plugin because Xamarin doesn't support this feature. So I do think it has to be fixed here. iOS works fine, Android does not.

Since a lot of apps in Europe will have to comply to accessibility features by law, we are looking for workarounds or other plugins. Would be great if it could be fixed.

Edit: for now I'm trying the hacky workaround by @WilliamWatterson86

[LuckyDucko]

@McFlemchSoda

I'm sorry I haven't dedicated time to this, keeping up with projects can be time consuming.

if you, or anyone else, could provide a quick zip project with the following that would be helpful.

1. an example of the talkback not working on android.
2. the same sample working on iOS.
3. a short video showing it fail. Something tiny, you can upload direct to this issue

Then I can swap out the nuget for the rg project, and begin debugging. I may be able to work out a fix. if you can figure out which version of iOS it started 'working' with, that would be helpful in figuring out if android can also have the same fix

[uday-lucky]

Hi @LuckyDucko , i am facing this issue too. I am attaching the sample zip file here. you can reproduce the issue with it. any solution to fix this, please update ASAP. Thank you :)
XFAccesibilitySample.zip

[uday-lucky]

@LuckyDucko here is the sample video !!

20210302_194347.mp4

[LuckyDucko]

@HirakCTS
@McFlemchSoda
@uday-lucky
@WilliamWatterson86
@johnshardman
So I have been giving this one some thought.

I believe that the current behaviour is intentional as it is visible even using displayalert

However, @WilliamWatterson86 stumbled on a nifty trick that would seem to fix this problem, except that you have to make the background page IsVisible = false then swap it to IsVisible = True when it comes back.

Using this trick, I was able to create a little snippet and attach it onto the AddAsync and RemoveAsync functions, so that if Android detects you are using accessibility functionality, it will do this automatically for you.

AccessibilityManager am = (AccessibilityManager)Android.App.Application.Context.GetSystemService(Android.Content.Context.AccessibilityService);
page.Parent = XApplication.Current.MainPage;
if (am.IsEnabled)
{
    XApplication.Current.MainPage.IsVisible = true;
}

Another way I was looking at it, which may be better, is to go about and switch off the accessibility of the children below the popup using

public Task AddAsync(PopupPage page)
{
    var decoreView = DecoreView;

    RecursivelyChangeAccessibilityOfViewChildren(XApplication.Current.MainPage.GetOrCreateRenderer().View, ImportantForAccessibility.No);

    page.Parent = XApplication.Current.MainPage;

    var renderer = page.GetOrCreateRenderer();
    decoreView?.AddView(renderer.View);

    return PostAsync(renderer.View);
}

private void RecursivelyChangeAccessibilityOfViewChildren(Android.Views.View view, ImportantForAccessibility important)
{
    if (view is ViewGroup vGroup)
    {
        for (int i = 0; i < vGroup.ChildCount; i++)
        {
            Android.Views.View vChild = vGroup.GetChildAt(i);
            vChild.ImportantForAccessibility = important;
            vChild.ClearFocus();
            RecursivelyChangeAccessibilityOfViewChildren(vChild, important);
        }
    }
}

The upside to this one, Background stays as is. Downside being there is a bunch more going on each time you add a page & if you made some accessibility customisations to the original page, they will be wiped out and reset to auto when the popup is removed.

They both won't work if you layer popups on top of one another, but that's an issue for later.

Regardless of the solution we go for, I think PopupPage will need a new property to allow developers to turn these features on, with the default being off. I haven't yet cracked on a good name for that yet. Open to suggestions.

If people think this solution is good enough (and give me a good name for the feature switch), I'll create a PR for it. 👍

[LuckyDucko]

The latest version, 2.0.0.11, has a special fix added in that should make talkback far more functional. see #657 for information

[uday-lucky]

@LuckyDucko i tried with the upgrade, now the focus is going to the entire layout of the backgroundscreen and talkback continuously reading all the child elements.

[LuckyDucko]

Really? can you share with me a sample project? It should read everything that's on the popup page, and ignore everything in the background. However, I am unsure how well it works if its popup on popup

[uday-lucky]

@LuckyDucko u can use the same sample https://github.com/rotorgames/Rg.Plugins.Popup/files/6069169/XFAccesibilitySample.zip . I just upgraded to latest version and tried. The background screen is getting focussed and It is reading all the buttons (all controls on the screen) text continuously :( .

[LuckyDucko]

did you add on the special flag for it?
AndroidTalkbackAccessibilityWorkaround

I haven't got any Android Devices floating around to double check unfortunately. However, when I do, I will update the sample to include the use as a demo

[uday-lucky]

@LuckyDucko yes i set it to true. The focus on to the individual elements(i.e buttons in this sample) in the background is not happening (as expected),but the entire page is getting focused and it starts reading all the controls(I.e buttons ). I wish you would fix it ASAP when u got an android device with you.

[LuckyDucko]

@uday-lucky I am confused, wouldn't you want it able to read the controls?

[uday-lucky]

@LuckyDucko you can see this , how it is behaving after upgrading ..

20210403_125835.mp4

[LuckyDucko]

@uday-lucky I see, thank you for quickly pulling this up for me, I got so carried way with talkback no longer focusing on the background I didn't wait to see it's completion.

I'll get right on it

[LuckyDucko]

@uday-lucky
XFAccesibilitySample.zip

this is an updated version, I think I got it mostly fixed, and made it much simpler, however, I dont know how to stop it announcing the previous pages title. See if this works, and ill make a new PR

[nguyentn1197]

@uday-lucky
XFAccesibilitySample.zip

this is an updated version, I think I got it mostly fixed, and made it much simpler, however, I dont know how to stop it announcing the previous pages title. See if this works, and ill make a new PR

Hi, can I ask what is the workaround for blocking talkback from speaking the child of the parent view? My current project really need this right now

[LuckyDucko]

@deadmanproqn

Hoping that my newer code works, you have several options
A) Waiting for the upgrade to be merged in and a new nuget released. This may take 2 weeks or so
B) while waiting, download the PR, and make it a reference into your current project so you can take advantage of the upgraded code.

option B is what you will see in the sample I uploaded that you quoted. I also added a video showing how its down (on vs4mac, but im sure its a similar process on normal VS) just make sure you have the correct branch (AccessibilityUpgrade585)

AddingAsReference.mp4

[nguyentn1197]

Thank you for your quick reply,

I will use option B for testing first and use the new release when it comes out

[LuckyDucko]

@deadmanproqn tell me how it goes, hope it works out well!

[suggyd]

  if (view is ViewGroup vGroup)
    {
        for (int i = 0; i < vGroup.ChildCount; i++)
        {
            Android.Views.View vChild = vGroup.GetChildAt(i);
            vChild.ImportantForAccessibility = important;
            vChild.ClearFocus();
            RecursivelyChangeAccessibilityOfViewChildren(vChild, important);
        }
    }

To recursively hide from TalkBack you can use View.ImportantForAccessibility = Android.Views.ImportantForAccessibility.NoHideDescendants;. This should affect the view and all it's children recursively. I'm doing this on the main layout of the page in Xamarin.Forms to stop any controls on the page behind the popup from getting the TalkBack focus. Generally it looks like you want to set it back to "Auto" afterwards. I found you also need to grab the Toolbar view and do the same on it otherwise any toolbar buttons will still take the TalkBack focus.

[LuckyDucko]

@suggyd we think alike, see #663

[suggyd]

@LuckyDucko , sorry missed that. Looks good.

[McFlemchSoda]

@LuckyDucko Thanks for your effort. I've implemented the AndroidTalkbackAccessibilityWorkaround on my popups and it works with most of them. I have a GenericPopupPage that I use throughout the app, but it doesn't work on all pages. So I was wondering if it matters which page it goes on top of. For example my root is of type TabbedPage, and it seems that it doesn't work specifically on that page. If I navigate to other pages where I have the same popup page, this fix works very well!
Any ideas about what I could do or is there an optimisation possible for this fix for maybe TabbedPages or similar?

[LuckyDucko]

@McFlemchSoda This is great news! I was kinda worried about another (big) issue coming up with these like my last attempt.

Could you provide a mini example and I'll try and see what's going on. Is it just tabbedpage you have noticed it with? I wonder if its because of the tabbedpage being a many-in-one page that perhaps is confusing my code? im sure there will be something I can do on my end that fixes this up!

[McFlemchSoda]

@LuckyDucko Hi! I tried making a test project but could not reproduce it. I went back to my own project and somehow it does work now. I'm not sure what changed, but I'll keep testing to see if it was all me or if I can find a combination where it doesn't work. Xamarin.Forms isn't always reliable either haha. Sorry for wasting your time, appreciate your help!

[LuckyDucko]

@McFlemchSoda that's all good, if it does show itself again let me know!

[BurkusCat]

@LuckyDucko I am still getting the issue that @uday-lucky was experiencing. I have updated their reproduction project to have the latest Rg.Plugins.Popup NuGet package.

However, in MainPage.xaml I have removed the nav bar. This causes the background content reading out issue to reappear:
NavigationPage.HasNavigationBar="False". Maybe this is related to "however, I dont know how to stop it announcing the previous pages title." where no matter what, it is just reading out the first thing?

XFAccesibilitySampleNavBar.zip

Thanks,
Ronan

[LuckyDucko]

@BurkusCat this is interesting, I'll look into this. As in #681 I noticed it would read anything visible with words on a page.

[alexandradec]

@LuckyDucko I found a strange behavior when using the workaround property. If at any point the workaround code gets triggered, the focus in different pages gets very confused. Repro steps:
I have a page (let's call it Page1) that is not a popup page, and has some Entry controls.
I have another page (Page2) that is a popup and uses the workaround.

Page1 and Page2 are not adjacent in the Navigation stack. What I mean is that you don't open the one from the other, they are completely unrelated.

If I never open Page2, everything is fine.

However, after Page2 opens, which means the workaround code gets called, the focus on Page1 gets confused and exists simultaneously at all the Entries.

This happens even if TalkBack is not active

I have tried setting the workaround to True only if TalkBack is enabled. However, if TalkBack gets disabled at any point, the issue reproduces.