The Columns block is confusing/unusable on NVDA

[diegohaz]

Description

While working on #29804, after @carolinan's feedback, we noticed that the Columns block usage on NVDA is really confusing and pretty much unusable.

Step-by-step reproduction instructions

1. Create a new post
2. Add a Columns block with a few columns
3. Try to navigate through the columns using NVDA and arrow keys/tab

Expected behaviour

We should be able to navigate through the columns just like we do without a screen reader, using arrow keys.

Actual behaviour

The experience is not great even for keyboard-only users, but it's worse for screen reader users. Once the Columns block is focused, NVDA automatically switches from focus mode to browse mode, which means keystrokes will not be passed to JavaScript anymore. Instead, they'll execute commands on NVDA. This breaks the writing flow.

I believe we should use a different role value on the element. Instead of group, we could try grid and implement the necessary behaviors.

Screenshots or screen recording (optional)

ScreenFlow.mp4

Device information

• Device: Desktop
• Operating system: Windows 10
• Browser: Firefox 87

[carolinan]

I am experiencing the same problems with the navigation block.

[MarcoZehe]

Has this ever gone anywhere? If using the "grid" role, make sure aria-readonly="false" is also set, since grids no longer get the editable trait applied automatically, as per the newest ARIA spec. That should cause NVDA to remain in focus mode. Note that if gridis used, the children must be rows, and those children must be gridcell or similar. See the ARIA spec for the allowed children.

[alexstine]

@diegohaz Do you have time to work on this or do you want me to give it a shot?

[diegohaz]

@alexstine Please, feel free to give it a try. I made some experiments with the grid role, but couldn't find a model that wouldn't feel as confusing.

[alexstine]

@diegohaz Alright, I'll give it a shot. I'm good with accessibility but really still learning with JavaScript. We'll see how it goes.

[alexstine]

@diegohaz Do you happen to know where the block markup is handled in the editor? Looks like it uses list block for columns?

[diegohaz]

@alexstine There are two blocks: Columns and Column. You can find the code on packages/block-library/src/columns/edit.js and packages/block-library/src/column/edit.js, respectively.

[diegohaz]

Just to add more context to this issue, I recorded a video with some thoughts about browse and focus modes on NVDA. At the end of the video, I propose that we render the whole editor as a tree view (without changing how it's visually presented). I still think that's a good idea to explore, but that would require several changes and a lot of effort: Screen reader browse/focus mode video

[alexstine]

@diegohaz So, here's our essential problem. Instead of being able to render this as a grid, each column is it's own block. Another issue is when you enter navigation mode (Gutenberg navigation mode), you can access all the columns where if you try to arrow around in the (Gutenberg edit mode) it really doesn't work.

The biggest problem is the block-list in block-editor is one size fits all. The current ARIA role makes it conform best for editing text such as Paragraphs and Headings, but the situation is different with the Columns and Column blocks as they are essentially placeholders and child blocks get added under them.

This is what I see being the most helpful:

1. We get rid of Gutenberg navigation/edit mode. This has caused so much pain and we need to make the editor accessible instead of adding band-aids.
2. We make the List View accessible. This is how I think it should work.
3. Open List View.
4. Screen reader gets the tree view you were talking about.
5. For Paragraph, List, Heading, etc. we simply allow an Enter keypress to move to the block.
6. For the Columns block, user would hear Columns (collapsed).
7. Once the user expanded the Columns item with Right Arrow, next items would be the columns.
8. Let's say user wants to expand Column 2, user presses Down Arrow to get to Column 2 and then expands it.
9. User can now select the Paragraph block inside the Columns -> Column 2 block.
10. For these placeholder blocks, we need to figure out a better way for some type of focus to occur. Maybe a readonly or disabled text field? There really isn't a specification for this.
11. A better option might be to just let users take action from the List View menu. E.g. they could expand the Column 2 Options menu and press Delete Column 2 without actually having to navigate to it. This in theory should be possible to pull off.

I will make the argument again. Screen readers and block editors live in totally separate universes and this only proves my point. The editor works great for the Paragraph block as it is a free flowing text field. It makes sense that you can simply press Shift+Tab or Alt+F10 to get a toolbar. However, without a text field in these placeholder blocks, I don't know how to communicate to users that it is a placeholder block that you really can't take any action on. I mean, I could render it as a button for screen readers, but then users would wonder why pressing Enter on it does nothing. I'm at a real loss because rendering these as a grid just isn't a viable option. At least, it doesn't appear to be a very easy one in the current state of the editor.

Any of this make sense?

Thanks.

[alexstine]

@MarcoZehe Would you be up for giving this a test?

http://gutenberg.run/37304

I don't think it is perfect, but I do think it is slightly better.

Can you let me know if I'm going in the right direction?

Thanks.

[alexstine]

I will note that this seems to not be working in Google Chrome but it works fine in Firefox. When pressing Right Arrow in Firefox, it will announce the aria-label and tell me exactly which gridcell I am on. In Google Chrome, it is completely silent.

[MarcoZehe]

I will note that this seems to not be working in Google Chrome but it works fine in Firefox. When pressing Right Arrow in Firefox, it will announce the aria-label and tell me exactly which gridcell I am on. In Google Chrome, it is completely silent.

This appears to be an NVDA bug. For some reason, because it doesn't get a row number, something causes a crash while handling the focus on the cell, that's why speech is silent in Chrome. If I use JAWS, your build works with both Firefox and Chrome.

Having said that: The problem is indeed that the columns/grid cells are directly below the table, but according to the correct ARIA spec, need to be inside a row. Even if this is a table with only one row and 2 or more columns, the table -> row -> gridcell rule must be followed. Tables are a complicated beast, and both browser accessibility and screen readers can easily be thrown off-step if the hierarchy isn't being followed.

[alexstine]

@MarcoZehe Do you have any suggestions on how to fix it?

Right now, Columns block gets:

role="grid"

and Column blocks which is child of Columns block gets:

role="gridcell"

Can I do this differently with roles?

Also, have you watched @diegohaz YouTube video? I like that much better and the whole editor would become much more accessible having a tree view type navigation to it. In my opinion, that would be more logical instead of trying to take a very visual concept and shove ARIA in to it until it works.

Thoughts?

Thanks. 💯

[MarcoZehe]

@alexstine The only way to fix this would be to insert an additional div that has the "row" role. And yeah, that might totally throw off styling and only wallpaper over the bigger problem. The reason why I didn't answer to the tree view proposal was that you specifically asked me to try out your approach, and that's what I provided you feedback for. 😉

Also, have you watched @diegohaz YouTube video? I like that much better and the whole editor would become much more accessible having a tree view type navigation to it. In my opinion, that would be more logical instead of trying to take a very visual concept and shove ARIA in to it until it works.

I have now, and totally love the concept. For this just being a proof of concept right now, this already works extremely well! 🎩

When reviewing an actual patch, I'd definitely request some changes to make it more compliant with the actual Tree View design pattern, like:

• When on an expandable item, Right Arrow should expand it, but not yet move to the child item.
• When on an expanded item, Left Arrow should collapse it, but keep focus there.
• When on a collapsed or non-expandable leaf item, Left Arrow should back up to the parent node. This is so if I am inside a block grouping that has 10 blocks, and I'm located on the very last or close to last entry, I can quickly back up to the parent without having to walk the other 9 blocks above the current item on the current level first. Efficiency is key, and having to walk up so many items just to get to the parent would be inconvenient.
• Debatable: Right Arrow, for consistency and symmetry, should not move forward item by item when not expanding a collapsed item. The vertical navigation should be reserved for up and down arrows. This is how it is done in most tree views, and not overloading right and left with too many behaviors will also lower cognitive load. This is not 100% agreed in the community, but I personally find this to be the preferred implementation.

So yeah, if we can get the powers that be to allow us to implement the Tree View approach @diegohaz suggests in his video, and has a proof of concept for, I think this could go a long way towards reducing friction and different exceptions for different use cases. It would, of course, also get rid of edit vs. selection mode, since the tree view would then be our place of selection. That would definitely also reduce ambiguity for keyboard users in general, who then would not need to remember which mode they're in, regardless of whether they're screen reader users or not.

[alexstine]

@diegohaz Do you have a POC PR for the tree view you described in your video? I think it would be well worth our time working on that instead of this issue specifically. It would fix this and so many more.

Thanks @MarcoZehe for the review. I'm hoping we could just implement the tree view and focus our energy on something that will really make a difference.

Thanks all.

[diegohaz]

@alexstine I don't. The POC in the video was a simple prototype I made to demonstrate how the tree view could work. The blocks are hardcoded there.

I also think the tree view pattern is a nice solution, but there's a catch. A tree view will reflect the block hierarchy really well, but not the layout. As a keyboard user, when a group of blocks was displayed horizontally, like the columns or navigation blocks, I found really disorienting that pressing the arrow right key wouldn't move focus to the block on the right. Instead, I would have to press the arrow down key, which I expected would move the focus to the block below. 🤔

It's even worse when you consider that the arrow left key would move to the parent block instead of the block on the left. I think it's still valuable to have a tree view with editing capabilities though, maybe in the list view panel (activated by clicking on the List View button at the Document tools toolbar). But I'm not sure about the editor itself.

In theory, it seems that the grid pattern is the most appropriate for that. The whole editor would be a grid with multiple rows. Rows with multiple blocks arranged horizontally would have multiple gridcells. Whereas rows with a single block would have a single gridcell with the colspan or aria-colspan attributes. But the last time I tried it, the screen reader support for colspan was really poor, especially on VoiceOver.

Another pattern I'd like to explore at some point is listbox. The top-level blocks would be options in a top-level vertical listbox. When activated, the option would open a modal with another listbox that could be either vertical or horizontal, and so on. That is, you would use arrow keys to move through blocks in the same level, Enter or Space to expand the block (in a modal), and Escape to go back to the parent level. This would only apply to the semantic of the elements, not to how they're visually presented.

I know we've discussed treating blocks as modal-like elements in the past. I'm not sure if ARIA lets us open modals in response to clicks on role="option" elements. But, if we can, I think this is also a good pattern to explore. This is also a simpler implementation and interaction model compared to grid and tree view.

[MarcoZehe]

In theory, it seems that the grid pattern is the most appropriate for that. The whole editor would be a grid with multiple rows. Rows with multiple blocks arranged horizontally would have multiple gridcells. Whereas rows with a single block would have a single gridcell with the colspan or aria-colspan attributes. But the last time I tried it, the screen reader support for colspan was really poor, especially on VoiceOver.

Yeah that depends largely on the browser. Safari should be the best in class on OS X, since it is made by Apple, but while implementing VoiceOver support in Firefox in 2020, the team found that a lot of the stuff was poorly documented, if at all, and implementation was largely guess work, and in some cases, digging through WebKit code. Chromium also has its own implementation which also has gaps. In Firefox, the team tried to get it right as best they could, and I think it is fairly decent all things considered. But yeah, this should definitely also work in Safari, and if they don't do anything on aria-colspan, that is a bummer.

But be aware that we're venturing into heavily advanced stuff here and also pushing the boundary of what screen reader designs might currently be able to cope with.

I know we've discussed treating blocks as modal-like elements in the past. I'm not sure if ARIA lets us open modals in response to clicks on role="option" elements. But, if we can, I think this is also a good pattern to explore. This is also a simpler implementation and interaction model compared to grid and tree view.

ARIA does not change browser behavior in any way. It only communicates semantics like "what is this element" and "what state is it in" or some other properties that screen readers can query. It doesn't cause anything to happen automatically, unlike native HTML elements. So to the browser, leaving aside the mapping to the accessibility APIs, ARIA is completely oblivious. So a div with ARIA role "option" is still just a div that you can do anything you want with in JavaScript. You can have a keyboard handler that reacts on enter, which then transitions focus to another set of divs that ARIA tells the screen reader is a modal thing, and JavaScript makes sure keyboard is trapped so this actually behaves to the user like a modal. ARIA just needs to be able to communicate to the user what is going on, the rest is equally applicable to any keyboard user without a screen reader.

[diegohaz]

ARIA does not change browser behavior in any way. It only communicates semantics like "what is this element" and "what state is it in" or some other properties that screen readers can query. It doesn't cause anything to happen automatically, unlike native HTML elements. So to the browser, leaving aside the mapping to the accessibility APIs, ARIA is completely oblivious. So a div with ARIA role "option" is still just a div that you can do anything you want with in JavaScript. You can have a keyboard handler that reacts on enter, which then transitions focus to another set of divs that ARIA tells the screen reader is a modal thing, and JavaScript makes sure keyboard is trapped so this actually behaves to the user like a modal. ARIA just needs to be able to communicate to the user what is going on, the rest is equally applicable to any keyboard user without a screen reader.

Yeah! That makes a lot of sense. Thank you!

[alexstine]

@diegohaz How hard would it be if the block layout stayed the same but the List View was converted in to a tree view? Might that avenue be one worth exploring?

[getdave]

Can I check whether the [Block]: Navigation label currently applied to this Issue is accurate in the sense that the problems described herein apply to the Navigation block (and not just the Columns block)?

If so I'd like to understand the specific issues in order that I can make a specific issue for this to be addressed as part of #38786.

Many thanks in advance for your help 🙇‍♂️

[alexstine]

@getdave Actually, it would be more accurate to say any Block that doesn't contain a text field is effected or contentEditable attribute. Basically, all of the new interactive Blocks. See this PR for more info.

#37934

[alexstine]

@MarcoZehe What is your opinion of the columns block now? After a couple PRs were merged, screen readers should at least get some help on the setup stage. After that, the columns block only has inner blocks with the parent columns block being empty of any content.

1. Open the post/page editor.
2. Insert a columns block.
3. Notice how the screen reader will place focus on the block variations panel. Also, the block placeholder description should be read allowed.

I think this part is working great up until a variation is selected. At this point, things change. Since the columns block no longer contains content, screen readers hear nothing. There are no instructions telling screen readers that they need to add blocks or edit the existing blocks. At this point, those are inner blocks.

Parent element: Columns block.
Child elements: Column blocks.
Child elements: Whatever block is inserted as an inner block of a column block.

I think the biggest struggle is having no way to communicate this to users. For example, if I landed in the columns block, I would want to hear the following.

Press Escape followed by Right Arrow to explore each column.

When I land on column 2 for example, I would expect to hear this.

Press Up and Down Arrows to navigate blocks within column 2. Press Left and Right Arrows to toggle between columns.

Any of this making sense?

Not quite sure how to implement something like this but I'd be willing to give it a go. I wouldn't want to shove this all in the aria-label and I am not sure aria-describedby would be read in the context of the document role. If aria-describedby could work, I could render the description right below the block and ensure it receives a unique ID. This seems simple enough in practice, but I won't start trying to implement such unless I get confirmation this might actually help users. As a blind person, I can say the instructions would be nice considering the fact of I likely wouldn't have ever figured it out without others explaining it to me first.

Thanks.

[MarcoZehe]

I think this could work. Alternatively, if you cannot get an element to work for aria-describedby, modern browser versions also support the newer aria-description property, which, similar to aria-label, just takes a translatable string. The other alternative would be an announcement via live region, but that has all sorts of problems, like being easily muted, drowning out other important announcements (e. g. focus), etc. aria-describedby/aria-description would be the obvious choice here. Think of VoiceOver hints on Mac or iOS, which are similar. And I think this complex UI justifies using this technique for such instructional messages.

[alexstine]

@MarcoZehe I just found a problem. I think we're going to require the aria-live region for this.

http://gutenberg.run/39558

If you check my PR #39558, it all seems to work fine up until you add a paragraph block to the column in a columns block. I believe what is happening is since the aria-description attribute gets added on one parent div, then another parent div, and finally the deepest div, the description is read multiple times for each div. I think what we'll need to do is change it so it updates the aria-live region on block selection. I really did not want to take this approach but I am not seeing any other options based on how the markup looks.

I will start playing with it locally and see what results I come up with.

[mrfoxtalbot]

I run into this issue triaging issues labeled as [Block] Navigation but I do not think it is related so I have removed the label.

[alexstine]

@mrfoxtalbot The label was added because that block was also effected. Actually, this label belongs for most blocks. 😞

Thanks.

[mrfoxtalbot]

I see! Thank you and apologies @alexstine!