Reflow and Mobile Testing

[kimviens]

Dear WCAG community,

I need help in deciphering if mobile testing is appropriate or not when it comes to reflow.

I have been reading threads such as:

• Reflow and diverging content behaviour desktop vs. mobile #668,
• Understanding for 1.4.10 and mobile browsers - absolve content authors ("not applicable") #1483,
• Testing 1.4.10 Reflow: No equivalence of testing at 320px or 1280px with 400% zoom? #648, and
• Can browser dev tools > device toggle be used to test SC 1.4.10 Reflow #520

Here are the four ways that reflow can be tested:

1. Browser Zoom: Using a 1280 pixels wide viewport and apply browser zoom to 400%
2. 320 Pixels Viewport: Opening up the code inspector (Ctrl + Shift + i) and docking it to the right side of the screen, then move the separator between the content and the code inspector to achieve a width of 320 pixels
3. Mobile Device Emulation: Opening up the code inspector (Ctrl + Shift + i) and using the device emulation to simulate a phone screen of 320 pixels wide
4. Real Mobile Testing: Opening the website on a real mobile phone

Now, I want to know the following:

• If one of these methods fail but others succeed, what do you report? Does your answer change depending on which method fails?
• If all of the methods fail except for one, what do you report? Does your answer change depending on which method fails?
• Should testers use all of the methods listed above? / Are they obliged to use all of the methods listed above?

Any help in settling on how this criteria is supposed to be tested would be great.

Thank you!

[patrickhlauke]

If one of these methods fail but others succeed, what do you report?

the idea with the tests is that you end up at the same endpoint...a viewport that's 320 CSS pixels wide. how would one fail while others succeed?

If all of the methods fail except for one, what do you report?

again, same question.

If you do see variance (possibly because a site is doing something like browser sniffing?), then you probably need to report this with more nuance (e.g. "it fails for users of browser X") when reporting a failure.

[kimviens]

I intend to circle back on this, I intend on testing different sites with those different tools to test. I had not yet a chance to because work has been busy. Thank you for always being so quick to respond. I feel super supported when testing against the WCAG knowing that there is a strong community dedicated in helping out confusions.
Big thank you!

[kimviens]

@alastc , @patrickhlauke I am @ you just in case you dont receive the notifications for replies because I often dont.

Real Life Example

Okay, I never got to building an exact example but thankfully we found a publically available example. Try Copilot Web which was previously Bing Chat.

Here is a screenshot of before testing reflow:

• Zoom Reflow: If you use Zoom, it fails reflow. The procedure is to set your width to 1280 pixels then Zoom to 400%. See screenshot:
  • 
• Web Dev Tools Reflow: If you use the web developer tools docked to the side and make the viewport 320 pixels of width, there is text clipping with the longer prompt suggestions. See screenshot:
  • 
• Device Emulation Reflow: If you use device emulation and you use a custom mobile phone with width of 320 pixels, it perfectly passes though everything appears small. See screenshot:
  • 
• Real Phone Reflow: If I use it on my real phone. Its a complete different experience. See screenshot:
  • 

Conclusions

• Zoom Reflow: The application is completely beyond unusable, fails WCAG 1.4.10 Reflow.
• Web Dev Tools Reflow: Minor clipping, fails WCAG 1.4.10 Reflow.
• Device Emulation Reflow: Small fonts and UI but passes WCAG 1.4.10 Reflow.
• Real Phone Reflow: Lovely experience it passes WCAG 1.4.10 Reflow though the width was not perfectly 320 pixels.

Disregarding the real phone reflow because this criteria is not about mobile, if an auditor only uses one of these three techniques to test 1.4.10 Reflow: the website has a 33% chance of passing WCAG.

[patrickhlauke]

@kimviens not sure what's happening there with the device emulator (suspect it's broken/buggy in chrome), but that's not correct. set an explicit new device with a 320 width instead

whatever actually happens on a real phone is irrelevant (likely they do actual browser sniffing etc), as this SC isn't about mobile phones per se

[patrickhlauke]

as for difference between zoom reflow and dev tools reflow, that's obviously down to the height/available height difference you have going there

[kimviens]

as for difference between zoom reflow and dev tools reflow, that's obviously down to the height/available height difference you have going there

You are right, I now notice that. However, there is no way for me to increase the height beyond my physical screen. But WCAG 1.4.10 does not have a height requirement. Its width OR height. So what if auditors fail reflow because they are using a very small height?

And what about the fact that dev tools introduces clipping while mobile device emulation does not?

[kimviens]

Oh sorry Patrick didnt see your first comment. My network is giving me issues.

[patrickhlauke]

Note that Reflow (which, let's be honest, is a very badly written SC) is being discussed further here in various issues to try and cover/explain things much better (within the confines of not just scrapping it completely or making normative changes) https://github.com/orgs/w3c/projects/56/views/1?filterQuery=reflow

[patrickhlauke]

and FWIW I'll be filing a bug with Chrome DevTools about that clearly broken "Responsive" behaviour

[kimviens]

I did my testing using Edge.

Yeah, ever since I started my WCAG journey, Reflow has been a rock in my shoe: annoying. I have rarely found content that passed that criteria unless someone really dedicated themselves to it. I am always told "320 pixels is not realistic!"

The link you shared cannot be accessed by me. I dont think I have the permissions to view it. That okay though, I can keep complaining without offering solutions, its always easier to complain~

(Kidding~ if I can help I would love to support the people that make what I call my "bible" but if not Ill patiently wait while devs are throwing me tomatoes!)

[mraccess77]

I find that many people get different results or fail this because they believe this criterion requires a specific height like 256 pixels for horizontal languages. They get this notion because of the 12 0 x 1024 400% zoom statements in the supporting materials. It would be good to get that cleared up

[patrickhlauke]

@mraccess77 I opined a few times before that it would be great to get some actual explanation in the understanding that explains exactly HOW low vision users use their device (with/without magnification software), and how that actually then shakes out for them with this success criterion. with the naive "they run their screen as 1280 by 1024 and zoom", the height limitation comes out logically. whereas if the scenario is more "they run their desktop at whatever size, then run magnification software, which turns their effective view to a small size, and they'll then resize their regular browser window a certain way to consume the content", which is a much more nuanced/granular scenario (e.g. they may end up with their browser more in a vertical/portrait aspect ratio when navigating vertical content). would love to chat some more specifically about this to try and bring that into the understanding/intent

[patrickhlauke]

Filed issue against Developer Tools and the "Responsive" view being strangely not what you'd expect here https://issues.chromium.org/issues/333831258

EDIT: doh, realised after filing that issue that in Chrome's emulation, it was because I had it set to "Mobile" rather than "Desktop". Switching it to the former makes the responsive view work as expected

@kimviens in Edge, that device type (Mobile/Desktop) option isn't visible by default. Enable it from the menu

then choose "Desktop" from the device type to avoid it forcing a minimum view width/height like mobile device browsers do

[kimviens]

Oh my!! Thanks @patrickhlauke ! This minor detail has huge consequences! With the device type, now both the device emulation and the dev tools reflows are consistent.

One problem remains: zooming.

The criteria does not say 320 pixels of width and 256 pixels of height. Its written, as I understand it, an "or". So if I am testing for this criteria, I may file a false positive simply because my screen does not have enough height. If its meant to be an "and", then Reflow should probably to be rewritten to something like this:

Content can be presented without loss of information or functionality, and without requiring scrolling in two dimensions for:

• a width equivalent to 320 CSS pixels and a height equivalent to 256 CSS pixels;

Except for parts of the content which require two-dimensional layout for usage or meaning.

[mraccess77]

@mraccess77 I opined a few times before that it would be great to get some actual explanation in the understanding that explains exactly HOW low vision users use their device (with/without magnification software), and how that actually then shakes out for them with this success criterion.

I believe that usage is all over the place

• Different resolution or viewport sizes
• use of scaling at platform level
• Use of increased font sizes on webpages
• Use of browser zoom
• Use of full screen mode in browser
• Use of screen magnification software
• Combinations of the above
• Mobile device in landscape or portrait

[patrickhlauke]

@mraccess77 as the SC specifically mandates things based on CSS pixel dimensions at which content must reflow, things like "use of increased font sizes" and "use of screen magnification software" (in isolation) won't be helped by this SC ... so somewhere along the line users must be able to "force" their user agent to those sizes somehow to trigger its reflow. that's the part where there's a disconnect, to me (and yes, having those CSS pixel dimensions defined as "at" rather than "down to" is another failing of this SC in my opinion...it just feels like what we landed on normatively actually doesn't really fundamentally address the variegated user needs)

[mraccess77]

For horizontal text - I don't believe any height was ever set - only the width of 320 - so the SC doesn't address any particular requirement for height. The 256 was only for vertical text - so I don't think we can rely on that as a requiremen.

[kimviens]

@mraccess77 I agree and I think it might be the root of issues when interpreting reflow.

[fstrr]

This issue is labelled as a discussion, so we’re moving this to Discussions. There doesn’t seem to be an update to make to the documentation, but if that changes, we can move it back to the issues list.