Report more object properties on mouse enter

[LeonarddeR]

Link to issue number:

Fixes #15420

Summary of the issue:

When the mouse option "Report role when mouse enters object" is enabled, NVDA reports the role of objects when the mouse enters them. However, the role is in pretty insufficient in many situations, for example:

• Check boxes: checked state is missing
• Table cells: coordinates are missing

Description of user facing changes

• Renamed "Report role when mouse enters object" to "Report object when mouse enters it"
• When enabled, additional properties (such as states, cell coordinates, table info) will be reported

Description of development approach

Implemented some additional logic to speak objects with reason controlTypes.OutputReason.MOUSE, and made event_mouseMove use speech.speakObject.

Testing strategy:

• Tested in Excel that cell coordinates are read correctly.

Known issues with pull request:

I didn't touch the config spec as I think it is a bit useless to do so, just for renaming a setting. Feel free to shout out if you think otherwise.

Code Review Checklist:

• Documentation:
  • Change log entry
  • User Documentation
  • Developer / Technical Documentation
  • Context sensitive help for GUI changes
• Testing:
  • Unit tests
  • System (end to end) tests
  • Manual testing
• UX of all users considered:
  • Speech
  • Braille
  • Low Vision
  • Different web browsers
  • Localization in other languages / culture than English
• API is compatible with existing add-ons.
• Security precautions taken.

[CyrilleB79]

I am a bit concerned by the fact that there seem to be few mouse users active on this GitHub repo.

Could we Cc some known people?
Maybe @k-kolev1985?

I also wonder if a person using the mouse may need so much information than someone who cannot see anything and uses only the keyboard. I understand the need to know the cell's coordinates; but I imagine that a link can be seen more easily by some people.

Note: I am not advocating against this PR. I just would like to double check if it's relevant and for which use case.

[XLTechie]

CC @britechguy

[cary-rowen]

cc @hwf1324

[britechguy]

Does this fix, by any chance, possibly resolve what I was asking about in issue #15397 ?

[Qchristensen]

Reads well, good work

[k-kolev1985]

Actually, I've asked about the mouse tracking being able to report object states many years ago.
For me that would be useful.

[hwf1324]

I also wonder if a person using the mouse may need so much information than someone who cannot see anything and uses only the keyboard. I understand the need to know the cell's coordinates; but I imagine that a link can be seen more easily by some people.

My thoughts on this. Reading out attributes such as clickable, links, etc. while in browse mode is sometimes a redundancy for me. So I would switch it off in the document format. This is the same for mouse navigation.

But after my testing, it doesn't follow the document formatting settings.

Not sure this can be done without making the behavior of mouse navigation and browsing modes consistent.

Of course normally I don't check this checkbox.

[hwf1324]

Does this fix, by any chance, possibly resolve what I was asking about in issue #15397 ?

Unfortunately, after my testing, it didn't improve #15397.
As I said in that question, I'm afraid this is a result of the accessibility interface used by this browser kernel.

[AppVeyorBot]

• PASS: Translation comments check.
• PASS: Unit tests.
• PASS: Lint check.
• FAIL: System tests (tags: installer NVDA). See test results for more information.
• Build (for testing PR): https://ci.appveyor.com/api/buildjobs/y8phy3nkexnrbjwp/artifacts/output/nvda_snapshot_pr15518-29444,0d1925b7.exe
• CI timing (mins):
  INIT 0.0,
  INSTALL_START 0.9,
  INSTALL_END 0.9,
  BUILD_START 0.0,
  BUILD_END 24.6,
  TESTSETUP_START 0.0,
  TESTSETUP_END 0.3,
  TEST_START 0.0,
  TEST_END 5.9,
  FINISH_END 0.2

See test results for failed build of commit 0d1925b773

[seanbudd]

Thanks @LeonarddeR

[Adriani90]

Not sure if this started after this PR, but NVDA reports Text every time when entering a plain text with the mouse. Can this be turned off somehow? Role Text is actually not really useful for the user to hear every time in the browser.

[LeonarddeR]

Pretty sure this was already the case before. You're best to create a new issue for this.

[hwf1324]

Not sure if this started after this PR, but NVDA reports Text every time when entering a plain text with the mouse. Can this be turned off somehow? Role Text is actually not really useful for the user to hear every time in the browser.

See #15397

This seems to be caused by the accessibility interface used by Chromium?Firefox does not have this problem.

[Adriani90]

@LeonarddeR is there anything that argues against having this enabled by default?

[CyrilleB79]

@Adriani90, do you mean having "Report object when mouse enters it" checked by default?

I imagine that some partially sighted people using the mouse can see objects role and property (e.g. checked) and just want the text to be read. I cannot say how many want just the text and how many also want the role and the properties. But it's not obvious to me that the default value of this setting should be changed.