Support Manual User Testing

[bbertucc]

What problem is the feature trying to solve?

Equalify has no way of inputting user-test data.

Desired Solution

I think this would involve building a chrome extension for my web accessibility platform, Equalify. Equalify creates reports of issues and accessibility pros can use it for free to visualize trends in issues they find.

Everything is always open for discussion, but from my experience with Equalify, the Chrome extension would probably need to do two things:

1. Add an “Add to Equalify” left click menu option when using inspector that will send the selected code, website url, tag(s), “status”, and message to Equalify.
2. Add a quick key for screen reader users to do the same. Screen reader users won’t be using inspector. The code of the active element would be sent to Equalify, in addition to the other info (tags, message, website url, status).

Alternatives

There are no alternatives.

Technical Details

• We have an existing list of messages, tags, and statuses. Users will often select from existing info via an autocomplete, but they can also create their own messages or tags (status options are fixed).
• We have API endpoints in Eqalify that you can use to post material. A user would have to log in to the extension to use it (authentication is handled by oAUth).
• We’ll also need to send the page’s source code to Equalify, so we can eventually train ML to find patterns and build rules to perform better automated tests. …I’m still trying to figure out how we’re storing this data and handling it when the source code changes..
• Users who rely on assistive devices should be incorporated into every aspect of the development process (planning + coding + launch)

[bbertucc]

@hcwiley and I have been talking about this issue for several weeks. It's a big lift. I do hope a bit of it can be done during #278, but it will no doubt require work outside the April time.

Let me also just underline that, if we do this, users who rely on on assistive technology should be incorporated in every aspect of the development process, starting from Day 1.

[bbertucc]

Approving @hcwiley to tackle this.

This will be part of the April work #278, but probably requires work outside April as well @hcwiley's work overview is available here: #278 (comment)

[bbertucc]

@hcwiley - feel free to create a branch for any specific updates to Equalify. I imagine you will also need to create a new repo for chrome extension development. We will probably move this issue to that repo when the build out starts.

[anphira]

@bbertucc from the agency standpoint, the biggest issues with existing "user issue" reporting tools is duplicates. That's both duplicates across pages (ie: users flag the same issue in the header across multiple pages) and duplicates across users (ie: Bob, Sue, and Alex all flag the same issue).

One of the biggest things lacking in most tools is to see already flagged issues. Ideally, if I'm on the about page and flag a header issue, then I would see that issue flagged when I visit the contact page. Many times on larger sites, you are testing over many days, and can't remember what you previously flagged, especially when getting into things like sidebars or template issues (ie: some product pages have this template area and some don't).

On a lot of sites when there are multiple users testing or multiple days of testing, more than 50% of issues are duplicates.

As a sighted user who likes code, it would be lovely for me to have both an option to see this as icons on the page (ie: visually see an icon in the header that an issue is there). But also to see a list of items (Ie: there are 8 items). And a "code option". Ie: without code option I'd see "header nav has an issue with color contrast". With code option I'd see "

has an issue with color contrast".

For screen reader users, if the area was already flagged, having the icon read out kinda like a button just before they begin the element might be nice. Ie: just before they start the

element have a button which tells them there is an error. Click button and get message. So they can know what has already been flagged for the element.

ok, that's my 2 cents.

[bbertucc]

Thanks @anphira! That's super helpful. Glad to have your perspective in this ticket so we can consider it as we build out the solution.

[kevinandrews1]

Ya, @anphira as a blind sr user+technical a11y professional the big thing that stands out to me here is knowing which element is getting flagged programmatically and providing that info via multiple modalities as you kinda alluded to e.g. code, visual, inserting it at the appropriate place in the DOM, etc.

[bbertucc]

Moved this to an issue related to a repo of it's own: #328

[bbertucc]

<rant>

I'm re-opening this because it's very very very important that we support manual testing. Like super important. I'm kind of pissed that this wasn't done and implemented in our Version 1, but I take that as my own failure. If I had my wish, we would weight manual issues that are reported through Equalify like 10 times more than every automated test. Manual testing isn't only necessary to find specific issues that prevent blind people from accessing content, it also creates the ability for us to hire accessibility pros who focus on manual testing. We need them to work with Equalify, so we can improve our own tests and user experience, making their lives easier and allowing them to focus on evermore important details.

Maybe @alexstine can jump on this work? @alexstine let me know if you have any questions after you read through this issue. Whenever you understand the work, I would love you to create a budget that we can then talk about.

</rant>

[alexstine]

@bbertucc I have no experience coding Chrome extensions but I'm happy to advise if someone picks up the work.

2. Add a quick key for screen reader users to do the same. Screen reader users won’t be using inspector. The code of the active element would be sent to Equalify, in addition to the other info (tags, message, website url, status).

How do you define inspector? If it is the browser inspector, blind people are perfectly capable of using it. Harder yes, impossible no.

[bbertucc]

@alexstine - not a lot of accessibility testers use inspector, so we would want to make sure this feature can be used by others.

Also, we might just make it more general for version 1, allowing people to simply write in issues when they come to a page. I believe @hcwiley did this already? I don't really understand the state of the current project.

[alexstine]

@bbertucc Is there any starting point I can look at?

[bbertucc]

Yeah, @alexstine - Everything here: https://github.com/equalifyEverything/equalify-raga

[bbertucc]

@alexstine might take a look in the comings weeks. Adding it to our next sprint cycle, so we follow up with this issue.

[bbertucc]

@wilsuriel03 is also interested in this issue.

[wilsuriel03]

Hey @bbertucc,

I'm interested in working on this bounty!

I noticed there's an existing repo (equalify-raga) that was started a while back but hasn't been finished or supported. What's the current status of that project? Should I build upon it or would you prefer starting from scratch?

Also, since we're currently refactoring for Equalify v2 and there will be changes to the API, which version should the extension target? Should we proceed with the current API or align with the upcoming changes?

Do you have any specific deadlines or timelines in mind?

Let me know your thoughts so I can provide an accurate estimate.

Thanks!