Skip to content

Latest commit

 

History

History
363 lines (180 loc) · 14.1 KB

research-plan.md

File metadata and controls

363 lines (180 loc) · 14.1 KB

Research Plan for Debt Resolution Team, Streamlined Waiver (eFSR) Accessibility Study, October 2023

Background

In this exploratory study, we seek to understand how Veterans that use screen readers navigate and fill out the enhanced Financial Status Report form (eFSR). We hope the results from the study will help inform accessibility best practices for our team going forward.

We believe it is important to help Veterans using screen readers maximize efficiency, build proficiency, enable auditory scannability, and allow a level of flexibility as they progress through the form.

Participants will log in to a test account on staging. Staging link: https://staging.va.gov/manage-va-debt/request-debt-help-form-5655/introduction

OCTO Priorities

This research supports OCTO priorities 1, 2, 3, 4, and 5. This research especially supports number 4 for an experience that is usable by Veterans who use screen readers.

  1. Veterans and their families can apply for all benefits online

  2. Logged-in users can update their personal information easily and instantly

  3. Veterans and their families can find a single, authoritative source of information

  4. Logged-in users have a personalized experience, with relevant and time-saving features

  5. Veterans and their families trust the security accuracy and relevancy of VA.gov

Veteran Journey

Where does your product fit into the Veteran journey? Are there moments that matter?

Debt can happen at any stage in the Veteran’s journey after the ‘Getting out’ stage. However, certain types of debt, such as Post-9/11 GI Bill debt, is more likely to occur at the ‘Finding something to do’ stage.

Other key moments this research touches in the Veteran journey are:

  • Attending to health needs

  • Taking care of myself

  • Managing my declining health

This research also touches on aging and managing new needs for those that develop vision issues or blindness later in life.

Research Goals

What are you trying to learn from this research?

  1. Understand accessibility of the non visual user experience of the eFSR (Streamlined Waiver) forms and what improvements can be made (efficiency, auditory scannability, proficiency, flexibility)

  2. Understand how this form compares in contrast to mental models Veterans that use screen readers have with other forms

  3. Understand Veterans level of satisfaction with the eFSR (Streamlined Waiver) form

Outcome

  • We believe the results from the study will help inform accessibility best practices for our team going forward.

Research questions

GOAL 1: Understand accessibility of the non visual user experience of the eFSR (Streamlined Waiver) forms and what improvements can be made (efficiency, auditory scannability, proficiency, flexibility)

Navigation & Overall Experience

  • How do Veterans with visual limitations who use screen readers navigate through the form?

  • How easy or difficult is it to fill out the form with a screen reader?

  • Which area of the form would be most helpful for Veterans using screen readers to hear first when they tab?

  • Should it be: The header (step x of 7) or should it be the question they are being asked?

  • How do they experience error states?

  • Do participants know how to expand additional information sections?

  • What is difficult to accomplish in this form, if anything?

  • How is the experience of signing the form?

GOAL 2: Understand how this form compares in contrast to mental models Veterans that use screen readers have with other forms

Comprehension

  • How is comprehension of the list loops from editing a section to the summary?

  • How clear is the transition from one section to the next of the form?

GOAL 3: Understand Veterans level of satisfaction with the eFSR (Streamlined Waiver) form

Satisfaction

  • How do Veterans that use screen readers feel about filling out the SW form? Does it meet their expectations? How can we improve their experience?

Hypothesis

  1. We hypothesize that the eFSR/Streamlined Waiver form will be usable by Veterans that use screen readers. They will be able to understand it and fill it out.

  2. While we think it will be usable, we also believe that accessibility issues identified in an internal accessibility audit, may take away from the experience.

  3. We further hypothesize that the accessibility of the form will line up with the mental models that Veterans who use screen readers have about online forms.

Methodology

  • Accessibility testing with a usability study approach

Location

  • Location: Remote, Zoom

  • Point of contact: Charlotte Cesana, Megan Gayle

  • Equipment used for research: Zoom, screen readers

Research materials

Note: your OCTO/VA lead must review and approve all research materials – including this plan – prior to submitting a recruitment request.

Provide a link to any materials you need to run your study, including any materials needed for set up and recruitment.

For moderated usability tests:

Recruitment

Link to recruitment request

Recruitment approach

Inclusive recruitment prioritizing a full sample of Veterans that use screen readers due to low vision or blindness.

**We would like to request a Kickoff Call with Perigean. **

Recruitment criteria

List the total number and type (Veterans, caregivers, etc.) of participants for this study.

6 completed sessions with Veterans that use screen readers, please recruit 15 Veterans

Primary criteria (must-haves) What demographics, experience, and scenarios do you need participants to meet to effectively run your study?

  • All participants must be Veterans
  • Inclusive, diverse sample with respect to race/ethnicity, age, gender, educational attainment, financial status
  • All must be on a computer or mobile device that utilizes screenreader technology
  • During the session, be willing to share a web browser window on their device
  • Have a working microphone on their device
  • Have Zoom downloaded to their device prior to the session and know how to share their screen

Please collect basic demographic information on participants in addition to the screener questions

Screener Questions:

  1. We’re asking these questions because we want to be sure our website works well for visually impaired people and assistive technologies.
    Please select the statement that best applies to you:
    a) I am visually impaired but not blind.
    b) I am blind.
    c) I have normal vision or near normal vision. (disqualify)

  2. How long have you been blind or visually impaired?
    a) In the last 2 years
    b) In the last 2 - 5 years
    c) In the last 5-10 years
    d) 10 + years
    e) My entire life

  3. Do you need to use assistive technology to use technology and the internet, such as VoiceOver on an iPhone or a Mac, TalkBack on an Android device, or JAWS on a Windows computer?
    a) Yes
    b) No (disqualify)

  4. Please check the statement(s) that best applies to you:
    a) I use assistive technology (VoiceOver, Talkback, JAWS, NVDA are examples)
    b) I use text-to speech software (Speechify or WordTalk are examples) (disqualify if they only select B)
    c) I do not use assistive technology (disqualify)
    d) I do not use text-to speech software

  5. Are you able to join the Zoom session using this assistive technology?
    a) Yes
    b) Yes, with some help
    c) No (disqualify)

  6. Please list the name of the assistive technology or software you will use during the session. (Fill in)

  7. Please check the statement) that best applies to you:
    a) I am very comfortable using screen readers
    b) I am comfortable using screen readers
    c) I am still learning how to use screen readers

  8. Please select the statement(s) that apply to you: (Check all that apply)
    a) I am currently experiencing financial hardship.
    b) I have experienced financial hardship in the past.
    c) I am not currently experiencing financial hardship.

  9. Please select the statement(s) that apply to you: (Check all that apply)
    a) I have VA medical copay currently or in the past
    b) I have VA overpayment debt currently or in the past
    c) I have never had VA medical copay or overpayment debt

Follow Up Call:

Please follow up by phone with prospective participants with these (4) questions and please let us know what they say.

  1. We want to confirm that you use a screen reader due to low vision or blindness
  2. What is the name of the screenreader technology you are using?
  3. Are you comfortable joining a Zoom call using your screen reader technology?
  4. We’ll have an accessibility specialist on the call. Are there any other accommodations that would be helpful during this session?

Secondary criteria (nice-to-haves) What criteria would strengthen your results? Note: The following demographic criteria is our goal, but we understand that historically it's been challenging to recruit participants who use screenreaders.

At least 40% of people must identify as Black, Hispanic/Latina/Latinx, Asian, or Native

At least 40% of people must identify as having a cognitive consideration

At least 50% of people must identify as other than male

At least 20% of people are without a degree or professional certificate

At least 20% of people are under 35

At least 40% of people are over 55

At least 20% of people must live in a rural area

At least 20% of people are non-native English speakers

Timeline

Please submit artifacts for Research Review 8-9 days prior to the first planned research day for remote studies so Perigean can begin recruiting one week prior. Perigean requires 2+ weeks for in-person.

Tuesday Oct 24, 2023 - Friday, November 3, 2023

Prepare

A pilot session is required. Please indicate the date and name of a mock participant for a pilot session.

  • Pilot participant email: Angela Fowler

  • Date and time of pilot session: Tuesday, Oct 6, 2023

Research sessions

  • Planned dates of research: Tuesday Oct 24, 2023 - Friday, November 3, 2023

Length of sessions

  • Session length: (e.g. 30 minutes, < 1 hour, up to 2 hours, up to 4 hours):

  • 90 minutes

  • Buffer time between sessions: (30 minutes recommended to reset between sessions, debrief with team, if a participant arrives late, or a session goes slightly over time)

  • 60 minutes buffer

  • Maximum Sessions per day: (We all have limits - how many sessions can you and your team conduct in one day considering the session length, the mental strain of conducting sessions, other work you still need to complete in a day, etc?)

  • 2 per day

Availability

When would you like sessions scheduled? Please list exact dates and times in EASTERN Standard Time. Note: we recommend providing availability outside of work hours, as many Veterans are only available before and after working times, and live across the U.S.

Please request enough dates and at least double the amount of time slots for the number of requested participants. (e.g. Monday 9-1, 3-6; Tuesday 9-6, etc.; 12 time slots for 6 participants). This helps Perigean book participants when there are more time slots available, and when sessions need to be rescheduled or filled in with further recruitment.

Tuesday Oct 24, 2023 - Friday, November 3, 2023

9am PT - 2pm PT: Tuesday, Wednesday

10am PT - 2pm PT: Monday, Thursday, Friday

2 max per day

Team Roles

Please list the people who will be serving in each role. Include the primary phone number for moderator and the emails for moderator, notetaker, accessibility specialist, and observers. If you need Perigean to take notes for you, indicate that next to Notetaker

Please note: we would like Perigean to take notes on these sessions.