Lessons learned

Last updated 31 August 2018

Post-research insights

A compendium of lessons learned from rounds of user research and actions taken. Other interesting iteration information can also be listed here.

Round 6 (August '18)

Sixth round of testing was conducted by our partners from CMS. The biggest takeaways from this round was a need to reconsider sections such as standards and conditions. Right now, every activity has a series of standards and conditions that need to be complete. This can be onerous for APDs with lots of activities.

Round 6 lessons:

• Reworking the key personnel checkout for a different solution, that once again removes key personnel into a different heading section incorporating contact personnel.
• Help text was separated into help text, instruction text, and reminders which used a gold background style. After some visual design iterations, a new design style around instruction text will be devised and implemented.
• Rewrote the legacy help text copy, and reworked dense sections of text throughout the prototype.

Round 5 (June '18)

During the fifth round of testing, we tested a single-page scroll versus a multi-page version of the eAPD application. We found that state users liked aspects of the multi-page scroll, but the lack of overall detail versus the (existing) single scroll was the biggest complaint during testing.

Round 5 lessons:

• Added checkboxes to select years which eAPD covered
• Removed a character limit to force brevity; this caused more confusion than intended
• Removed a need for names in vendor/contractor sections, generalized to vendor type
• Fixed cross-browser issues discovered in testing
• Added a quarterly costs table in cost allocation

Round 4 (May '18)

Immediately after the May round, a visual refresh of the application was implemented. (see screenshots)

Refined the key personnel section. At this stage, key personnel was a separate section where users input separately the 'key' members involved in all activities across the APD submission. After round 4 of testing, we found that state users were unclear about what constituted key personnel. We removed the key personnel section and integrated key personnel into each activity.

Other round 4 lessons:

• Standardized dollar sign fields across the application; previously $ was not uniform
• Improved help text to key sections after feedback indicated a need for better detail

Round 2 (March '18)

After our March (Round 2) user testing round, we received feedback that indicated a need to revisit how we communicated about the eAPD app to state stakeholders. This resulted in a shift away from a guided interaction experience (where screens of help text would aid users through the app flow) towards an experience more consistent with completing a form.

The single-flow interaction form tested across our rounds of user testing. Feedback has indicated that state users preferred seeing all of the sections of the form, given they do not complete it in a linear fashion, and usually encompassing a bevy of different internal state analyst/users responsible for individual sections.

Other lessons and decisions from the Round 2 included:

• Request for PDF export of eAPD
• Removed names and job descriptions from personnel sections
• Increase field sizes for milestone detail sections
• Begin to indicate what's required vs. what's optional in form fields