Self-voicing: Verbose and brief options for Interactive Mode

[terracoda]

@jessegreenberg, I think I am done a first draft of the verbose and brief options for Interactive mode. Let me know if you have any questions.

Here's a link to the design doc:
https://docs.google.com/document/d/1XFTD2HTcGCEOP2yQ9_2SkEeXmCceZYUCmNeyiJfcCHs/edit#heading=h.6mboifb41tl8

[jessegreenberg]

Extension of #193

[terracoda]

@jessegreenberg, I added brief and verbose "density phrases" to the design doc for when Constant Size is checked.

[terracoda]

Here's my evaluation notes of our initial design & implementation. We can chop these notes into new issues as needed.

Issues for 1.1.0-dev.2

When set to Brief Explore mode & Brief Interactive mode

1. Brief explore strings seem correctly implemented.
2. Issues with Brief Interactive Strings - Mass Controls Issues
   • 1. Not hearing changed mass value, and only hearing the context string (Bigger mass, bigger forces). Interaction should go like this:
     1. Hover/Focus: Mass 1, 2 billion kilograms
     2. Increase: 3 billion kilograms. Forces bigger, now {{new force value}}
     3. Decrease: 2 billion kilograms. Forces smaller, now {{new force value}}
   • 2. Wording issues:
     1. When Brief Interaction is chosen, I sometimes hear “direction” when it should just be “left” or “right” depending on which mass sphere moves - sometimes it is mass 2 and sometimes it is mass 1.
   • 3. Design issue: Brief interactive mode for mass controls has wording that is too direct - could use same approach as when changing distance, and just add force change after the new mass value. This means we would not describe size or density change:
     1. Increase: 3 billion kilograms. Forces bigger, now {{new force value}}.
     2. Decrease: 2 billion kilograms. Forces smaller, now {{new force value}}.
     3. Increase with movement where changed sphere pushes other sphere
        1. 7 billion kilograms. Mass {{1}} moves mass {{2}} right. Forces bigger, {{forceValue}}.
     4. Increase with movement changed sphere moves back because other sphere is at edge
        1. Hover/Focus: Mass 1, 5 billion kilograms.
           1. Then an Increase with movement where mass 2 is at edge:
           2. 6 billion kilograms. {{Mass 1 moves left}}. Forces bigger, forces now {{forceValue}}.
        2. Mass 2, 3 billion kilograms. {{Mass 2 moves right}}. Forces bigger,
        3. Increase where mass 1 is at edge
           1. 6 billion kilograms {{Mass 1 moves left}}. Forces bigger, {{forceValue}}.

When set to Brief Explore mode & Verbose Interactive mode

• 3. Missing Response on checkboxes
  1. Don’t hear verbose interactive response when checkboxes are changed, but everything else seems fine.

When set to Verbose Explore mode & Verbose Interactive mode

• 4. Similar Wording issue:
  1. When Verbose Interactive is chosen, I sometimes hear “left or right” when it should just be “left” or “right” depending on which mass sphere moves - sometimes it is mass 2 and sometimes it is mass 1
  2. Everything else seems correctly implemented.

When set to Verbose Explore mode & Brief Interactive mode

• 5. Wording issue seems to be:
  1. When Brief Interaction is set, I hear “direction” when a mass change moves the other sphere.
  2. When Verbose Interaction is set, hear “left or right”.

When either Verbose or Brief Interactive options are selected

• 6. Interactive Response occurs when no change in position is possible:
  1. I hear an interactive response (context response) even when I cannot move the spheres closer together. To reproduce - put spheres together, clear interaction by mousing over another object, go back spheres, and try to move one closer even though it can’t get closer. I am getting a distance and force increase response in this case. Basically, the forces bigger or smaller part is problematic, the rest of the information is accurate current state information. Possible pathways:
     1. Prevent responses if there is no change to the model?
     2. Provide current state information when learner tries go beyond the max. We did this with regular Interactive Descriptions. There are special Context Responses that deliver current state information when the learner tries to go beyond the boundary. Design Questions:
        1. Is it possible to do the same in S-V?
        2. Would it be useful? Might be better to remain silent. Here are 2 example Context Responses from Interactive Description for Distance and Mass
           1. Closest to mass 1, force arrows small, forces 315.9 newtons.
           2. Extremely close to mass 1, force arrows very small, forces 184.7 newtons.
           3. Much much larger than mass 2, force arrows medium size, forces 551.6 newtons.
           4. Extremely dense compared to mass 2, force arrows very large, forces 1579.7 newtons.

Additional issues found by JG:

• When masses are at the egde of the screen, sometimes the incorrect mass will be described as moving when the other gets pushed.
• As part of improvements to this version I will try to resolve the issue where you can't hear "Explore" responses when moving the cursor twice over the same object.

[terracoda]

@jessegreenberg, I don't know how to make the bullets tidier. I did my best. We can break these issues up into separate issues as we decide to address them.

[terracoda]

@jessegreenberg, can you you give me an idea of when you can work on the issues in #196 (comment) think they are small. Do you need me to split them up a bit?

[jessegreenberg]

When Brief Interaction is chosen, I sometimes hear “direction” when it should just be “left” or “right” depending on which mass sphere moves - sometimes it is mass 2 and sometimes it is mass 1.

Hmm, I tried to get this to happen with brief interactive mode and haven't been able to observe it in about 10 minutes of trying. Its possible this has been fixed since #196 (comment). @terracoda if you observe this on master please let me know, but checking off for now.

EDIT: Same thing for

When Verbose Interactive is chosen, I sometimes hear “left or right” when it should just be “left” or “right” depending on which mass sphere moves - sometimes it is mass 2 and sometimes it is mass 1

I haven't been able to get this to happen after several minutes, checking off for now.

EDIT: Same thing for

When Verbose Interactive is chosen, I sometimes hear “left or right” when it should just be “left” or “right” depending on which mass sphere moves - sometimes it is mass 2 and sometimes it is mass 1

Checking off for now.

[jessegreenberg]

For #6 I went with

Prevent responses if there is no change to the model

Just because it was very easy to do. Custom alerts at the end of the track are possible however, if we want to explore that more.

[jessegreenberg]

@terracoda here is an updated link with items in the list addressed:
https://phet-dev.colorado.edu/html/gravity-force-lab-basics/1.1.0-dev.3/phet/gravity-force-lab-basics_en_phet.html?selfVoicing

This still doesn't include output from focus yet. Ill be working on that next but it doesn't seem related to this issue.

As I mentioned, I wasn't able to reproduce the bugs you reported. Code around those has changed, so perhaps they were accidentally fixed. Let me know if you still encounter them!

[jessegreenberg]

Here is an updated version that includes output from focus:
https://phet-dev.colorado.edu/html/gravity-force-lab-basics/1.1.0-dev.4/phet/gravity-force-lab-basics_en_phet.html?selfVoicing

[terracoda]

@jessegreenberg, I have found a few additional issues

1. In Verbose/Verbose mode, I don’t always get mass movement description when a mass change moves the other mass. The strings are correct when I hear them.

• Would it be difficult to get this a little more regular in Verbose/Verbose mode?

2. In Brief/Brief mode, with Constant Size checked, I hear mass changes intermittently and almost never hear anything about movement when movement happens. If this is a difficult issue to address due to the custom strings, I suggest we just leave movement off in brief mode all together. If it is just a simple issue, here are the strings that I should hear regularly with values checked and not checked (I think these are already implemented):

• 5 billion kilograms. Mass 1 moves mass 2 right. Forces bigger, now 333.7 newtons.
• 5 billion kilograms. Mass 1 moves mass 2 right. Forces bigger.
• 5 billion kilograms. Mass 2 moves mass 1 left. Forces bigger, now 333.7 newtons.
• 5 billion kilograms. Mass 2 moves mass 1 left. Forces bigger.

3. With Constant Size checked, movement never happens so no issue for movement. However, I did not give you a new phrase pattern that was less leading. I am actually, a bit stumped on this. I am wondering if we can do the following pattern for Brief/Brief. I realize this wording is different than when Constant Size is not checked. Is it a problem to use this phrasing? For example:

• 5 billion kilograms. More density, force arrows bigger. Forces now {{forceValue}} newtons.
• 5 billion kilograms. More density, force arrows bigger.
• 4 billion kilograms. More density, force arrows smaller. Forces now {{forceValue}} newtons.
• 4 billion kilograms. More density, force arrows smaller.

@jessegreenberg, on another note, @emily-phet and I had a deep conversation about next steps, which differs a lot from this version, so if any of the above are really tough issues, let's discuss before spending too much effort on a fix we might not use. We can address them in the new approach, if need be.

[terracoda]

This Self-Voicing design is what we are calling Paradigm 1.

[jessegreenberg]

@terracoda I am curious about your thoughts on this issue at the moment. Should we do any more to improve this prototype or is this one phasing out?

[emily-phet]

@jessegreenberg This version is an "artifact". One we want to be able to refer to, share with colleagues as a contrasting example, etc. but not move forward with for further design iterations or to move towards publication.

[terracoda]

@jessegreenberg, this version might be a good candidate for a build that we can save to share in papers - something we can freeze in time and refer back to. I don't think we would need to return to the code to work on this version any more.

[terracoda]

@jessegreenberg, can you save me a version and post the link in this issue.
I'll copy the link to the Design Benchmarks document.

(no rush)

[jessegreenberg]

The above commits should address the problems you identified @terracoda. Here is a version with these changes that we can reference in the future:

https://phet-dev.colorado.edu/html/gravity-force-lab-basics/1.1.0-selfVoicingParadigm1.1/phet/gravity-force-lab-basics_en_phet.html?selfVoicing

Once you have this version documented where you want it I think this can be closed @terracoda.

[jessegreenberg]

Discussed today - good to have this version documented and we can close this issue and save to delete branches.

[jessegreenberg]

Assigning to myself to delete branches and then close.

[jessegreenberg]

Branch was deleted, closing.