Sound audit

[zepumph]

Now that #158 has been completed, and we have a good first draft of #173 in place, we want to take a holistic look at the sound implementation (including description) to see if we need to make any changes.

The latest dev version is https://phet-dev.colorado.edu/html/ratio-and-proportion/1.0.0-dev.63/phet/ratio-and-proportion_en_phet.html

Please ask questions and comment below, no matter how small!

[jbphet]

I just reviewed the version linked above, and wow - we've come a long ways with this. I like having the "grid line bump" sounds behind the "Enhanced Sounds" checkbox.

Here are some thoughts and observations from my review:

• I think something got a little messed up with some of the pitches for the pluck sounds, because they don't seem to be consistent at all proportion levels, and can jump around a bit. To reproduce, go to the first screen, turn on the numbered grid lines, and move the right hand to grid line 5 and hold it there. Not all of the pluck sounds are at the same pitch.
• I like the boundary sounds for the top and bottom limits of the hand motion, and it felt a little unnatural to me that we didn't have them for the right and left limits of their motion, especially since the visual cue isn't as strong for this.
• With enhanced sound turned on, I feel like I was hearing the bump sound in some cases where it shouldn't be there. To reproduce, turn on enhanced sound, go to the first screen, select challenge 2, turn on the grid lines, put the left hand at 2, and move the right hand back and forth over 6. I'm pretty sure the bump sound was being played in this case, but not when the right hand crossed 5 or 7.
• On the 2nd screen I was able to get some odd sound behavior when locking ratios. To reproduce, turn on the grid, then move the left hand to just below two - far enough that it doesn't snap back. Then move the right hand to just barely be in the "success" position and release it. Lock the ratio. Then keyboard nav to the left hand and move it up and down. The pluck, success, and angels all play at various times.

[zepumph]

With enhanced sound turned on, I feel like I was hearing the bump sound in some cases where it shouldn't be there. To reproduce, turn on enhanced sound, go to the first screen, select challenge 2, turn on the grid lines, put the left hand at 2, and move the right hand back and forth over 6. I'm pretty sure the bump sound was being played in this case, but not when the right hand crossed 5 or 7.

Implemented above. @Ashton-Morris, @jbphet and I felt like a quorum enough to implement this change.

[Ashton-Morris]

When you choose from the accordion box "Challenge 2" and then choose "Challenge 1", since it starts In Proportion it rings out the in proportion sound.

This feels like the continuity is off because when you switch from difference scenes or from home to different scenes, it doesn't play the In Proportion sound even though the sim starts In Proportion (full green) for both scenes.

[emily-phet]

@terracoda mentioned that the radio buttons sound popping may have sounded like Jaws changing forms mode. We haven't heard this before from other JAWS users, but something to think about for future reference.

[zepumph]

I didn't get the feeling that we "signed off" on this issue this week. Let's touch base again next week after I implement all the changes from today.

[emily-phet]

@zepumph Please bring this up in the tuesday morning meeting. I think at that point, @terracoda and @Matthew-Moore240 will be in a good place to weigh in on whether or not there is a lot gained by having that sound for non-visual access.

Here's how I'm thinking of the problem:

• Screen 1 - challenge combobox. Is it important for visual and non-visual access to have the success sound play when interacting with challenge combobox?
  We decided we didn't want the sound today to support visual access, and think it is probably fine to not have it for non-visual access, but want @terracoda and @Matthew-Moore240 to weigh in on this. I think we should remove the success sound from being heard when you are not interacting with the hands unless @Matthew-Moore240 and @terracoda feel like it's providing important information for non-visual access. So if they're neutral or negative about the sound, we should remove it.

• Screen 2 - my challenge number spinners (these feel like different scenarios to be thought out). Is it important for visual and non-visual access to have the success sound play when interacting with the number spinners? We didn't think deeply about this today from a visual perspective, and @terracoda and @Matthew-Moore240 are still working out some kinks to the non-visual access. I would lean towards not having the success sound play during use of number spinners unless there is a strong reason (visual or non-visual) or a strong gut feeling from a designer to keep it there.

So essentially, we need to think through both the Screen 1 combobox (non-visual access), and the Screen 2 number spinners (visual and non-visual scenarios), and if we don't have a strong reason that it would provide significant benefit or introduce significant confusion, we should nix the sound.

[emily-phet]

@terracoda @Matthew-Moore240
Can you all weigh in on this (#174 (comment)), or let me know if you have questions? If so, we could have a quick chat about this later next week. It relates only to when you hear the "success" sound from interaction not directly involving moving the hands in the play area.

[terracoda]

@jbphet said, "I like having the "grid line bump" sounds behind the "Enhanced Sounds" checkbox."

Question: What is the grid line bump? I don't hear anything indicating grid lines.

[terracoda]

@jbphet said, "I like the boundary sounds for the top and bottom limits of the hand motion."

Ah Ha! I didn't hear boundary sounds while using a keyboard. I see now that's why I don't hear the grid line bumps, either. They both need to be added for keyboard input.

I think both these sounds will be useful for blind keyboard users!

[terracoda]

Related to the success sounds when using the secondary controls, this is tough question.
The main issue I have is that screen changes color and the sounds communicate those changes.

@emily-phet, are you suggesting that we don't communicate changes in proximity to the newly selected or newly set ratio challenge at all, or are you suggesting that we only communicate that change through the primary access modality (visuals or description) as a way of encouraging more exploration with the hands?

Possible alternative would be to make the sounds more distant or in the background when using the secondary controls, just an idea?? That might be confusing?

Related to other sounds that I had noticed until today, I personally, do not think the grid line bump sound should be an enhanced sound, it's really nice. And I think it would be quite important to blind learners. This sound was not heard during interviews. I am not sure it was implemented yet, or just not heard because participants used the keyboard.

[jbphet]

@terracoda said:

Question: What is the grid line bump? I don't hear anything indicating grid lines.

To hear this sound, you need to use the PhET menu to turn on Enhanced Sound. Try that, and you should be able to hear it.

[terracoda]

@emily-phet, I also want to remind ourselves that early on in the design process, a decision was made to put all the controls in the Play Area.

Is there something in your thinking about the sound that would also indicate that the other controls would be better placed in the Control Area?

I am not saying we should, I am just wondering about the controls themselves and what we have done in other sims.

If our goal is to make sure learners play with the Hands more than the other controls, maybe we should at least briefly revisit if the Play Area/Control Area division. Is it truly what we think it should be?

[terracoda]

@jbphet, thanks, I had enhanced sounds on. The issue is that I don't hear the enhanced sounds when navigating using the keyboard. I hear them with the mouse interaction only.

I think @zepumph can fix this :-) I'll make a new issue.

[emily-phet]

@terracoda - re #174 (comment)

From our past decisions on Play Area / Control Panel organization, I think the current setup aligns. People would have a really hard time getting much out of the sim if they do not use the radio buttons and combo box. Previously that has been part of the reasoning behind putting those sorts of things in the Play Area vs Control Panel.

This doesn't have to be a hard and fast rule, but unless we see a specific issue (or think an issue is likely) being caused by our current organization, it seems reasonable to me.

Did you see an issue you think reorganization could address? (also, I don't think this has to do with sound, so if you think more discussion is merited here, we should probably take it to another issue. :)

[terracoda]

@emily-phet, I agree, let's leave the Play Area/Control Area division as it is.
It is also not about sound, so not part of this issue.

[terracoda]

It's possible, as long as we have a voiced response that the ratio situation with the hands has changed (we could keep it very general), I think we can could avoid using the sounds.

The latest description ideas for the Challenge Ratio combobox and the My Challenge number spinners would indeed voice the changes to the proximity to the challenge ratio. We could see how learners respond to having sound in one case, but not the other.

If the description is working as we hope, it should indicate that playing with the hands after making a change to the challenge ratio is the fun thing to do.

[zepumph]

@Matthew-Moore240 do you have any thoughts to add to this issue? Otherwise I think we are ready to close.

[Matthew-Moore240]

Yeah I think its ready to close. Closing