Sound: Implement new slider click sounds and get feedback

[Ashton-Morris]

I spoke with Ariel and we liked the subtlety of the step forward button.

So here is a shortened version of that to try

click-right
click-left

[Ashton-Morris]

@samreid I can't add assignees so hopefully you are notified of this.

[samreid]

Since this will be implemented in the "wave interference" repo, I'll move the issue there for tracking.

[samreid]

I added the proposed sounds. @ariel-phet or @Ashton-Morris can you please review?

[ariel-phet]

I think the sound needs to be volume balanced, but it certainly seems much less "geiger counter" to me. I think the sim as a whole needs to be reviewed once volume balanced, but to me this issue solved the main problem I was hearing.

[emily-phet]

1. (most important!) The sounds I hear when I move the slider are more “click"-y (and less "bump"-y) than I remember form the last design meeting that @jbphet or @Ashton-Morris demoed. @samreid and @Ashton-Morris - can you confirm that this is latest sound for the sliders - Here's the sound I hear (note - slight warping at the end of each click sound is an artifact of my recording this as it is played from my computer speakers). Let's make sure we're all considering the correct bump-y sound (and not the default click-y sound in common code) before proceeding.

2. Timing - Also I wish the timing of the clicks sound felt more consistent with the visual tick marks across the slider range. It lines up nicely when moving slowly, but feels slightly off at times when moving more quickly. Is there any way to associate the click sound with the literal visual location of the slider/grid marks? I feel like we’re discussed this before, but I think the more associated the click sound is to exactly the grid marks on the slider, the more “intuitive” the sound will feel and less attention-grabbing. @jbphet - thoughts on this?

3. End sounds - one thing that might help us out in tweaking the sounds so that they feel more like they are associated with the tick marks is to have a slightly different sound for the extreme ends of the slider (when you've "hit the edge" so to speak). @Ashton-Morris - once we are clear on which version of the sound is the right one to move forward with (the bump-y one!), please make a variant that is a subtle indicator max/min has been reached. I imagine max/min can be the same sound, but if you think it's better to have these be slightly different, please go ahead.

4. Volume - I think if the volume is decreased further in the final sound balancing (these should be barely noticeable at medium volume), it would help. Let's not deal with that in this issue, though. It will depend if there is a mix-up on the sound timbre, a "bump"-y sound may need to be slightly louder than a more "click"-y sound to be perceived as the same volume.

[samreid]

Another issue about the slider "clicking" sounds: on the water drop screen, the slider clicks sound kind of like water drops: may be a little confusing.

[jbphet]

The sounds I hear when I move the slider are more “click"-y (and less "bump"-y) than I remember form the last design meeting

I just listened to the current master version, and I'm pretty sure it is the same sound. It sounds less "clicky" in headphones to me, more so over my computer speakers.

[T]he timing of the clicks...feels slightly off at times when moving more quickly...@jbphet - thoughts on this?

I just tested this, again using the current master version, on Chrome and Firefox on my Win 10 machine, and I don't find the timing to be noticeably off when moving fast. This might be a browser-specific thing. @emily-phet - can you let us know what browser and OS you're using when you see this, and can you test whether it occurs on other browsers on that same machine.

My thoughts are that I would have to duplicate this, make some measurements, and see what changes could be made to improve it. Different browsers may be handling the situation where there are a lot of sounds to play at nearly the same time differently, and we may be able to make some adjustments to improve the performance in such situations.

[emily-phet]

(Updating my previous comment after discussion with @jbphet)

1. (most important!) The sounds I hear when I move the slider are more “click"-y (and less "bump"-y) than I remember form the last design meeting that @jbphet or @Ashton-Morris demoed. @samreid and @Ashton-Morris - can you confirm that this is latest sound for the sliders - Here's the sound I hear (note - slight warping at the end of each click sound is an artifact of my recording this as it is played from my computer speakers). Let's make sure we're all considering the correct bump-y sound (and not the default click-y sound in common code) before proceeding.

With @jbphet it's clear I'm hearing the correct sound. We concluded that my impression of it during last week's meeting was influenced by hearing the original click sound...comparing the new sound to the old, the newer one definitely sounds 'bump'-y, but I think we can still improve upon it. @Ashton-Morris can you please propose one or a few variations of a 'bump'-y sound, rather than a 'click' sound. We can discuss further in tomorrow's meeting if that would be helpful. Whatever we go with, we'll also want sounds specifically for the max/min points (from 3) below).

2. Timing - Also I wish the timing of the clicks sound felt more consistent with the visual tick marks across the slider range. It lines up nicely when moving slowly, but feels slightly off at times when moving more quickly. Is there any way to associate the click sound with the literal visual location of the slider/grid marks? I feel like we’re discussed this before, but I think the more associated the click sound is to exactly the grid marks on the slider, the more “intuitive” the sound will feel and less attention-grabbing. @jbphet - thoughts on this?

@jbphet had a number of suggestions as to why the timing might feel off, and why the sound may be perceived as amplified or more noticeable when the slider is moving quickly (I believe he'll list these in a later comment).

3. End sounds - one thing that might help us out in tweaking the sounds so that they feel more like they are associated with the tick marks is to have a slightly different sound for the extreme ends of the slider (when you've "hit the edge" so to speak). @Ashton-Morris - once we are clear on which version of the sound is the right one to move forward with (the bump-y one!), please make a variant that is a subtle indicator max/min has been reached. I imagine max/min can be the same sound, but if you think it's better to have these be slightly different, please go ahead.

See 1) above.

4. Volume - I think if the volume is decreased further in the final sound balancing (these should be barely noticeable at medium volume), it would help. Let's not deal with that in this issue, though. It will depend if there is a mix-up on the sound timbre, a "bump"-y sound may need to be slightly louder than a more "click"-y sound to be perceived as the same volume.

No update on 4) to add.

Unassigning @samreid for now, as I think there's some things here for me, @Ashton-Morris and @jbphet to sort out for the moment.

[jbphet]

In the previous comment, @emily-phet mentioned that she and I discussed why the slider sounds a bit odd and artificial. I think a possible cause of this is that the sounds are initiated only at very specific times based on the 60 frames/sec rate at which the browser runs. As a result of this, sound can end up with the same phase relationship all the time, which can add together and boost the volume. In a situation where a lot of sounds are being initiated at once, such as when the slider is moving very quickly, it may even be that the sound clip is being initiated more than once in a given animation frame. There are several things that I can think of that might serve to address this.

• Use a set of sounds that are similar but slightly different, and choose between them randomly each time a new one is played. We do this in the "ratchet" sound for John Travolatage's arm, and it works pretty well there.
• Prevent the sounds from being initiated in the same animation frame. I'll need to check if this is actually happening first.
• Once a sound is played, play it more softly if it is played again within a certain time window.
• Insert a small, random delay for each sound that is played (it would probably be best to make this an option that is off by default) to prevent them from overlapping and adding and thus increasing the volume.

Once @Ashton-Morris has created the new sounds and they have been integrated, @samreid and I can collaborate and try some of these out and find which ones are the most effective at addressing the issue.

[Ashton-Morris]

I have experimented with some more slider clicks for everyone to listen to. I have individual files but I also recorded a few examples of how it would sound dragging it from left to right. Hear below.

slider-clicks-idea-a-example.mp3
slider-clicks-idea-b-example.mp3
slider-clicks-idea-c-example
slider-clicks-idea-d-example.mp3
slider-clicks-idea-e-example.mp3
slider-clicks-idea-f-example.mp3

[jbphet]

I listened to each of the sounds that are linked in the previous comment and below is how I would rank them, with notes on each. I listened on headphones and did not evaluate on any speakers. These are my opinions, your mileage may vary.

1. idea-c - This is similar to the current sound but nicer, a bit more of a "bump". There are two end sounds in the example, and I think we should just use one to signify an endpoint, and my preference would be the 2nd one.
2. idea-d - Also pretty similar to the current sound, but better. I just like c a little more.
3. idea-f - This one is a little much, but might work if the volume is set pretty low.
4. idea-e - A little too "toneful". Sounds kind of like a slot machine.
5. idea-b - A bit too game-like and would thus bring more attention to the sound than I think we're after.
6. idea-a - Sounds too much like a small laser being fired in a video game.

[jessegreenberg]

Are we guaranteed that drag will be called before the property value changes?

This seems like a very deliberate decision in Slider.js, I assume this not going to change.

I wonder if we need to make sure that using the keyboard puts it at 1.00000 instead of 0.9999999 in the first place, so there's no discrepancy there. How could this be done?

Could we use constrainValue for this, something like

constrainValue: value => Utils.toFixedNumber( value, 5 )

I don't see a better way currently within the drag callback to get what the value is going to be and get around the timeout. We could potentially add a new callback option to Slider.js and AccessibleValueHandler.js to support this. But maybe it would be acceptable to have sound from alternative input happen on endDrag?

      endDrag: event => {
        if ( event.isFromPDOM() ) {
          if ( property.value === property.range.min || property.value === property.range.max ) {
            sliderBoundaryClickSoundClip.play();
          }
          else {
            sliderClickSoundClip.play();
          }
        }
      }

The downside of this is that you wouldn't get any sound while pressing and holding an arrow key, only on release.

[zepumph]

Are we guaranteed that drag will be called before the property value changes?

This seems like a very deliberate decision in Slider.js, I assume this not going to change.

It looks like it was added as part of phetsims/sun#561

At first glance it seems quite buggy to have the callback before the Property changes. I commented over in phetsims/sun#561 (comment) about that.

Everything else here is looking very nice!

[samreid]

Blocked until phetsims/sun#561 is improved.

[emily-phet]

@jbphet I'd like to discuss today any tweaking we can do to decrease the total number of slider sounds heard when moving the slider very fast.

[jbphet]

We made a mod in today's sound design meeting, @emily-phet will try it out and report back.

[emily-phet]

@jbphet - Checked latest in phettest - changes from today sound good to me! Let's go with that. Next step would be a final sound volume mix check-off from @Ashton-Morris, and then send out to full team for final input (ideally on sound volumes only, limiting other feedback to only if absolute deal breaker for moving forward for publication).

[Ashton-Morris]

I reviewed the levels today and determined that the were actually spot on! No changes were needed from my perspective.

@emily-phet You can get this to the full team for final input.

[emily-phet]

Off to final feedback from the team. Closing this issue, as feedback will be requested on the sim as whole, not just the slider sounds.

[jessegreenberg]

Reopening for the remaining questions about alternative input. phetsims/sun#561 is done so this is no longer on hold and we should be able to remove the timeout.

The other question from #484 (comment) was

I wonder if we need to make sure that using the keyboard puts it at 1.00000 instead of 0.9999999 in the first place, so there's no discrepancy there. How could this be done?

Can constrainValue be used for this?

[samreid]

I used constrainValue and the new slider interface, works great! Thanks @jessegreenberg, closing.