Prototype the sound design

[jbphet]

The sound design group has been working on a sound design for this sim for a while, and even created an initial shot at the sound design a while back that can be seen (and heard) at https://phet-dev.colorado.edu/html/phet-io-wrapper-sonification/1.0.0-dev.125/build/phet-io-wrapper-sonification/molecules-and-light/html/molecules-and-light-sonification.html?accessibility. A decision was recently made to do sound prototypes on master rather than in phet-io wrappers, so this issue is about implementing that sound design.

This issue will track the commits for adding the sound design prototype and the feedback on it. Once the design is finalized, a separate issue should be created for completing the sound implementation.

[jbphet]

I have a version that is ready to review and get feedback on. It allows selection of different photon sounds via PhET menu->Options..., and integrates the currently set of common UI sounds (this was done by building it as a one-off on a branch). It can be found at https://phet-dev.colorado.edu/html/molecules-and-light/1.5.0-soundDesignProto01.1/phet/molecules-and-light_en_phet.html?supportsSound.

[jbphet]

@Ashton-Morris and @emily-phet - there is a prototype available at the link in the comment just above this one. Please review and provide your comments, or we can review in real time at the next meeting. This version includes the current set of common UI sounds.

[emily-phet]

Did a quick review - a few quick thoughts:

• timing of photon sound seems off - I think the sound should fire as we can see the photon appearing from the source (in real time and in step forward mode)
• boingy vibration sound (e.g., Infrared + CO) sounds too game-y. Makes me think of a literal spring. "Feels" a little better for CO2 than CO, not sure why!
• (bug) boingy/glowing sound gets stuck on
• Glowy sound (e.g., visible/NO2) sounds a little too heavy
• Molecule break apart sound feels too "physical"

[jbphet]

An issue that I'll need to address is that the sounds keep going if they are in progress when the pause button is pressed. @Matthew-phet noticed this too.

[jbphet]

Here are some notes from today's review meeting:

• Most folks thought the vibration sound was a little too "cartoon-y". @Ashton-Morris was in the meeting, and will take the feedback and provide some new options to try out. I'll add them along with some options selections for choosing between them.
• The emission of photons from the sources sounds the same as the emission from the molecules, and this makes it hard to understand what's going on. We decided to try to use some of the alternative photon emission sounds for what comes out of the molecules. I'll add this capability, with an options selection to choose.
• A handful of people thought that the glowing sound was a little too "heavy" @Ashton-Morris wiil try to "lighten" the sound a bit
• Several people thought that the break-apart sound was a little to "literal". @Ashton-Morris suggested that we try one of the sounds that is currently in use in the tambo demo. @jbphet will integrate this. Depending on the time frame, this will either be done as another option selection or just a replacement for the existing one.

[jbphet]

I've incorporated the feedback from the last meeting and some new sounds supplied by @Ashton-Morris into a new version, see https://phet-dev.colorado.edu/html/molecules-and-light/1.5.0-soundDesignProto01.3/phet/molecules-and-light_en_phet.html?supportsSound. Remember that different sounds can be selected using the "Options..." dialog from the PhET menu. We will review at today's meeting and decide which configurations we like best and what the next steps should be.

[jbphet]

Review during 10/22/2019 meeting

• The group likes the combo of photon sound set 3 for initial emission and set 1 for re-emmision.
• We like the new, softer break-apart sound over the harsher one demoed last week
• The higher pitched glow sound is approved
• Here are some of the opinions expressed about the glow sound
  • option 3 is probably too low, since it doesn't come across on laptop speakers and perhaps a little too physical
  • option 5 people really like, but it's perhaps "too fun" or "too silly"
  • option 1 sounds a bit too much like the glow sound
  • We didn't come to any consensus on this. @Ashton-Morris will try to mess around with options 4 and 5, which people seemed to like but thought were a bit too much.

[jbphet]

After using the sim quite a bit, I'd like to suggest that we soften the attack of the primary photon emission sound (i.e. when they come out of the sources). I like the basis of the sound itself, but it wears on my a bit if I use the sim for a while, and it think it's the strong "ping" at the onset of the sound that's causing this effect for me, and I think softening the attack will help.

[terracoda]

I agree with @jbphet, the light sources are little loud.
I also find the rotating sound subtle and the break apart sound quite subtle.
I am not sure if volume would decrease the subtle nature of these 2 sounds.

I always find the reset button loudish ;-) I have no idea if anyone else feels this way.

[brettfiedler]

Generally on board with JB & TS. It does feel a little like something is being physically hit with the light souecw, though I still like the quality of the tone after the attack.

Also, I agree on the break-apart volume. It seems to me that clicking the Light Spectrum button is louder than the molecule breaking apart? Though that could be wrong. Doesn't seem like it needs to change too much.

For me the rotation is fine, but it seems more subtle with that primary photon attack dominating the noise-space for me. I think it might be less subtle after that has been diminished?

[Ashton-Morris]

I think the emitting photon sounds good but the main photon might have too much attack to it. I agree the reset is louder. We should bring up the levels of the other sounds a bit to adjust to that.

[jbphet]

Notes from 11/2/2019 review meeting:

• I modified the existing V3 photon emission sounds to have a softer attack and demonstrated them, and there was general agreement that this was an improvement. @Ashton-Morris also created some similar variations, but I didn't have time to integrate them before the meeting, so I'll compare the two sounds sets and choose one.
• We're okay with the stronger attack on the sounds when emitted from the photons. This happens much less frequently, so isn't as wearing on the listener as those coming from the lamps, and the stronger attack helps to make the sound more distinct.
• We adjusted the volume of the break-apart sound and are good with it now.
• We discussed using distinct-but-similar-to-one-another sounds for the different directions of rotation in order to cue listeners that something different is happening but without drawing too much attention to it. @Ashton-Morris will try to generate a new pair of sounds to use for this based on the current rotation sound (since the general consensus is that we like that sound).
• @BLFiedler pointed out that the default radio button sounds for the frequency selection go down in pitch, whereas the frequencies selected are going up. After discussing this for a while, we decided that these should probably be reversed for this particular case.

Adding @Ashton-Morris as an assignee for the rotation sounds. I'll follow up on the other items.

[jbphet]

Notes from 12/3/2019 review meeting:

• softer attack sound for initial photon emission (i.e. from the lamps) sounds good, we'll go with this
• increasing frequency for emission selector radio buttons sounds good
• for the rotation sound, option 1 for clockwise and option 2 for counterclockwise sounded best to the group

The dev version at https://phet-dev.colorado.edu/html/molecules-and-light/1.5.0-dev.17/phet/molecules-and-light_en_phet.html?supportsSound has all of these as the default sounds. This is probably the last version that I'll publish with options, since we've agreed upon the sounds.

[jbphet]

The next step will be to work out the mix.

[jbphet]

The prototyping work is essentially complete, and the sound selections have been finalized. Various published versions of the prototypes can be found at the links in comments above. I'm going to close the issue and track the final implementation steps in #233.