Finalizing sound for the sim

[zepumph]

Copied from https://github.com/phetsims/tambo/blob/3c84d4124b7aa20ea6d33ab5a5e440b4db3e318c/doc/adding-sound-to-a-sim.md. . .

Checklist for Adding Sound to a PhET Simulation

This checklist is intended to guide developers through the process of adding sound to a simulation. It is written at
a fairly high level, and assumes certain knowledge of how the tambo library works and of PhET's development processes.
Before using this for the first time, it is highly recommended that a developer reads through the Users Guide.

• Create a sound design document and work with the sound designer(s) to get to the point where enough has been
  worked out that it becomes desirable to start prototyping. For an example, please see the Molecules and Light Sound
  Design document.
• Create an issue for adding sound to the simulation, use the title "Implement sound design". Here is an example
  GitHub issue, though it doesn't use the suggested name. See Implement sound design #9
• Figure out if it's okay to turn sound on for the simulation. This will hinge upon whether there will need to be
  releases made off of master before the sound implementation is complete. Generally you'll want to turn sound on, but in
  cases where that isn't possible, skip the two sub-steps below, and the query parameters supportsSound and
  supportsEnhancedSound can be used to turn sound on for testing during development.
  • Turn on sound by adding the soundSupported: true key-value pair to the phet sub-object in the package.json
    file for this simulation. See https://github.com/phetsims/friction/blob/master/package.json.
  • If the sound design includes enhanced sound, turn this on by adding the enhancedSoundSupported: true key-value
    pair to the phet sub-object in the package.json file for this simulation. The main effect that this is has is to add
    the 'Enhanced Sound' item to the PhET menu. See https://github.com/phetsims/friction/blob/master/package.json.
  • Run grunt update to get the updated configuration into the HTML file.
• Explicitly turn off sound for any common UI components that now produce sounds that are not needed, if there are
  any. This is done by looking at the options and setting any sound players to Playable.NO_SOUND. Search through the
  code base for example usages.
• Add any behavior for any common-UI sound generation that is different from the default behavior. This is done
  by creating a Playable (often a SoundClip instance) and passing it in as an option for the sound player.
• Decide whether or not to have a separate "sound view" or to use the existing ScreenView files. Both approaches
  have been used, and both are legit, and it probably depends on how much sound generation is needed and whether adding it
  all to the ScreenView file(s) is likely to make the file too large and/or difficult to maintain. As of this writing,
  WavesScreenSoundView is an example of a separate class where the sounds are hooked to the model, and
  FrictionScreenView is an example of where sound was interwoven with the visual view code. Note that in some cases it
  is necessary to know what the user was doing that triggered the need for sound generation, and in these cases the sound
  generation will need to be inside the view element so that the code has access to all the needed information. An
  example where this often comes up is if different behavior is needed when a value was changed via keyboard interaction
  versus mouse or touch interaction.
• Decide whether a "Sound Options" dialog is needed to allow designers to compare different sound design ideas in
  context. This has been found to be very useful when iterating on a sound design. There is more information on this in
  the User Guide, and [an example in the Tambo demo]
  (https://github.com/phetsims/tambo/blob/master/js/demo/SoundOptionsDialogContent.js).
• Add sim-specific sound generation. See previously sonified simulations for examples on how to do this, but the
  general idea is to create sound generators and hook them up to the model and/or view elements that they are meant to
  sonify, all based on the sound design document.
• Iterate on the sound design. Regular meetings and good note taking (generally in GitHub issues) have been found
  to be quite helpful for this. This is basically like the implementation process for all other portions of the sim, and
  involves publishing dev versions, getting feedback, refining the sim, rinse, and repeat. One observation: It seems to
  be more difficult for people to imagine how they will like a sound in context than a visual design element, so be
  prepared for a lot of iteration, and use the 'Options' dialog if and when it can help.
• Once all of the sounds have been finalized, do a "mix" step where the primary sound designer goes through the
  sim and sets the volume level for all sim-specific sounds.
• Once the sound design is thought to be complete, or nearly complete, publish a dev version and create a checklist
  of all of the sound designers and other stakeholders in the original sound design GitHub issue and ask them to either
  approve of the implementation or log their objections. Example:
  Finalize the sound implementation molecules-and-light#233 (comment)
• Finalize the names of all sound files that have been added and get uncompressed versions of them and add them to
  the assets directory. Make sure the names match, e.g. bonk.mp3 in the sounds directory and bonk.wav in the
  assets directory.
• If an options dialog was added or if content was added to an existing options dialog to support comparitive
  evaluation of sounds, remove said content
• If there were individuals who were only involved in the sound design and thus not already on the team or
  other lists in the credits, add them using the soundDesign key.
• Once the sound design is approved, mark the issue as ready-for-testing and have it tested with the other
  aspects of the simulation through the publication process.

[zepumph]

soundDesign credits are @jbphet @Ashton-Morris.

[zepumph]

Notes on mixing:

• We want to keep the grid line crossing bump a quiet, and we like that it won't play when the staccato sound does.
• We may want the staccato sound a bit quieter

[zepumph]

• Boundary sound for the ratio hands when at the edges should play even when the grid lines aren't visible.

[Ashton-Morris]

@zepumph The one thing I thought needed changing but couldn't find was the sound for when you click on the PhET icon on the bottom right. Since it is an abrupt sound, it comes across to me as too loud. That sound play at 80% its current volume.

Index: js/common/view/sound/StaccatoFrequencySoundGenerator.js
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- js/common/view/sound/StaccatoFrequencySoundGenerator.js	(revision 87a2e1170850f119e02a3d4784cefd04c663cb0e)
+++ js/common/view/sound/StaccatoFrequencySoundGenerator.js	(date 1598305530000)
@@ -57,7 +57,7 @@
    */
   constructor( fitnessProperty, fitnessRange, model, options ) {
     options = merge( {
-      initialOutputLevel: 0.4
+      initialOutputLevel: 0.3
     }, options );
     super( options );
 
Index: js/explore/view/ExploreScreenView.js
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- js/explore/view/ExploreScreenView.js	(revision 87a2e1170850f119e02a3d4784cefd04c663cb0e)
+++ js/explore/view/ExploreScreenView.js	(date 1598305780000)
@@ -20,7 +20,7 @@
 
 // constants
 const SELECTION_SOUND_OPTIONS = {
-  initialOutputLevel: 0.5
+  initialOutputLevel: 0.4
 };
 
 class ExploreScreenView extends RatioAndProportionScreenView {
Index: js/common/view/RatioHalf.js
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- js/common/view/RatioHalf.js	(revision 87a2e1170850f119e02a3d4784cefd04c663cb0e)
+++ js/common/view/RatioHalf.js	(date 1598306267000)
@@ -120,7 +120,7 @@
     // Sound for the wave slider clicks
     const addSoundOptions = { categoryName: 'user-interface' };
     const soundClipOptions = {
-      initialOutputLevel: 0.3, // TODO: I made this louder than waves intro, https://github.com/phetsims/ratio-and-proportion/issues/45
+      initialOutputLevel: 0.15, // TODO: I made this louder than waves intro, https://github.com/phetsims/ratio-and-proportion/issues/45
       enableControlProperties: [ designingProperties.ratioUISoundsEnabledProperty, playUISoundsProperty ]
     };
     const commonGrabSoundClip = new SoundClip( commonGrabSound, soundClipOptions );
Index: js/common/view/sound/MovingInProportionSoundGenerator.js
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- js/common/view/sound/MovingInProportionSoundGenerator.js	(revision 87a2e1170850f119e02a3d4784cefd04c663cb0e)
+++ js/common/view/sound/MovingInProportionSoundGenerator.js	(date 1598305514000)
@@ -22,7 +22,7 @@
    */
   constructor( ratioFitnessProperty, model, options ) {
     options = merge( {
-      initialOutputLevel: .15
+      initialOutputLevel: .13
     }, options );
 
     super( options );

[zepumph]

Create a sound design document and work with the sound designer(s) to get to the point where enough has been
worked out that it becomes desirable to start prototyping. For an example, please see the Molecules and Light Sound
Design document.

Who is the lead sound designer? @BLFiedler I feel like you may be our guy because @emily-phet is in grant land. Could you do this please?

[zepumph]

Once all of the sounds have been finalized, do a "mix" step where the primary sound designer goes through the
sim and sets the volume level for all sim-specific sounds.

See #158

Once the sound design is thought to be complete, or nearly complete, publish a dev version and create a checklist
of all of the sound designers and other stakeholders in the original sound design GitHub issue and ask them to either
approve of the implementation or log their objections. Example:

See #174

[emily-phet]

@Ashton-Morris has been creating these. He may have already started one.

[zepumph]

Finalize the names of all sound files that have been added and get uncompressed versions of them and add them to
the assets directory. Make sure the names match, e.g. bonk.mp3 in the sounds directory and bonk.wav in the
assets directory.

See #191

[zepumph]

Once the sound design is approved, mark the issue as ready-for-testing and have it tested with the other
aspects of the simulation through the publication process.

I think that we should close this issue, and testing should be done via QA when we are publishing the sim. If there is something that QA is not set up to test sound-wise during testing, then that is a problem, and we should work on that. @jbphet can you speak to how QA is currently expected to test sound?

Everything else that hasn't been done, has side issues created for them, and have been checked off above.

All that is left is @Ashton-Morris to work on the design doc, and @jbphet to answer the question on QA testing.

[Ashton-Morris]

I have started the sound design doc and will complete it this week

[jbphet]

@jbphet can you speak to how QA is currently expected to test sound?

I generally make it clear that sound is a part of what needs to be tested when I set up the QA task. That's about it. So far, they just verify that it works, not that it matches a spec or anything.

[zepumph]

@jbphet, that makes sense to me, and things seem to be working well. Since we are on the topic, would you please look at https://github.com/phetsims/QA/blob/master/documentation/qa-book.md#sound-test. This is currently a single sentence:

To test sound in a simulation, make sure the sound effects work and sound good on all platforms.

Can you think of anything more to add to this to assist in the QA testing of sound?

[jbphet]

Can you think of anything more to add to this to assist in the QA testing of sound?

Sure, here are some thoughts, and I'll add a refined version of this to the sound implementation checklist sometime soon:

• Verify that all basic UI controls create some sort of sound.
• Verify that none of the sounds produced are disproportionately loud compared to the others, i.e. the balance is good. Note that the sound designers often use the Reset All button's sound as the volume reference.
• Play all sounds on multiple platforms and verify that there are no crackles, glitches, or anything that seems broken or significantly different on one platform versus the others.
• If the simulation supports "Enhanced Sound", test with it on and off and verify that the enhanced sounds play when this is on and don't when it is off. You may need to consult the sound design document or ask the developer to figure out which sounds are in the "Enhanced" category.
• The sound design has gone through numerous iterations by the time it is in testing, so the sound design team is past the point of soliciting general feedback at this time, but if a QA tester encounters something with the sound design that truly strikes them as misleading, frustrating, distracting, or otherwise problematic, it is okay to log an issue.

[Ashton-Morris]

I have completed the sound design document @emily-phet can you take a look?

• Ratio and Proportion Sound Design Document

[zepumph]

@Ashton-Morris had a great idea of how to embed sounds into the sound design document. We were able to add this guide in https://stackoverflow.com/questions/26897862/playing-sound-with-google-script to the above linked design document. It now has a side bar that can have arbitrary HTML (like sound) written to it. This could be a good way to add a "sound board" of sorts to the design doc.

[Ashton-Morris]

After spending some time looking into it further I don't think this is the solution for in document audio previewing. It works but it should be immediately accessible. Currently you have to choose add-ons, then allow permissions, then sign in with your google id, then run the script.

I also looked into publishing a pdf or spreadsheet with sound and that did not work either. I'll have this idea on the back burner and see if any new options come up in the future.

[emily-phet]

@Ashton-Morris Agreed!

I anticipate there will be some of @jbphet's time available toward the end of this quarter, that he could use for creating the version of sound board we discussed on Tuesday available through phettest. We'll talk more about how that could fit into this quarter tomorrow (thursday) in our planning meeting.

[zepumph]

Once the sound design is approved, mark the issue as ready-for-testing and have it tested with the other
aspects of the simulation through the publication process.

This will be done outside of this issue.

If the sound design document is in good shape, then this issue can be closed. @Ashton-Morris please create a new issue if you want to track discussion around providing sounds in design documents.