Instrument the game screen for PhET-iO

[samreid]

See also #107 and phetsims/chipper#567 (comment)

@kathy-phet said:

Perhaps goal of having Build an Atom game instrumented by end of summer -- so we can figure out whether PhET-iO is working for games by then?

The game screen is mostly instrumented using the traditional approach, but part of this process will be checking to make sure that things work as desired and to see if there are new features required for this support. Having @jbphet involved in this step would be great since he has the most knowledge of the game implementation.

Tagging @ariel-phet and @kathy-phet for priority and @jbphet to put it on his radar.

[ariel-phet]

@samreid @jbphet I think priority should wait until expression exchange is published (or at least in the RC process)

[zepumph]

@jbphet I'm happy to be a part of this as well if needed, just let me know. I'm always excited for some uncharted terrain!!!

[samreid]

I added a wrapper that exercises some PhET-iO api features. We could still investigate adding buttons and controls that let you choose how many problems and which level type (level type is currently hard-coded).

[samreid]

I tagged @zepumph because it will probably be good to collaborate on parts of this.

We probably also need to hide the "Start Over" button (if this becomes a real example of how to deal with games).

[samreid]

One thing that may be different for games is that many tandems are random or delayed (not available on startup)--this won't cause problems for the data stream but it could make customization and generating documentation tricky.

[zepumph]

One thing that may be different for games is that many tandems are random or delayed (not available on startup)--this won't cause problems for the data stream but it could make customization and generating documentation tricky.

Agreed. I feel like this has a potential to be a problem for other places in sims as well, like if there is a default array of dynamic particles that aren't created until you click a button or something.

I think the best way to proceed here is to dive in and see the shortcomings first hand. That will give us a better idea on how to fix them. I think that a poorly documented, but fully instrumented BAA Game may be the first goal. I'm free through Thursday and time around meetings.

[samreid]

What we have been doing for PhET-iO is "instrumenting what is easy and seeing what that makes possible." @pixelzoom recommended the opposite approach of "deciding what behavior we would like to enable, then implementing that." (I am paraphrasing). @kathy-phet I am presuming that for games we will want to be able to:

1. Collect a data stream
2. Start a single level or single challenge, and see what the student did and what the right answer was.
3. Allow customization--timer on/off, sound on/off, get rid of the "start over" button

Can we have a design meeting to see what else we want to tackle with a PhET-iO Build an Atom Game? Do we want to make it possible to create game level selection screens with a subset of levels available? Or be able to set a specific challenge (rather than a specific type of challenge)?

[samreid]

At yesterday's status meeting, @kathy-phet said @arouinfar can schedule a design meeting for this on Thursday August 24.

[samreid]

@arouinfar do you think it will be possible to schedule this meeting for 1pm-2pm Thursday August 24? More details in slack.

[arouinfar]

@arouinfar do you think it will be possible to schedule this meeting for 1pm-2pm Thursday August 24? More details in slack.

Sure thing @samreid. I've set a reminder to schedule Build and Atom PhET-iO discussion for 1-2 at the 8/24 design meeting.

[samreid]

I'm a bit confused between setAllowedProblemTypesByLevel, startGameLevel and it seems there is an implicit correspondence between level type and level index which would be good to discuss with @jbphet once we start on this.

[samreid]

Here are the requested features from today's meeting: we should try to order these from easy to difficult and try to quantify (1) = less than an hour and straightforward (10) = I'm not even sure that's possible. We can also quantify the value. So we'll update/edit this issue. Then starting next week (when @zepumph is back), @jbphet will dedicate 1/2 day per week to collaborate on this.

Timer button should be hidable
Sound button should be hidable
Timer on should be settable
Sound on should be settable
Make it possible to show a subset of level types on the choose your game screen.
Choose the type of challenge with a random challenge instance
Choose an exact challenge instance
(On the far end of instrumentation)
Make it possible to change the number of challenges anywhere between 1 and some max
Get rid of the start over button
instead of 2 tries, make it possible to have only 1 try or unlimited tries
Make it possible to hide the score
Make it possible to hide the entire scoreboard
Update the number of stars based on the number of challenges
Control random seed so all students get same challenges
Kathy’s “start with these” list:
being able to choose which levels show on the game homescreen, and which are hidden. Timer and sound
Then being able to manipulate scoreboard and get rid of score and start over.
Changing the number of challenges in a level
(And changing the type of challenge within each level)
Kathy: It may increase the complexity significantly to be able to specify exact challenges, we should investigate how difficult it would be and go for it if not too complex
Make the phet button unclickable
Make sure it shows a data stream what the user did even before they pressed submit
assessment mode: the student doesn’t know if they got it right or wrong
priority: after CCK
It’s important in terms of understanding how this impacts the overall approach to phet-io, will we use TTypes, etc.
Kathy recommends 1/2 day for JB starting when MK returns

[samreid]

On second thought, let's leave the preceding comment as "raw notes" then use this next comment to organize everything in a table:

[samreid]

Feature	Value	Complexity	Complete
Timer button should be hidable			✓
Sound button should be hidable			✓
Timer on should be settable			✓
Sound on should be settable			✓
Make it possible to show a subset of level types on the choose your game screen.			✓ but layout doesn't reflow
Get rid of the start over button			✓
Make it possible to hide the score			✓
Make it possible to hide the entire scoreboard			✓
being able to choose which levels show on the game homescreen, and which are hidden. Timer and sound			✓
Then being able to manipulate scoreboard and get rid of score and start over.			✓
Make the phet button unclickable			✓
Make it possible to change the number of challenges anywhere between 1 and some max (or unlimited)			✓via query string
Choose the type of challenges with a level			✓ via api call
Choose an exact challenge instance @jbphet			✓
Make sure it shows a data stream what the user did even before they pressed submit			✓
assessment mode: the student doesn’t know if they got it right or wrong @jbphet			✓ (see questions)
Update the number of stars based on the number of challenges OR allow to be separately configured via query parameter			✓
Control random seed so all students get same challenges			✓ via query parameter or API call, and also possible to get seed when it was random
Better tandem names for periodic table buttons? (see PeriodicTableNode.js)			✓
Dealing with random/dynamic tandems in the API (for customization,data stream, API, API diff) see https://github.com/phetsims/phet-io/issues/1201
Add full support for saving/restoring state

[jbphet]

Today we resolved some issues related to the reward node that is shown when a level is completed. Next time we should work on making the stars update correctly in the level selection button (see #179) and then work on the problem where a new level can't be started once a level is completed.

[samreid]

It's been great to have @jbphet involved in this process, I'll unassign until we are collaborating on this again.

[jbphet]

Work seems to have ground to a halt on this effort, and there are some problems occurring due to the partially implemented code (see #185). I'm going to reassign this to @samreid since he is the PhET-iO tech lead, and he can work with the powers that be to determine if and when work on this should resume.

[samreid]

@ariel-phet can you comment on proposed timeline for PhET-iO API design and implementation for this sim?

[ariel-phet]

@samreid I honestly cannot comment, and I do not know timeline. marking the issue as deferred for the moment until @kathy-phet decides it is a priority. My suggestion would be to revisit once @jbphet has published EFAC. Assigning you to be aware of that suggestion, feel free to clear assignees or get @jbphet to make a calendar reminder or such.

[samreid]

@jbphet how do you feel about a calendar reminder for checking on this after EFAC? We would start with a PhET-iO design process, deciding on the PhET-iO specific features as a new first step.

[jbphet]

I've added a calendar reminder for next Jan, since that's when EFAC is supposed to be out. Unassigning for now.

[zepumph]

The game screen does not (still) support state, and is not set up to be maintained for phet-io yet. As @samreid and I work on updating phet-io tests, we decided to remove that screen from the PhET-iO sim above.

[zepumph]

From #156 (comment)

being able to choose which levels show on the game homescreen, and which are hidden. Timer and sound

This was the request from a user over in https://github.com/phetsims/build-an-atom/issues/222

[zepumph]

Presumably we decided that this was covered by PhET-iO because you can hide each button that was desired (with the visibleProperty). But this isn't supported via QP right now, all 4 are always created:

build-an-atom/js/game/view/StartGameLevelNode.js

Lines 51 to 80 in fd8338e

	// buttons for starting a game level
	const periodicTableGameButton = createLevelSelectionButton(
	gameModel,
	periodicTableIcon,
	'periodic-table-game',
	'periodicTableGame',
	tandem
	);
	const massAndChargeGameButton = createLevelSelectionButton(
	gameModel,
	massChargeIcon,
	'mass-and-charge-game',
	'massAndChargeGame',
	tandem
	);
	const symbolGameButton = createLevelSelectionButton(
	gameModel,
	symbolQuestionIcon,
	'symbol-game',
	'symbolGame',
	tandem
	);
	const advancedSymbolGameButton = createLevelSelectionButton(
	gameModel,
	questionMarkIcon,
	'advanced-symbol-game',
	'advancedSymbolGame',
	tandem
	);