Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RC test: Natural Selection 1.4.0-rc.1 #662

Closed
pixelzoom opened this issue Jun 22, 2021 · 10 comments
Closed

RC test: Natural Selection 1.4.0-rc.1 #662

pixelzoom opened this issue Jun 22, 2021 · 10 comments

Comments

@pixelzoom
Copy link
Contributor

pixelzoom commented Jun 22, 2021

@KatieWoe, @kathy-phet, @amanda-phet, natural-selection/1.4.0-rc.1 is ready for RC testing. Document issues in https://github.com/phetsims/natural-selection/issues and link to this issue.

Assigning to @kathy-phet and @KatieWoe for prioritization.

General RC Test

What to Test

  • Click every single button.
  • Test all possible forms of input.
    • Test all mouse/trackpad inputs.
    • Test all touchscreen inputs.
  • If there is sound, make sure it works.
  • Make sure you can't lose anything.
  • Play with the sim normally.
  • Try to break the sim.
  • Test all query parameters on all platforms. (See QA Book for a list of query parameters.)
  • Download HTML on Chrome and iOS.
  • Make sure the iFrame version of the simulation is working as intended on all platforms.
  • Make sure the XHTML version of the simulation is working as intended on all platforms.
  • Complete the test matrix.
  • Don't forget to make sure the sim works with Legends of Learning.
  • Test the Game Up harness on at least one platform.
  • Check this LoL spreadsheet and notify AR or AM if it not there.
  • If this is rc.1 please do a memory test.
  • When making an issue, check to see if it was in a previously published version
  • Try to include version numbers for browsers
  • If there is a console available, check for errors and include them in the Problem Description.
  • As an RC begins and ends, check the sim repo. If there is a maintenance issue, check it and notify developers if
    there is a problem.

Focus and Special Instructions

There were no known changes to sim-specific code, so any regressions would be in common code. This release cycle is primarily for PhET-iO changes, so see that section for more info.

Link(s)


PhET-iO RC Test

What to Test

  • Make sure that public files do not have password protection. Use a private browser for this.
  • Make sure that private files do have password protection. Use a private browser for this.
  • Make sure standalone sim is working properly.
  • Make sure the wrapper index is working properly.
  • Make sure each wrapper is working properly.
  • Launch the simulation in Studio with ?stringTest=xss and make sure the sim doesn't navigate to youtube
  • For newer PhET-iO wrapper indices, save the "basic example of a functional wrapper" as a .html file and open it. Make
    sure the simulation loads without crashing or throwing errors.
  • For an update or maintenance release please check the backwards compatibility of the playback wrapper.
    Here's the link to the previous wrapper.
  • Load the login wrapper just to make sure it works. Do so by adding this link from the sim deployed root:
    /wrappers/login/?wrapper=record&validationRule=validateDigits&&numberOfDigits=5&promptText=ENTER_A_5_DIGIT_NUMBER
    
    Further instructions in QA Book
  • Conduct a recording test to Metacog, further instructions in the QA Book. Do this for iPadOS + Safari and one other random platform.
  • Conduct a memory test on the stand alone sim wrapper (rc.1).
  • Test one platform combination with ?phetioDebug on the Studio and State wrapper.
  • If Pan/Zoom is supported, make sure that it works when set with PhET-iO State.
  • If filled in by issue creator, please test the diff wrapper against https://phet-io.colorado.edu/sims/natural-selection/1.3/
  • Test that the sim works offline:

Focus and Special Instructions

This release is primarily focused on recent changes to PhET-iO, particularly Studio, save/load, and API files. If you need info on what (or how) to focus on those aspects, please consult with @samreid or @kathy-phet.

Link(s)


My question isn't in here... What should I do?

You should:

  1. Consult the QA Book.
  2. Google it.
  3. Ask Katie.
  4. Ask a developer.
  5. Google it again.
  6. Cry.


@KatieWoe
Copy link
Contributor

PhET Brand Memory Test:
phetbrandmem
PhET-iO Memory Test:
iomem

@KatieWoe
Copy link
Contributor

KatieWoe commented Jun 29, 2021

iPadOS 13.7 Safari, the sim crashed and reloaded fairly quickly (phet-brand). The device hadn't been used in a while and restarting it seems to have improved the issue, so I haven't made an issue. @pixelzoom let me know if you would like one anyway. I'll also make an issue if the problem reoccurs.

@pixelzoom
Copy link
Contributor Author

... . @pixelzoom let me know if you would like one anyway. ...

No need unless the problem reoccurs. But do please attempt to reproduce with that device.

@KatieWoe
Copy link
Contributor

Performance note: On Lovelace (Slower ChromeOS device) several wrappers seem to slow down significantly when the sim is in use (such as state and recording). Nothing seems broken, so I don't think it needs an issue, but let me know if you want more info.

@KatieWoe
Copy link
Contributor

KatieWoe commented Jul 2, 2021

I saw the same sort of lag in the playback wrapper described in phetsims/molecule-shapes#177 and phetsims/gravity-and-orbits#376, particularly between generations. It seemed more severe than I've seen before, but we've been passing on these types of issues. So let me know if you want me to make an issue anyway.

@pixelzoom
Copy link
Contributor Author

pixelzoom commented Jul 2, 2021

I saw the same sort of lag in the playback wrapper ... So let me know if you want me to make an issue anyway.

Yes, please make an issue. I'm not familiar with what the performance expectations are, or how performance might have changed due to recent PhET-iO changes. So please assign the issue to the PhET-iO team for evaluation.

@KatieWoe
Copy link
Contributor

KatieWoe commented Jul 2, 2021

QA is done

@KatieWoe KatieWoe assigned pixelzoom and unassigned KatieWoe Jul 2, 2021
@pixelzoom
Copy link
Contributor Author

Thanks @KatieWoe and QA team. I'll leave this issue open until all related issues have been addressed. Another RC test (1.4.0-rc.2) will be required, probably to spot check specific issues.

@pixelzoom
Copy link
Contributor Author

pixelzoom commented Jul 13, 2021

There's confusion about which issues are blocking for the next RC. Assigning to @kathy-phet and @amanda-phet to define next steps. Let me know when you're ready for the next RC, and what type of RC test you'd like.

@pixelzoom pixelzoom assigned kathy-phet and amanda-phet and unassigned pixelzoom Jul 13, 2021
@pixelzoom
Copy link
Contributor Author

1.4.0-rc.2 has been published, see #672.

Closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

No branches or pull requests

4 participants