Introduce UI tests for hints & solutions #1273
Labels
bug
End user-perceivable behaviors which are not desirable.
Impact: Medium
Moderate perceived user impact (non-blocking bugs and general improvements).
Work: Medium
The means to find the solution is clear, but it isn't at good-first-issue level yet.
Z-ibt
Temporary label for Ben to keep track of issues he's triaged.
Milestone
Comments
BenHenning
added
Type: Improvement
Priority: Essential
BenHenning
added a commit
that referenced
this issue
Jun 9, 2020
behavior graph (and added an actual visual representation of this graph in code to simplify maintaining the solution in the future). Tests will be added as part of solving #1273.
This was referenced Jun 9, 2020
BenHenning
added a commit
that referenced
this issue
Jun 10, 2020
* Initial check in for the question data controller * Removed progress controller test * Address the renames suggested in #217. * Introduce thoroughly stubbed QuestionAssessmentProgressController + empty test suite. * Review changes * Review changes * Pass data providers to the assessment controller, and set caps in the training controller * Removed unnecessary imports/variables * Add fake question data for stubbed interfaces * Remove duplicate questions while fetching in training controller * Comment explaining the filter function * Improve duplicate checking - check it while filtering instead of after filtering * Add linked skill id values * Review changes * add a new module for questions constants * Review changes * add a test to verify questions were fetched properly * reformatted code * Re-add QuestionTrainingController removed in merge. * Finalize question progress controller interface. * Implement QuestionAssessmentProgressController. This includes some basic refactoring of internal structures used by the exploration progress controller to share common functionality between the two progress controllers. There's still some duplication, but this seems like a reasonable split since there's likely to be further differences in the progress controllers in the future. The question assessment progress controller tests pass, but no new ones have yet been added to thoroughly test the implementation. * Add initial phase of tests for the progress controller based on ExplorationProgressController. They haven't yet been verified as correct. * Fix typo in QuestionTrainingController. * Post-merge fixes and adjustments. * Fix broken ProfileManagementControllerTest. * Post-merge fixes. * Update tests to build, but not pass. * Started UI * setup ViewModel * added binding adapter * Handles basic functionality * Moved code around * Added replay button * Make questions partially work. This introduces back button behavior to ensure training sessions can be reset. This also attempts to introduce proper asset loading, but it doesn't seem to be working currently. * Temporarily remove check in StateDeck that breaks questions. * Ensure all state navigation buttons are on the same line and the same size. This required reworking how nav button binding works in the state fragment such that multiple models are used depending on which type of navigation scenario the fragment is currently in. * Fix dp adjustment that caused previous button to disappear. Fix previous button showing up twice for continue interaction. * Undo .idea/misc.xml change that snuck in. * Update & fix StateFragmentTest (at least for Espresso). See #495 description for details on the changes necessary to make these tests work. * Post-merge fixes with some temporary stop-gap solutions. * Fix questions image rendering and consolidate GCS resource bucket names. * Fix question session not properly being reinitialized for new sessions. Also, ensure that notifications do not regenerate the training session. * Fix index accounting for questions. Before, the index incremented upon correct answer submission rather than navigation. * Remove support for backward navigation in questions since it isn't allowed. * Set title of question activity to 'Train Mode'. * Add support for a synthetic ephemeral question to represent the end of a training session. * Update question progress tracking to be translatable and to ensure that the synthetic terminal state shows the correct progress: 'completed'. * Move most state player functional support into a new configurable assembler class to enable sharing between the exploration and question players. * Hook up StatePlayerRecyclerViewAssembler to the question player fragment presenter. This also enables the congratulations text. * Introduce proper terminal page and disable overscroll. Note that the end session page introduced here has not been mocked yet, so it's mostly a placeholder until a mock is available. * Add support & button for training session replay. * Update/add TODOs to correspond to GitHub tracked issues. * Fix broken import. * Finish tests for Question{Training,AssessmentProgress}Controllers. Note that the nature of the assessment progress controller is such that a new, dynamic DataProvider is needed since the data provider for the current assessment question is actually a transformation of different data providers depending on which questions are passed to the controller. * Initial introduction of test coroutine dispatchers to replace Kotlin's test coroutine dispatcher. This includes introducing a test-only module to contain testing dependencies. * Introduce a new LiveData mechanism to bridge coroutines from DataProviders in a way that doesn't have the same limitations as the previous MutableLiveData-based bridge. * Introduce new nested data provider that effectively allows a DataProvider to be set up like transformAsync() but with the ability to change the root DataProvider. * question player progress-bar hifi * nit * revert * revert * Early work at introducing FakeSystemClock tests (not yet complete). * Post-merge fixes. * Fix the submit button disabled states: with the new refactor, the submit button is now properly disabled when an invalid answer is pending. * Revert accidental changes to .idea/misc.xml. * Address reviewer comments. * Remove unnecessary meta file for Mockito. * Minor import cleanup. * Post-merge fixes. * Fix audio playback & generalize it to facilitate enabling it for questions in the future. * Address reviewer comments & fix broken tests. * Fix hints & solutions functionality after simplification (some bugs & regressions were introduced due to the simplification; these have been fixed). * Revert the assembler delays to match the intended durations rather than the test-only values used for development. * Address reviewer comments. * Remove infeasible testing structures, add documentation, and clean up implementation to prepare for code review. * Add notice that the dispatchers utility is temporary. * Cleanup new LiveData bridge, add tests for it, and migrate other DataProviders tests to using TestCoroutineDispatchers utility. * Add AsyncResult tests, fix & re-enable an earlier test in PersistentCacheStoreTest, and fix FakeSystemClock so that it works properly in test environments. * Use ktlint to reformat TestCoroutineDispatchers per reviewer comment thread. * Reformat files failing linter check. * Reformat new DataProviders code. * Add new tests for the NestedTransformedDataProvider. * Address reviewer comment. * Address reviewer comments. * Address reviewer comments & fix lint issue. * Update hints & solutions handling to more closely follow the intended behavior graph (and added an actual visual representation of this graph in code to simplify maintaining the solution in the future). Tests will be added as part of solving #1273. Co-authored-by: vinitamurthi <[email protected]> Co-authored-by: James Xu <[email protected]> Co-authored-by: marysoloman <[email protected]>
nikitamarysolomanpvt
added a commit
that referenced
this issue
Jun 12, 2020
* Initial check in for the question data controller * Removed progress controller test * Address the renames suggested in #217. * Introduce thoroughly stubbed QuestionAssessmentProgressController + empty test suite. * Review changes * Review changes * Pass data providers to the assessment controller, and set caps in the training controller * Removed unnecessary imports/variables * Add fake question data for stubbed interfaces * Remove duplicate questions while fetching in training controller * Comment explaining the filter function * Improve duplicate checking - check it while filtering instead of after filtering * Add linked skill id values * Review changes * add a new module for questions constants * Review changes * add a test to verify questions were fetched properly * reformatted code * Re-add QuestionTrainingController removed in merge. * Finalize question progress controller interface. * Implement QuestionAssessmentProgressController. This includes some basic refactoring of internal structures used by the exploration progress controller to share common functionality between the two progress controllers. There's still some duplication, but this seems like a reasonable split since there's likely to be further differences in the progress controllers in the future. The question assessment progress controller tests pass, but no new ones have yet been added to thoroughly test the implementation. * Add initial phase of tests for the progress controller based on ExplorationProgressController. They haven't yet been verified as correct. * Fix typo in QuestionTrainingController. * Post-merge fixes and adjustments. * Fix broken ProfileManagementControllerTest. * Post-merge fixes. * Update tests to build, but not pass. * Started UI * setup ViewModel * added binding adapter * Handles basic functionality * Moved code around * Added replay button * Make questions partially work. This introduces back button behavior to ensure training sessions can be reset. This also attempts to introduce proper asset loading, but it doesn't seem to be working currently. * Temporarily remove check in StateDeck that breaks questions. * Ensure all state navigation buttons are on the same line and the same size. This required reworking how nav button binding works in the state fragment such that multiple models are used depending on which type of navigation scenario the fragment is currently in. * Fix dp adjustment that caused previous button to disappear. Fix previous button showing up twice for continue interaction. * Undo .idea/misc.xml change that snuck in. * Update & fix StateFragmentTest (at least for Espresso). See #495 description for details on the changes necessary to make these tests work. * Post-merge fixes with some temporary stop-gap solutions. * Fix questions image rendering and consolidate GCS resource bucket names. * Fix question session not properly being reinitialized for new sessions. Also, ensure that notifications do not regenerate the training session. * Fix index accounting for questions. Before, the index incremented upon correct answer submission rather than navigation. * Remove support for backward navigation in questions since it isn't allowed. * Set title of question activity to 'Train Mode'. * Add support for a synthetic ephemeral question to represent the end of a training session. * Update question progress tracking to be translatable and to ensure that the synthetic terminal state shows the correct progress: 'completed'. * Move most state player functional support into a new configurable assembler class to enable sharing between the exploration and question players. * Hook up StatePlayerRecyclerViewAssembler to the question player fragment presenter. This also enables the congratulations text. * Introduce proper terminal page and disable overscroll. Note that the end session page introduced here has not been mocked yet, so it's mostly a placeholder until a mock is available. * Add support & button for training session replay. * Update/add TODOs to correspond to GitHub tracked issues. * Fix broken import. * Finish tests for Question{Training,AssessmentProgress}Controllers. Note that the nature of the assessment progress controller is such that a new, dynamic DataProvider is needed since the data provider for the current assessment question is actually a transformation of different data providers depending on which questions are passed to the controller. * Initial introduction of test coroutine dispatchers to replace Kotlin's test coroutine dispatcher. This includes introducing a test-only module to contain testing dependencies. * Introduce a new LiveData mechanism to bridge coroutines from DataProviders in a way that doesn't have the same limitations as the previous MutableLiveData-based bridge. * Introduce new nested data provider that effectively allows a DataProvider to be set up like transformAsync() but with the ability to change the root DataProvider. * question player toolbar shadow * question player progress-bar hifi * nit * revert * revert * Early work at introducing FakeSystemClock tests (not yet complete). * Post-merge fixes. * Fix the submit button disabled states: with the new refactor, the submit button is now properly disabled when an invalid answer is pending. * Revert accidental changes to .idea/misc.xml. * Address reviewer comments. * Remove unnecessary meta file for Mockito. * Minor import cleanup. * Post-merge fixes. * Fix audio playback & generalize it to facilitate enabling it for questions in the future. * Address reviewer comments & fix broken tests. * Fix hints & solutions functionality after simplification (some bugs & regressions were introduced due to the simplification; these have been fixed). * Revert the assembler delays to match the intended durations rather than the test-only values used for development. * Address reviewer comments. * Remove infeasible testing structures, add documentation, and clean up implementation to prepare for code review. * Add notice that the dispatchers utility is temporary. * Cleanup new LiveData bridge, add tests for it, and migrate other DataProviders tests to using TestCoroutineDispatchers utility. * Add AsyncResult tests, fix & re-enable an earlier test in PersistentCacheStoreTest, and fix FakeSystemClock so that it works properly in test environments. * Use ktlint to reformat TestCoroutineDispatchers per reviewer comment thread. * Reformat files failing linter check. * Reformat new DataProviders code. * Add new tests for the NestedTransformedDataProvider. * Address reviewer comment. * Address reviewer comments. * Address reviewer comments & fix lint issue. * Update hints & solutions handling to more closely follow the intended behavior graph (and added an actual visual representation of this graph in code to simplify maintaining the solution in the future). Tests will be added as part of solving #1273. Co-authored-by: vinitamurthi <[email protected]> Co-authored-by: Ben Henning <[email protected]> Co-authored-by: James Xu <[email protected]>
nikitamarysolomanpvt
added a commit
that referenced
this issue
Jun 16, 2020
* Initial check in for the question data controller * Removed progress controller test * Address the renames suggested in #217. * Introduce thoroughly stubbed QuestionAssessmentProgressController + empty test suite. * Review changes * Review changes * Pass data providers to the assessment controller, and set caps in the training controller * Removed unnecessary imports/variables * Add fake question data for stubbed interfaces * Remove duplicate questions while fetching in training controller * Comment explaining the filter function * Improve duplicate checking - check it while filtering instead of after filtering * Add linked skill id values * Review changes * add a new module for questions constants * Review changes * add a test to verify questions were fetched properly * reformatted code * Re-add QuestionTrainingController removed in merge. * Finalize question progress controller interface. * Implement QuestionAssessmentProgressController. This includes some basic refactoring of internal structures used by the exploration progress controller to share common functionality between the two progress controllers. There's still some duplication, but this seems like a reasonable split since there's likely to be further differences in the progress controllers in the future. The question assessment progress controller tests pass, but no new ones have yet been added to thoroughly test the implementation. * Add initial phase of tests for the progress controller based on ExplorationProgressController. They haven't yet been verified as correct. * Fix typo in QuestionTrainingController. * Post-merge fixes and adjustments. * Fix broken ProfileManagementControllerTest. * Post-merge fixes. * Update tests to build, but not pass. * Started UI * setup ViewModel * added binding adapter * Handles basic functionality * Moved code around * Added replay button * Make questions partially work. This introduces back button behavior to ensure training sessions can be reset. This also attempts to introduce proper asset loading, but it doesn't seem to be working currently. * Temporarily remove check in StateDeck that breaks questions. * Ensure all state navigation buttons are on the same line and the same size. This required reworking how nav button binding works in the state fragment such that multiple models are used depending on which type of navigation scenario the fragment is currently in. * Fix dp adjustment that caused previous button to disappear. Fix previous button showing up twice for continue interaction. * Undo .idea/misc.xml change that snuck in. * Update & fix StateFragmentTest (at least for Espresso). See #495 description for details on the changes necessary to make these tests work. * Post-merge fixes with some temporary stop-gap solutions. * Fix questions image rendering and consolidate GCS resource bucket names. * Fix question session not properly being reinitialized for new sessions. Also, ensure that notifications do not regenerate the training session. * Fix index accounting for questions. Before, the index incremented upon correct answer submission rather than navigation. * Remove support for backward navigation in questions since it isn't allowed. * Set title of question activity to 'Train Mode'. * Add support for a synthetic ephemeral question to represent the end of a training session. * Update question progress tracking to be translatable and to ensure that the synthetic terminal state shows the correct progress: 'completed'. * Move most state player functional support into a new configurable assembler class to enable sharing between the exploration and question players. * Hook up StatePlayerRecyclerViewAssembler to the question player fragment presenter. This also enables the congratulations text. * Introduce proper terminal page and disable overscroll. Note that the end session page introduced here has not been mocked yet, so it's mostly a placeholder until a mock is available. * Add support & button for training session replay. * Update/add TODOs to correspond to GitHub tracked issues. * Fix broken import. * Finish tests for Question{Training,AssessmentProgress}Controllers. Note that the nature of the assessment progress controller is such that a new, dynamic DataProvider is needed since the data provider for the current assessment question is actually a transformation of different data providers depending on which questions are passed to the controller. * Initial introduction of test coroutine dispatchers to replace Kotlin's test coroutine dispatcher. This includes introducing a test-only module to contain testing dependencies. * Introduce a new LiveData mechanism to bridge coroutines from DataProviders in a way that doesn't have the same limitations as the previous MutableLiveData-based bridge. * Introduce new nested data provider that effectively allows a DataProvider to be set up like transformAsync() but with the ability to change the root DataProvider. * question player progress-bar hifi * nit * revert * revert * Early work at introducing FakeSystemClock tests (not yet complete). * hifi lanscape question player * Post-merge fixes. * Fix the submit button disabled states: with the new refactor, the submit button is now properly disabled when an invalid answer is pending. * Revert accidental changes to .idea/misc.xml. * Address reviewer comments. * Remove unnecessary meta file for Mockito. * Minor import cleanup. * Post-merge fixes. * Fix audio playback & generalize it to facilitate enabling it for questions in the future. * Address reviewer comments & fix broken tests. * Fix hints & solutions functionality after simplification (some bugs & regressions were introduced due to the simplification; these have been fixed). * Revert the assembler delays to match the intended durations rather than the test-only values used for development. * Address reviewer comments. * Remove infeasible testing structures, add documentation, and clean up implementation to prepare for code review. * Add notice that the dispatchers utility is temporary. * Cleanup new LiveData bridge, add tests for it, and migrate other DataProviders tests to using TestCoroutineDispatchers utility. * Add AsyncResult tests, fix & re-enable an earlier test in PersistentCacheStoreTest, and fix FakeSystemClock so that it works properly in test environments. * Use ktlint to reformat TestCoroutineDispatchers per reviewer comment thread. * Reformat files failing linter check. * Reformat new DataProviders code. * Add new tests for the NestedTransformedDataProvider. * Address reviewer comment. * Address reviewer comments. * Address reviewer comments & fix lint issue. * Update hints & solutions handling to more closely follow the intended behavior graph (and added an actual visual representation of this graph in code to simplify maintaining the solution in the future). Tests will be added as part of solving #1273. * nit * nit Co-authored-by: vinitamurthi <[email protected]> Co-authored-by: Ben Henning <[email protected]> Co-authored-by: James Xu <[email protected]>
BenHenning
added a commit
that referenced
this issue
Jun 24, 2020
* Update hints & solutions handling to more closely follow the intended behavior graph (and added an actual visual representation of this graph in code to simplify maintaining the solution in the future). Tests will be added as part of solving #1273. * Add support for sharing the test application component between UI dependencies and tests. Also, fix the test coroutine dispatcher to properly manage time in tests. This enables the test coroutine dispatcher for app module tests. * Clean up the documentation for test utilities. * Address reviewer comment.
BenHenning
added a commit
that referenced
this issue
Jul 17, 2020
* Initial check in for the question data controller * Removed progress controller test * Address the renames suggested in #217. * Introduce thoroughly stubbed QuestionAssessmentProgressController + empty test suite. * Review changes * Review changes * Pass data providers to the assessment controller, and set caps in the training controller * Removed unnecessary imports/variables * Add fake question data for stubbed interfaces * Remove duplicate questions while fetching in training controller * Comment explaining the filter function * Improve duplicate checking - check it while filtering instead of after filtering * Add linked skill id values * Review changes * add a new module for questions constants * Review changes * add a test to verify questions were fetched properly * reformatted code * Re-add QuestionTrainingController removed in merge. * Finalize question progress controller interface. * Implement QuestionAssessmentProgressController. This includes some basic refactoring of internal structures used by the exploration progress controller to share common functionality between the two progress controllers. There's still some duplication, but this seems like a reasonable split since there's likely to be further differences in the progress controllers in the future. The question assessment progress controller tests pass, but no new ones have yet been added to thoroughly test the implementation. * Add initial phase of tests for the progress controller based on ExplorationProgressController. They haven't yet been verified as correct. * Fix typo in QuestionTrainingController. * Post-merge fixes and adjustments. * Fix broken ProfileManagementControllerTest. * Post-merge fixes. * Update tests to build, but not pass. * Started UI * setup ViewModel * added binding adapter * Handles basic functionality * Moved code around * Added replay button * Make questions partially work. This introduces back button behavior to ensure training sessions can be reset. This also attempts to introduce proper asset loading, but it doesn't seem to be working currently. * Temporarily remove check in StateDeck that breaks questions. * Ensure all state navigation buttons are on the same line and the same size. This required reworking how nav button binding works in the state fragment such that multiple models are used depending on which type of navigation scenario the fragment is currently in. * Fix dp adjustment that caused previous button to disappear. Fix previous button showing up twice for continue interaction. * Undo .idea/misc.xml change that snuck in. * Update & fix StateFragmentTest (at least for Espresso). See #495 description for details on the changes necessary to make these tests work. * Post-merge fixes with some temporary stop-gap solutions. * Fix questions image rendering and consolidate GCS resource bucket names. * Fix question session not properly being reinitialized for new sessions. Also, ensure that notifications do not regenerate the training session. * Fix index accounting for questions. Before, the index incremented upon correct answer submission rather than navigation. * Remove support for backward navigation in questions since it isn't allowed. * Set title of question activity to 'Train Mode'. * Add support for a synthetic ephemeral question to represent the end of a training session. * Update question progress tracking to be translatable and to ensure that the synthetic terminal state shows the correct progress: 'completed'. * Move most state player functional support into a new configurable assembler class to enable sharing between the exploration and question players. * Hook up StatePlayerRecyclerViewAssembler to the question player fragment presenter. This also enables the congratulations text. * Introduce proper terminal page and disable overscroll. Note that the end session page introduced here has not been mocked yet, so it's mostly a placeholder until a mock is available. * Add support & button for training session replay. * Update/add TODOs to correspond to GitHub tracked issues. * Fix broken import. * Finish tests for Question{Training,AssessmentProgress}Controllers. Note that the nature of the assessment progress controller is such that a new, dynamic DataProvider is needed since the data provider for the current assessment question is actually a transformation of different data providers depending on which questions are passed to the controller. * Initial introduction of test coroutine dispatchers to replace Kotlin's test coroutine dispatcher. This includes introducing a test-only module to contain testing dependencies. * Introduce a new LiveData mechanism to bridge coroutines from DataProviders in a way that doesn't have the same limitations as the previous MutableLiveData-based bridge. * Introduce new nested data provider that effectively allows a DataProvider to be set up like transformAsync() but with the ability to change the root DataProvider. * question player progress-bar hifi * nit * revert * revert * Early work at introducing FakeSystemClock tests (not yet complete). * Post-merge fixes. * Fix the submit button disabled states: with the new refactor, the submit button is now properly disabled when an invalid answer is pending. * Revert accidental changes to .idea/misc.xml. * Address reviewer comments. * Remove unnecessary meta file for Mockito. * Minor import cleanup. * Post-merge fixes. * Fix audio playback & generalize it to facilitate enabling it for questions in the future. * Address reviewer comments & fix broken tests. * Fix hints & solutions functionality after simplification (some bugs & regressions were introduced due to the simplification; these have been fixed). * Revert the assembler delays to match the intended durations rather than the test-only values used for development. * Address reviewer comments. * Remove infeasible testing structures, add documentation, and clean up implementation to prepare for code review. * Add notice that the dispatchers utility is temporary. * Cleanup new LiveData bridge, add tests for it, and migrate other DataProviders tests to using TestCoroutineDispatchers utility. * Add AsyncResult tests, fix & re-enable an earlier test in PersistentCacheStoreTest, and fix FakeSystemClock so that it works properly in test environments. * Use ktlint to reformat TestCoroutineDispatchers per reviewer comment thread. * Reformat files failing linter check. * Reformat new DataProviders code. * Add new tests for the NestedTransformedDataProvider. * Address reviewer comment. * Address reviewer comments. * Address reviewer comments & fix lint issue. * Update hints & solutions handling to more closely follow the intended behavior graph (and added an actual visual representation of this graph in code to simplify maintaining the solution in the future). Tests will be added as part of solving #1273. * Add support for sharing the test application component between UI dependencies and tests. Also, fix the test coroutine dispatcher to properly manage time in tests. This enables the test coroutine dispatcher for app module tests. * Clean up the documentation for test utilities. * Add tests for hints & solutions. These are Robolectric-only tests since we need to carefully coordinate timing and very quickly drive through a bunch of different UI screens. * Address reviewer comment. * Address reviewer comments & fix broken tests after merging in develop. * Post-merge fixes. * Fix linter errors.
|
Not completed since this still needs to be done in QuestionPlayerActivityTest. |
24 tasks
Broppia
added
issue_type_infrastructure
Impact: Medium
BenHenning
added
Impact: Low
BenHenning
moved this to Needs Triage
in [Team] Core Learner and Mastery flows & UI Frontend - Android
seanlip
moved this from Needs Triage
to Todo
in [Team] Core Learner and Mastery flows & UI Frontend - Android
seanlip
added
bug
adhiamboperes
added
Impact: Medium
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hints & solutions logic was largely updated in #500, but an unanticipated regression caused the PR to be delayed by about a week. This would normally be caught via a test. This issue is tracking introducing thorough tests for hints & solutions to ensure such situations can be avoided in the future.
The text was updated successfully, but these errors were encountered: