Test plan for 16.7 Beta version of WordPress Android app

[fluiddot]

Due to the recent issue regarding the merged PR into WordPress-Android repository, we have to execute a specific test plan to ensure that it didn't break anything.

This plan will be executed only on Android.

• Writing flow tests - @fluiddot
• Random tests from the sanity check test suites (preferably one test from each section to cover a wide range) @ceyhun
• Smoke test the editor flow in general by doing: @fluiddot
  • Entering exiting the editor
  • Rotate the device in various cases
  • Open and dismiss the bottom sheet
  • Background the app and foreground it again

[ceyhun]

Android Sanity Tests 🟡

WPAndroid 16.7-rc1, Huawei P20 Lite, Android 10

Gallery-1

• Gallery block - Close/Re-open post with an ongoing image upload - steps

MediaText-1

• Media Text block - Insert image from device (failing) - steps

MultipleUpload-1

• Simultaneous uploads - steps

DarkMode-1

• Gallery block - Dark mode - steps

Shortcode-1

• Shortcode block - Add a youtube link - steps

Spacer-1

• Spacer block - Spacer is rendered without crash - steps

Buttons-1

• Buttons block - Buttons block is added with focused Button with gray background color and white text color - steps

Group-1

• Group - AppenderButton is rendered - steps

Cover-1

• Cover - Add image background from WordPress Media library - steps

Columns - 1

• Columns - Columns percentage mechanism - column's preview - steps

Editor Theme - 1

• Default Colors - Check that default colors still load - steps

Color Settings - 1

• Colors Segmented Controls - steps

Social Icons - 1

• The newly created Social Icons block is added with 4 icons - steps

Story block - 1

• Story block - Verify is available in Block Picker - steps
• Story block - Verify is not available in Block Picker- steps
• Story block - Add media to empty Story block - steps 🟡
  • While uploading multiple images, the progress bar at the top keeps disappearing and re-appearing for me. Same issue exists on version 16.6. (internal reference: pbD5rk-mE-p2#comment-452)
• Story block - Add media to non-empty Story block - steps
• Story block - Close/Re-open post with an ongoing Story upload - steps

[fluiddot]

Android: Writing Flow checklist 🟡

Tested on Samsung Galaxy S20 FE 5G, Android 10 - Version 16.7-rc-1

General

• TC001 - Block insertion flow
• TC001 - Paste formatted text copied from website
• TC001 - Multiline components 🟡
  • Text flickering when removing a line on some multiline blocks - Text flickering when removing a line on some multiline blocks #3096
  • Pullquote Block - Wrong cursor position when merging back - Tapping on a Page Break block does make the block-holder selected but keep the keyboard up, and the caret to the Para block #308

Rich Text Format

• TC001 - Bold, Italic, strikethrough buttons
• TC002 - Alignment buttons
• TC003 - Alignment Split
• TC004 - Link button works without selection
• TC005 - Link button works with a selected word
  • Link button - Remove link when the word is selected doesn't remove it - Link button - Remove link when the word is selected doesn't remove it #3134
• TC006 - Adding a link from a copied URL
• TC007 - Test format detection under the cursor
• TC008 - Test formatting doesn't remove leading or trailing whitespace

Splitting and merging

• TC001 - Merge after writing
• TC002 - Merge after selection 🟡
  • Heading block stops being rendered as a header when deleting the first word - Heading block stops being rendered as a header when deleting the first word #3106
  • Splitting paragraph block - Deleting the first word after selecting it requires tapping backspace two times - Splitting paragraph block - Deleting the first word after selecting it requires tapping backspace two times #3135
• TC003 - Merge after deleting text
• TC004 - Merge after deleting all
• TC005 - Merge multiple blocks
• TC006 - Splitting/merge list block

Undo / Redo - Test Cases

• TC001 - Undo/redo block actions
• TC002 - Undo/redo text
• TC003 - Undo/redo text format 🟡
  • Undo/redo - Apply text format to a link breaks the undo functionality - Undo/redo - Apply text format to a link breaks the undo functionality #3136

Result

I've spotted three new issues I missed when testing the editor release but none of them are new regression issues, I tested on the latest released version 16.6 and they are also reproducible there.

Issues spotted on editor release 1.46.0 (not new)

• Text flickering when removing a line on some multiline blocks - Text flickering when removing a line on some multiline blocks #3096
• Pullquote Block - Wrong cursor position when merging back - Tapping on a Page Break block does make the block-holder selected but keep the keyboard up, and the caret to the Para block #308
• Heading block stops being rendered as a header when deleting the first word - Heading block stops being rendered as a header when deleting the first word #3106

Issues spotted on this run (new but not regression)

• Link button - Remove link when the word is selected doesn't remove it - Link button - Remove link when the word is selected doesn't remove it #3134
• Splitting paragraph block - Deleting the first word after selecting it requires tapping backspace two times - Splitting paragraph block - Deleting the first word after selecting it requires tapping backspace two times #3135
• Undo/redo - Apply text format to a link breaks the undo functionality - Undo/redo - Apply text format to a link breaks the undo functionality #3136

[fluiddot]

Android: Smoke test checklist 🟡

Tested on Samsung Galaxy S20 FE 5G, Android 10 - Version 16.7-rc-1

• Entering exiting the editor
• Rotate the device in various cases
  • Rotate with no block selected
  • Rotate with a block selected
  • Rotate with the keyboard opened 🟡
    • Keyboard window is dismissed on second rotation - Keyboard window is dismissed on second rotation #3137
  • Rotate with the bottom sheet opened
• Open and dismiss the bottom sheet
  • Open the bottom sheet
  • Dismiss it by tapping outside
  • Dismiss it by dragging it to the bottom
• Background the app and foreground it again
  • Go to opened apps and back to the app
  • Go to home screen and back to the app
  • Close the app and open it again
• Make some changes in a post, close the app, open it again and verify that the changes are saved

Result

Only one issue has been spotted and it's not new, I tested on the latest released version 16.6 and it's also reproducible there.