Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Set up screenshot tests #9

Closed
manuroe opened this issue Apr 20, 2022 · 0 comments · Fixed by #130
Closed

Set up screenshot tests #9

manuroe opened this issue Apr 20, 2022 · 0 comments · Fixed by #130
Assignees
@ismailgulek
Labels
A-DevX Anything that can improve the DevX Z-BBQ-Alpha Required for ElementX - BBQ - Alpha

Comments

@manuroe
Copy link
Member

manuroe commented Apr 20, 2022 •
edited by VolkerJunginger
Loading

We would like to use snapshots for our UI tests to speed up repetitive tasks and better resolution.

It seems that the 2 main contenders are:

We should play around with both and see which one we like better.

We also need to find a solution for restoring artefacts between successive runs so we have something to compare with.

It's still up for discussion whether this should be done on Github Actions or Xcode Cloud, mainly because we don't have access to Xcode Cloud yet.

We also talked about adding debug borders around all views to improve snapshot accuracy and, if speedy enough, we can use snapshots within the integration tests as well.
@manuroe manuroe added Z-Project-Setup A-DevX Anything that can improve the DevX labels Apr 20, 2022
@manuroe manuroe moved this to To Refine in Mobile DevX Reset Project Apr 20, 2022
@stefanceriu stefanceriu added Z-BBQ-Alpha Required for ElementX - BBQ - Alpha and removed A-DevX Anything that can improve the DevX Z-Project-Setup labels Jun 29, 2022
@stefanceriu stefanceriu added the A-DevX Anything that can improve the DevX label Jun 29, 2022
@ismailgulek ismailgulek self-assigned this Jun 30, 2022
ismailgulek added a commit that referenced this issue Jul 4, 2022
@ismailgulek
#9 Add snapshot testing library
23d93ca
ismailgulek added a commit that referenced this issue Jul 4, 2022
@ismailgulek
#9 Create script to boot test simulators
88b1bb9
ismailgulek added a commit that referenced this issue Jul 4, 2022
@ismailgulek
#9 Create the UI test plan
167347c
ismailgulek added a commit that referenced this issue Jul 4, 2022
@ismailgulek
#9 Create shared schemes for test targets
b7ac956
ismailgulek added a commit that referenced this issue Jul 4, 2022
@ismailgulek
#9 Disable split view for UI tests
dd4480c
ismailgulek added a commit that referenced this issue Jul 4, 2022
@ismailgulek
#9 Fix fastlane dependencies
bd4bb39
ismailgulek added a commit that referenced this issue Jul 4, 2022
@ismailgulek
#9 Add snapshot testing to the application
cb78f4f
ismailgulek added a commit that referenced this issue Jul 4, 2022
@ismailgulek
#9 assert screenshots
8a149c1
ismailgulek added a commit that referenced this issue Jul 4, 2022
@ismailgulek
#9 fix swipe gestures on iPad
b377d89
ismailgulek added a commit that referenced this issue Jul 4, 2022
@ismailgulek
#9 Fix accessing items in session verification screen
accadeb
ismailgulek added a commit that referenced this issue Jul 4, 2022
@ismailgulek
#9 Workaround for flaky unit test
12c917c
ismailgulek added a commit that referenced this issue Jul 4, 2022
@ismailgulek
#9 Add reference screenshots
d08380e
@ismailgulek ismailgulek mentioned this issue Jul 4, 2022
Merged
ismailgulek added a commit that referenced this issue Jul 4, 2022
@ismailgulek
#9 Add missing screenshots
edf4672
ismailgulek added a commit that referenced this issue Jul 4, 2022
@ismailgulek
#9 Specify scheme for alpha build
4d64cde
ismailgulek added a commit that referenced this issue Jul 5, 2022
@ismailgulek
#9 Specify scheme for alpha build
34759e3
ismailgulek added a commit that referenced this issue Jul 5, 2022
@ismailgulek
#9 Add reference screenshots
2a3cea9
ismailgulek added a commit that referenced this issue Jul 25, 2022
@ismailgulek
#9 Specify scheme for alpha build
9fc9223
ismailgulek added a commit that referenced this issue Jul 25, 2022
@ismailgulek
#9 Add reference screenshots
67ac3db
ismailgulek added a commit that referenced this issue Jul 25, 2022
@ismailgulek
#9 Add snapshot testing library
dad1faa
ismailgulek added a commit that referenced this issue Jul 25, 2022
@ismailgulek
#9 Create script to boot test simulators
2b46323
ismailgulek added a commit that referenced this issue Jul 25, 2022
@ismailgulek
#9 Create the UI test plan
7600fee
ismailgulek added a commit that referenced this issue Jul 25, 2022
@ismailgulek
#9 Create shared schemes for test targets
d18fd55
ismailgulek added a commit that referenced this issue Jul 25, 2022
@ismailgulek
#9 Disable split view for UI tests
6d7be34
ismailgulek added a commit that referenced this issue Jul 25, 2022
@ismailgulek
#9 Fix fastlane dependencies
a57a256
ismailgulek added a commit that referenced this issue Jul 25, 2022
@ismailgulek
#9 Add snapshot testing to the application
c079c9b
ismailgulek added a commit that referenced this issue Jul 25, 2022
@ismailgulek
#9 assert screenshots
ca08bf0
ismailgulek added a commit that referenced this issue Jul 25, 2022
@ismailgulek
#9 fix swipe gestures on iPad
8666f7f
ismailgulek added a commit that referenced this issue Jul 25, 2022
@ismailgulek
#9 Fix accessing items in session verification screen
70c7dab
ismailgulek added a commit that referenced this issue Jul 25, 2022
@ismailgulek
#9 Workaround for flaky unit test
fd87eb7
ismailgulek added a commit that referenced this issue Jul 25, 2022
@ismailgulek
#9 Specify scheme for alpha build
2a33e2b
ismailgulek added a commit that referenced this issue Jul 25, 2022
@ismailgulek
#9 Add reference screenshots
78fa537
ismailgulek added a commit that referenced this issue Jul 29, 2022
@ismailgulek
#9 Add snapshot testing library
16be137
ismailgulek added a commit that referenced this issue Jul 29, 2022
@ismailgulek
#9 Create script to boot test simulators
bb62b5d
ismailgulek added a commit that referenced this issue Jul 29, 2022
@ismailgulek
#9 Create the UI test plan
a842113
ismailgulek added a commit that referenced this issue Jul 29, 2022
@ismailgulek
#9 Create shared schemes for test targets
819fae3
ismailgulek added a commit that referenced this issue Jul 29, 2022
@ismailgulek
#9 Disable split view for UI tests
44c381e
ismailgulek added a commit that referenced this issue Jul 29, 2022
@ismailgulek
#9 Fix fastlane dependencies
00d84ce
ismailgulek added a commit that referenced this issue Jul 29, 2022
@ismailgulek
#9 Add snapshot testing to the application
0e20566
ismailgulek added a commit that referenced this issue Jul 29, 2022
@ismailgulek
#9 assert screenshots
def2a70
ismailgulek added a commit that referenced this issue Jul 29, 2022
@ismailgulek
#9 fix swipe gestures on iPad
132f27a
ismailgulek added a commit that referenced this issue Jul 29, 2022
@ismailgulek
#9 Fix accessing items in session verification screen
3599d4e
ismailgulek added a commit that referenced this issue Jul 29, 2022
@ismailgulek
#9 Workaround for flaky unit test
476de71
ismailgulek added a commit that referenced this issue Jul 29, 2022
@ismailgulek
#9 Specify scheme for alpha build
91a7436
ismailgulek added a commit that referenced this issue Jul 29, 2022
@ismailgulek
#9 Add reference screenshots
0e62980
ismailgulek added a commit that referenced this issue Aug 11, 2022
@ismailgulek @pixlwave
Screenshot tests (#130)
2cb6dc1 
* #9 Add snapshot testing library

* #9 Create script to boot test simulators

* #9 Create the UI test plan

* #9 Create shared schemes for test targets

* #9 Disable split view for UI tests

* #9 Fix fastlane dependencies

* #9 Add snapshot testing to the application

* #9 assert screenshots

* #9 fix swipe gestures on iPad

* #9 Fix accessing items in session verification screen

* #9 Workaround for flaky unit test

* #9 Specify scheme for alpha build

* #9 Add reference screenshots

* Update python script path and check assets for png check

* Update script path

* Use static timezone for simulator time

* Fix build after SwiftFormat

* Add changelog

* Upload failed screenshots artifact

* Always upload artifacts

* Update boot simulator script

* Update simulator overridden time

* Install pytz before tests

* Get time from Ruby script

* Disable SwiftUI animation when running UI tests

* Update screenshots after animation setting

* Include reference images in the artifact

* Update matching precision

* Update image matching precision & revert artifact content

* Include Xcode result in the artifact

* Update test output directory

* Disable gradient on splash screen for tests

* Tap next button explicitly

* Wait a bit before checking alert

* Wait 1 second

* Run SwiftFormat on project

* Ignore temporary screenshots

* Fix most of the PR remarks

* Fix conflicts

* Bump Python version to 3

* Update reference screenshots for authentication screens

* Update SwiftFormat

* Fix flakey session verification test.

* Update scheme.

Co-authored-by: Doug <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ismailgulek ismailgulek

Labels
A-DevX Anything that can improve the DevX Z-BBQ-Alpha Required for ElementX - BBQ - Alpha
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Screenshot tests element-hq/element-x-ios
3 participants
@stefanceriu @ismailgulek @manuroe