-
Notifications
You must be signed in to change notification settings - Fork 2
/
testplan.txt
180 lines (126 loc) · 4.77 KB
/
testplan.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
#### KaVE Test Plan ####
## Activity Events ##
* See whether mouse movement causes Activity Events
* Wait for the timeout (3s)
* cause longer event by moving the mouse for ~10s in smaller intervals with ~1-2s breaks in between
## Commands ##
* Invoke Cleanup Code from ReSharper
* save file
* build solution
* Use R# BulbAction (e.g. use fully qualified class name for static call)
* added code should be redundant, use AltEnter zu remove code again
* refactor variable name
KaVE specific events (make sure the following actions create command events)
* opening the "Event Manager"
* All (relevant) buttons in the toolbar
* All (relevant) buttons in the menu
* delete one event
* delete multiple events
* delete one day
* delete multiple days
(proke reminder by changing AppData\Roaming\JetBrains\Shared\vAny\GlobalSettingsStorage.DotSettings)
* open export window from reminder popup (3 days)
* open export window from strong reminder popup (7 days)
* open manual upload site from export window
## CodeCompletions ##
* Open Small file, trigger Code completion
* assert recorded time
* assert recorded duration of window open
* abort selection and assert correct terminationState
* apply selection and assert correct terminationState
* assert proposals are recorded
* assert selections are recorded (pay attention to timing)
* assert context and trigger location
## Build Events ##
-- ReSharper Build
* assert that at least the command is recorded
-- vsbuild
* build solution and project
* assert correct "Scope"
* assert "Action" is "Build"
* clean solution, assert Action
* "Build" and "Rebuild" solution, assert Actions
## Debugger ##
-- on a test
* Debug a successful test
* Debug a successful test with a breakpoint
* step over/in/out
* continue
* abort
* regular end of program
* hit another breakpoint by stepping over
* hit an unhandled exception
-- on an application
* debug an application (and do the steps above)
-- attach to an application
* attach to a running process (and do the steps above) -- e.g., console application with Console.ReadLine
* detach from a running process
## Document ##
* open/close/save a document
* close several files at once and lookup for multiple Document.Close
## Edit Events ##
(edit events have a timeout, always try with and without -- as of now, should be 3s)
* open or create a file
* change SST relevant stuff
* add whitespace
* combine with code completion selection
* change a file, close the solution, change the same file... should work without slowdown/exception
## Find Events ##
* Right now, only the "Find all" search does return any meaningful values.
* only non-canceled searches are tracked
* neither "prev/next", nor in document searching is tracked
## IDE State Event ##
* quit IDE
* start IDE
* kill IDE (alt-f4) -- has an event
* kill IDE (task manager) -- no event
* close while multiple windows(!) are open (should appear in "OpenWindows")
* start by opening a specific file (should appear in "OpenDocuments") -- don't know how to test with experimental instance
## Solution Event ##
* open solution
* close solution
* rename solution
* add/delete/rename solution items
* add/delete/rename project
* (add/delete/rename projectitem not working)
## Window Event ##
* switch between files and several open windows in visual studio
* detach a tool window (e.g., test runs) and switch to the external window
* test VS focus loose/gain events
* move a window
## Version Control Event ##
* git init in running instance does not do anything
* comit one file, still does not show up
* re-open solution (commit should show up now)
* commit something else
* re-open again (all commits show up now)
## User Profile Event ##
export zip locally and assert existence of event
## Navigation Event ##
* click into a method
* click into the class body
* key navigate from one method to another (location: method, class, method)
* ctrl-click within a file
* ctrl-click to another local type
* ctrl-click to another local method
* ctrl-click to disassembled type (only sometimes working)
* ctrl-click to disassembled method (only sometimes working)
## System Event ##
* (dis-) connect remote session
* (un-) lock local login
* send computer to sleep, wake it up (not possible in VM)
## Test Run Event ##
* run a testsuite with all possible outcomes
* abort a test
* run a single test
* debug a test
* make sure that the events are not thrown twice during testing (e.g., aborted/finished)
* Analyze esp. the timing information in all cases
########################
## Intelligent Code Completion
* trigger code completion on unresolved type
* trigger code completion on non-existing type
* trigger code completion on StringBuilder
* trigger code completion on StringBuilder in diff. context
## KaVE Settings
* assert that settings (esp. user id!) are preserved throughout the update