-
Notifications
You must be signed in to change notification settings - Fork 1
/
Copy pathtestplan.tex
247 lines (193 loc) · 15.3 KB
/
testplan.tex
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
\documentclass[]{article}
\begin{document}
\title{Wifiology Test Plan}
\author{
Robert Cope, Jason Nguyen, Baiyu Chen, \\
Jasper Niemeyer, Peng Jiang, Ryan Campbell\\
Group: 404 Group Not Found}
\maketitle
\section{Test Plan Overview}
\subsection{Application Description}
The Wifiology application is a wireless profiling and monitoring tool, which utilizes physically colocated nodes
actively listening to 802.11 (WiFi) traffic, in conjunction with a central service to aggregate the collected data.
Our service allows users to deduce current wireless congestion, watch for rogue access points, gauge the number of
connected users at any physical location, and in large deployments, help users find locations that aren't overcrowded
and have reasonable wireless networking performance. Our application is a combination of Python on the physical nodes
themselves, with a centralized NodeJS server aggregating data.
\subsection{Target Users}
For this application, we plan to have two sets of target users. The first set of users are owners and administrators
of wireless networks, the second set of users being the actual consumers of the wireless networks. Owners and
administrators will purchase this service to help better understand their wireless network user experience and to
augment the process of administering and policing spectrum; once installed the second set of users can use the
service to plan visits to locations with 802.11 hotspots and check for poor connectivity.
\paragraph{}
To help guide our user acceptance testing, we plan to find a group of early adopter network admins, for whom
we will offer the service at a discount or for free; in return we will gather their feedback and guidance in
reviewing our user acceptance tests and providing feedback on the deployed application.
\subsection{Test Execution Team}
For the execution of our manual UAT tests, the group has elected Robert Cope and Jasper Niemeyer to
fulfill the role of manual tester in executing our tests. The automated tests, as described below will
be executed as part of our continuous integration/continuous deployment pipeline. The role of test
script developer for end-to-end automated user acceptance tests so far has been conducted by Robert Cope,
but has been agreed to be conducted by Jason and Ryan as well.
\section{User Acceptance Testing}
\subsection{Tools}
For the implementation of our automated UTA test cases, which will interact with the both the API and our web user interface,
we use the Python ``unittest" framework in conjunction with Selenium webdriver to automate a user interacting with our
web service. We current record results by hand and store our test plans this document, but plan make use of the qTest
tool in the near future, for automated recording of both manual and automated test cases and their results.
\subsection{Feature: Export Data To CSV}
\subsubsection{Description}
In order to facilitate external analysis and data recording, our tools allows administrators to download the collected
and aggregated data from their nodes in CSV format. This is done by logging into the application, selecting the node
the user is interested in, and clicking a button to trigger the creation of a CSV from data stored in our database.
\paragraph{}
Our BDD statement for this test is as follows: ``As an admin consuming the Wifiology service, I want to be able to download the
raw data my nodes have been collecting, so that I can perform additional analyses and import my data into other tools
to facilitate my network administration job."
\subsubsection{Entry Criteria}
To begin user acceptance testing, a build of the central server code base must be available and be free of syntax errors.
This build must also have passed all of our unit and integration tests, as described below. Data will be populated from
a sample set, described below, and the application will be started locally for the user acceptance tester. Additionally
the final entry criterion for this is that an additional document is produced explicitly detailing how to conduct the
manual test for this feature. Once this document is completed, we will examine how much of this can
be converted to Selenium based automated tests; needless to say, this additionally gates the automated testing portion
of our test plan.
\subsubsection{Exit Criteria}
The exit criteria for this test is that the tester is successfully able to log in, generate a valid CSV file from
the data stored in the database and download the data locally, and that the CSV data matches the expected values
given knowledge of the prepopulated data.
\subsubsection{Test Preparation}
To facilitate consistency in testing of this feature, the application developers and test scripter(s) will generate
a prerecorded set of sample data, and create a script to update a target database with this information. Prior to
the test being executed, this script will be run to prepopulate the database with this information. The server
will then be started for the user to begin testing.
\subsubsection{Manual Testing}
In order to conduct the manual testing of this feature, the tester will do the following:
\begin{itemize}
\item The manual tester point their browser to the supplied instance of the application.
\item The tester will login using the provided credentials, and verify that they are able to successfully log in.
\item The tester will use the application menu to browse to the listener node of interest.
\item The tester will click the button for CSV download.
\item The tester will verify that the downloaded CSV is a valid format, and may be opened using a common spreadsheet tool like Excel.
\item The tester will verify that the information stored in the CSV matches the expected values.
\end{itemize}
Additionally the tester should score the following subjective questions as satisfactory or unsatisfactory,
and provide feedback if unsatisfactory:
\begin{itemize}
\item Did I find the process of finding where to download the CSV data I needed intuitive?
\item Did the CSV data include all of the fields of data I was interested in?
\end{itemize}
If the tester is able to complete all of these, the test will be considering passing; any failure in this process will
mark this test as failed. If the test fails, a bug will be opened against the product for the developers to analyze the
failure and rectify the code base.
\subsubsection{Automated Testing}
Automated testing for this feature will be identical to the manual test plan above, with the exception being that
a script is generated to perform this test using the Selenium webdriver toolkit. The test will still be performed
manually as a gate to release, but this automated UAT test will provide a means to save tester time.
\subsection{Feature: Show Aggregate 802.11 Traffic on an Individual Node}
\subsubsection{Description}
This feature allows both wireless network administrators to gauge the amount of traffic (and hence
congestion and potentially oversubscription) for a given physical proximity around a listener node. Administrators
will be able to see the number of devices (and their MAC addresses) that have been seen recently, and with what
access points have they been interacting with.
\paragraph{}
Our BDD statement for this test is as follows: ``As a network admin consuming the Wifiology service, I want to
be able to see the amount of 802.11 traffic and the devices creating this traffic, so that I can gauge
the current and historic utilization of my wireless spectrum."
\subsubsection{Entry Criteria}
To begin user acceptance testing, a build of the central server code base must be available and be free of syntax errors.
This build must also have passed all of our unit and integration tests, as described below. Data again will be populated from
a sample set, described below, and the application will be started locally for the user acceptance tester. As before,
the final entry criterion for this is that an additional document is produced explicitly detailing how to conduct the
manual test for this feature, once completed.
\subsubsection{Exit Criteria}
The exit criteria for this test is that the tester is successfully able to log in, see a detailing listing of the
current traffic, as well as browse historic traffic data for the historic window of their choosing.
\subsubsection{Test Preparation}
Again, to facilitate consistency in testing of this feature, the application developers and test scripter(s) will generate
a prerecorded set of sample data, and create a script to update a target database with this information. Prior to
the test being executed, this script will be run to prepopulate the database with this information. The server
will then be started for the user to begin testing.
\subsubsection{Manual Testing}
In order to conduct the manual testing of this feature, the tester will do the following:
\begin{itemize}
\item The manual tester point their browser to the supplied instance of the application.
\item The tester will login using the provided credentials, and verify that they are able to successfully log in.
\item The tester will use the application menu to browse to the listener node of interest.
\item The tester will examine the page to see the ``latest" traffic report, and verify that the data is intelligible.
\item The tester utilize an input on the page to select a given historic range at random.
\item The tester will make the service execute the historic lookup for the range specified and again verify the data is intelligible.
\end{itemize}
Additionally the tester should score the following subjective questions as satisfactory or unsatisfactory,
and provide feedback if unsatisfactory:
\begin{itemize}
\item Did I find the process of finding where the node data page intuitive or possible?
\item Was I able to input the historic range of interest with relative ease?
\item Did data read out include all of the information I was interested in?
\end{itemize}
If the tester is able to complete all of these, the test will be considering passing; any failure in this process will
mark this test as failed. If the test fails, a bug will be opened against the product for the developers to analyze the
failure and rectify the code base.
\subsubsection{Automated Testing}
Some portions of this, once the we demonstrate the ability to execute the test manually, will be converted to automated
Selenium testing (though this will not replace the manual tests). We expect our Selenium tests to also perform the
same checklist above, logging in, checking for predefined data, and verify that the correct data fields are present on
the page.
\subsection{Feature: Show Discovered 802.11 Access Points/Service Sets}
\subsubsection{Description}
The final feature for this UAT test plan to the ability for our admin users to see all ``service sets" (colloquially knowns
as ``wireless networks") that have been seen broadcasting their existence within listening distance of a deployed node. This
feature enable network admins to track rogue access points and clamp down on forms of unauthorized access.
\paragraph{}
Our BDD statement for this test is as follows: ``As a network administrator, I want a listing of access points or service
sets that have been discovered near my listener nodes, so that I can clamp down on rogue access points, and ensure secure
network access."
\subsubsection{Entry Criteria}
To begin user acceptance testing, a build of the central server code base and the listener node code base
must be available and be free of syntax errors. Both builds must also have passed all of our unit and integration tests,
as described below. Because data will not be prepopulated for this test, the tester or any supporting staff must
stand up both the listener and server, configure the listener with the service, and deploy a new
``rogue" access point in the vicinity of the listener node. As before, the final entry criterion for this is that an
additional document is produced more explicitly detailing how to conduct the manual test for this feature, once completed;
once this document is completed.
\subsubsection{Exit Criteria}
The exit criteria for this test is that the tester is successfully able to log in, and see the
``rogue access" point in the listing of access points/service sets discovered by the given listener node.
\subsubsection{Test Preparation}
In order to allow the testing to occur the tester will have both a on the shelf wireless access point
(like those commonly available at electronics stores), as well as a physical listener node with a
working build of the listener node code base. The listener node will be turned on, and configured to
talk to an instance of the central server, which has been deployed for this test. The tester will
place the access point and listener node within relative proximity of each other (no close than 3 feet,
but no further than 100 feet), and let data collection occur for at least 30 minutes.
\subsubsection{Manual Testing}
In order to conduct the manual testing of this feature, the tester will do the following:
\begin{itemize}
\item The manual tester point their browser to the supplied instance of the application.
\item The tester will login using the provided credentials, and verify that they are able to successfully log in.
\item The tester will use the application menu to browse to the listener node of interest.
\item The tester will examine the page for the latest service sets seen report.
\item The will verify that the deployed ``rogue access" point is listed in the report.
\end{itemize}
Additionally the tester should score the following subjective questions as satisfactory or unsatisfactory,
and provide feedback if unsatisfactory:
\begin{itemize}
\item Did I find the process of finding where the node data page intuitive or possible?
\item Did I find all of the data I needed about the service sets on this page?
\end{itemize}
If the tester is able to complete all of these, the test will be considering passing; any failure in this process will
mark this test as failed. If the test fails, a bug will be opened against the product for the developers to analyze the
failure and rectify the code base.
\subsubsection{Automated Testing}
We do not foresee any automated testing being utilized for user acceptance testing on this feature.
\section{Unit and Integration Testing}
\subsection{Listener Node Code Base}
The Python code base for the remote nodes which listen for 802.11 traffic will continue to utilize unit and
integration tests written in the Python ``unittest" framework. We aim to have 80-90\% code coverage for this
code base. We require new tests be written for all new feature work introduced to this code bases.
\subsection{Central Server Code Base}
Our NodeJS code base, which is the implementation of the central server and web service for Wifiology, also has unit
and integration tests. We utilize the ``Mocha" and ``Chai" test framework tools to provide both unit and integration
tests on our NodeJS code base. We also aim to have 80-90\% code coverage for our tests.
\end{document}