Add UI/frontend test to pbench-dashboard #2980
Conversation
- Integrate WTRobot with pbench-dashboard - Automate valid and invalid login testcases
| .vscode/ | ||
| tmp/ | ||
| wtlog.log | ||
| geckodriver.log |
There was a problem hiding this comment.
Is there an equivalent for Chrome?
| "locale": "en_US", | ||
| "log": "wtlog.log", | ||
| "dev": false | ||
| } |
There was a problem hiding this comment.
Trailing newlines are generally a good idea; if nothing else, it keeps GitHub from showing the big red warning icon!
|
|
||
| > NOTE: | ||
| - Selenium_drivers folders have your selenium webdrivers geckodrivers(for firefox) and chromedrivers(for chrome and chromium) | ||
| - If script fails due to drivers issue, you need to find appropriate selenium webdriver according to your browser version(choose latest if your browser is upto to date) |
There was a problem hiding this comment.
s/version(/version (/ ... although perhaps the parenthesized phrase should really be a separate sentence.
s/upto/up to/
Are those version numbers browser versions or driver versions and how are they related? If they're driver versions, is there a key for relating those to browser versions?
Finally, since browser independence is going to be a strong requirement, is there a way to have the test system pre-configured for both so that we can easily wire a functional test system to be sure both run correctly?
| target: http://10.1.170.201 | ||
| targets: | ||
| - http://10.1.170.201 |
There was a problem hiding this comment.
We don't want to hardcode IP addresses here; is there a way of importing these by variable from outside? This, in particular, is my personal test server and ideally nobody else would be depending on it at all ... in general, we'll be running our functional tests against a transient pod set up and torn down by the Jenkins pipeline and we need to run the dashboard tests against that server.
| - step 3: | ||
| action: input | ||
| target: //*[@id="username"] | ||
| value: vishalvvr |
There was a problem hiding this comment.
We don't want to hardcode user and password, either; and certainly not to your account! While we could stand up a CI pipeline pod with a pre-configured test user, it'd be much better to have this imported from outside via an environment variable or similar.
|
|
||
| > NOTE | ||
|
|
||
| - If config(in `config.json`) is missing on initial run, tool will ask you for few configuration question and create `config.json` file. |
There was a problem hiding this comment.
There is no config section in config.json, right? If so maybe you can replace
config(in config.json) with just config.json
|
|
||
| - `action`: what to perform (e.g. click, input and etc) | ||
|
|
||
| - `target`: on what to perform (e.g. Text widget on web page, xpath of widget and etc) |
There was a problem hiding this comment.
The example you showed above
target: text | xpath | css path only takes the type of the target. Is specifying only the type enough or do we need to be more verbose than that like you mentioned here?
| target: http://10.1.170.201 | ||
| targets: | ||
| - http://10.1.170.201 |
There was a problem hiding this comment.
Do we need to specify both targets and target?
There was a problem hiding this comment.
In all the steps the two keys always seem to have the same value....
| ```console | ||
|
|
||
| > pip install wtrobot | ||
|
|
||
| ``` |
There was a problem hiding this comment.
I don't think the blank lines here affect the formatted result, and I don't think they make the source code easier to read; so, I recommend omitting them.
Ditto lines 27-31.
| > NOTE | ||
|
|
||
| - Selenium_drivers folders have your selenium webdrivers geckodrivers(for firefox) and chromedrivers(for chrome and chromium) | ||
|
|
||
| - If script fails due to drivers issue, you need to find appropriate selenium webdriver according to your browser version | ||
|
|
||
| - [firefox](https://github.com/mozilla/geckodriver/releases) & [chrome/chromium](https://chromedriver.chromium.org/downloads) | ||
|
|
||
| - Unzip or untar the executable and place in selenium_drivers dir. | ||
|
|
There was a problem hiding this comment.
Placing "NOTE" in a block quote is an interesting formatting choice, but placing the bulleted list under it outside the block quote is weird.
I suggest either including the bullet list in the block quote or replacing the > with bold and/or italics or with four or five-#'s.
Ditto lines 33-36.
|
|
||
| ## Executing Script | ||
|
|
||
| - Write all your test cases into test.yaml and execute |
There was a problem hiding this comment.
Put "test.yaml" in mono-space font (i.e., in back-ticks).
|
|
||
| > NOTE | ||
|
|
||
| - If config(in `config.json`) is missing on initial run, tool will ask you for few configuration question and create `config.json` file. |
There was a problem hiding this comment.
I think the phrase, "ask you for few configuration question", should be "ask you a few configuration questions".
| > NOTE | ||
|
|
||
| - If config(in `config.json`) is missing on initial run, tool will ask you for few configuration question and create `config.json` file. | ||
| - Make sure files which you mention as config(in `config.json`) should exist else will exit with error. |
There was a problem hiding this comment.
This text seems sort of obvious, so I would omit it. However, if you want to include it, perhaps phrasing like, "If any files referenced in config.json don't exist, the tool will exit with an error."
|
|
||
| > NOTE: | ||
| - Selenium_drivers folders have your selenium webdrivers geckodrivers(for firefox) and chromedrivers(for chrome and chromium) | ||
| - If script fails due to drivers issue, you need to find appropriate selenium webdriver according to your browser version(choose latest if your browser is upto to date) |
There was a problem hiding this comment.
This statement seems self-evident, so I would either omit it or rephrase it along the lines of, "You must provide a Selenium webdriver which matches your choice of browser and browser version."
| > NOTE: | ||
| - Selenium_drivers folders have your selenium webdrivers geckodrivers(for firefox) and chromedrivers(for chrome and chromium) | ||
| - If script fails due to drivers issue, you need to find appropriate selenium webdriver according to your browser version(choose latest if your browser is upto to date) | ||
| - [firefox](https://github.com/mozilla/geckodriver/releases) & [chrome/chromium](https://chromedriver.chromium.org/downloads) |
There was a problem hiding this comment.
There is no need to have these references in their own list of one bullet. Just append them to the previous line.
| - Selenium_drivers folders have your selenium webdrivers geckodrivers(for firefox) and chromedrivers(for chrome and chromium) | ||
| - If script fails due to drivers issue, you need to find appropriate selenium webdriver according to your browser version(choose latest if your browser is upto to date) | ||
| - [firefox](https://github.com/mozilla/geckodriver/releases) & [chrome/chromium](https://chromedriver.chromium.org/downloads) | ||
| - Unzip or untar the executable and place in selenium_drivers dir. |
There was a problem hiding this comment.
s/in selenium_drivers dir/in the `selenium_drivers` directory/
| test: | ||
| - testcase 1: | ||
| - scenario: invalid pbench-dashboard login | ||
| - step 1: | ||
| action: goto | ||
| target: http://10.1.170.201 |
There was a problem hiding this comment.
This structure seems a little...odd. It makes reasonable sense that a test would be a sequence of "testcases". It's slightly weird that each testcase would be a mapping with a single key mapped to a sequence, although I suppose that lets you store the name of the test case as its key instead of an attribute. But, it's really weird that the first entry in the testcase sequence is a "scenario" while the other entries are "steps".
A more natural structure might look like this:
test:
- name: testcase 1
scenario: invalid pbench-dashboard login
steps:
- name: step 1
action: goto
target: http://10.1.170.201
...
...
...
That is, test is a sequence of test cases, each of which is a mapping containing a name, scenario, and a sequence of steps; each step is a mapping containing a step name, an action, a target, and so forth.
Although, if you really like the "name attribute as key", it might look like this:
test:
- testcase 1:
scenario: invalid pbench-dashboard login
steps:
- step 1:
action: goto
target: http://10.1.170.201
...
...
...
where test is a sequence of test cases, each of which is a mapping of a single key to a mapping containing a scenario and a sequence of steps; each step is a mapping of a single key to a mapping containing an action, a target, and so forth.
| target: http://10.1.170.201 | ||
| targets: | ||
| - http://10.1.170.201 |
There was a problem hiding this comment.
In all the steps the two keys always seem to have the same value....
PBENCH-800