Skip to content

Latest commit

 

History

History
202 lines (152 loc) · 7.17 KB

TastyDemo.md

File metadata and controls

202 lines (152 loc) · 7.17 KB
Raw

Using webdriver-w3c with tasty

It's possible to run "raw" WebDriver sessions, but it's much more convenient to use the tasty integration. With tasty we can incorporate WebDriver tests alongside, say, quickcheck and HUnit tests, and get test reports, stats, filtering, and parallel execution for free.

This module demonstrates how to set up a basic test executable and configure it with command line options.

{-# LANGUAGE OverloadedStrings #-}
module Main where

We'll need some imports. These are the usual tasty modules:

import Test.Tasty
import Test.Tasty.ExpectedFailure

These are the webdriver-w3c modules:

import Web.Api.WebDriver

And this is the module that integrates the two.

import Test.Tasty.WebDriver

Define your tests

First things first: to make a WebDriver test suite, we need some WebDriver tests. These are just values of type WebDriver IO (). (Or more generally, Monad eff => WebDriver eff (), but that's not important for now.) Here are a few dweeby examples. It's not necessary for the tests to start with _test or use snake_case; I'm doing it here out of habit.

_test_one :: (Monad eff) => WebDriver eff ()
_test_one = do
  navigateTo "https://google.com"

_test_two :: (Monad eff) => WebDriver eff ()
_test_two = do
  navigateTo "https://yahoo.com"
  assertSuccess "time travel achieved"

Define your main

As usual, our program starts with main.

The simplest way to make a test executable with tasty is to use defaultWebDriverMain, which has the following signature:

defaultWebDriverMain :: TestTree -> IO ()

This function wraps tasty's defaultMain, which handles command line option parsing, and adds some magic of its own. TestTree is tasty's type for an executable test suite. There are several functions for building these out of WebDriver sessions; they live in Test.Tasty.WebDriver and have names starting with testCase.

Here's an example main.

main :: IO ()
main = defaultWebDriverMain $
  testGroup "Test Demo"
    [ testCase "navigate to google.com" _test_one
    , testCase "check status" _test_two
    , ifDriverIs Chromedriver ignoreTest $
        testCase "navigate to google.com" _test_one
    ]

We can run different sets of tests based on the value of an option using askOption, and we can change the value of an option locally using localOption. Changing one option based on the value of another option is a common task; for example, some tests should run differently in headless mode.

Several common uses of the askOption pattern are defined in Test.Tasty.WebDriver; for instance, the helper function ifDriverIs lets us adjust options for different drivers, and ifTierIs lets us change behavior between development and testing deployments.

Start your remotes

To actually run our test suite, we need to have at least one remote end running. These are the proxy servers that accept WebDriver requests via http and reach into a browser to make it do stuff. For now, the library has only been tested with geckodriver and chromedriver. Make sure you've got one or both of those installed, then kick off an instance of each one.

For example,

geckodriver > /dev/null 2> /dev/null &

starts a geckodriver instance in the background (but suppresses its otherwise voluminous debug output).

You'll want to take note of which host and port your remote end is listening on. By default, geckodriver listens on localhost, port 4444, and chromedriver listens on 9515.

Run your tests

This demo executable is named wd-tasty-demo. If you install and run it, you'll get an error message:

Error: no remotes defined for geckodriver

What does this mean? To run a webdriver session, we have to tell our program the URIs of the remote ends it should use -- it does not assume one. There are two ways to do this, and you can use either one (or both).

--wd-remote-ends lets us supply the remote end URIs on the command line directly. Suppose I've got geckodriver listening on port 4444 and chromedriver on port 9515 (which they do by default). Then I'd use the following option:

--wd-remote-ends 'geckodriver: https://localhost:4444 chromedriver: https://localhost:9515'

(Note the explicit https scheme; this is required.) This is fine if you have a small number of remote ends running, but the command line quickly gets unwieldy if you have tens or hundreds of remote ends ready to run tests in parallel. So we can also specify the remote end URIs in a specially formatted config file. The config file must look something like this:

geckodriver
- https://localhost:4444
- https://localhost:4445
chromedriver
- https://localhost:9515
- https://localhost:9516

The drivers can come in any order and don't have to be contiguous, and blank lines are ignored. Suppose this file is called ~/.wd/config; then we supply this to our test executable with the following option:

--wd-remote-ends-config ~/.wd/config

webdriver-w3c can also run your tests in parallel. To take advantage of this, you'll need to compile your executable with -threaded -rtsopts -with-rtsopts=-N and start it with the --num-threads N option. You'll also need to start more than one remote end of each type. Note that if you want to run N tests in parallel, then you'll need N instances of each remote end (geckodriver and chromedriver) running in the background. This is because the tests are processed sequentially, even if they run in parallel. For instance, if you have 100 firefox tests followed by 100 chrome tests, but run them with one geckodriver and one chromedriver, the tests will run sequentially.

There are a bunch of other command line options for tweaking the behavior of your webdriver tests; use wd-tasty-demo --help to see a list. Most of these are pretty specialized. Other options are pretty common. In addition to --wd-remote-ends and --wd-remote-ends-config, there's --wd-driver, for specifying which driver to use, and --wd-response-format, which is required when using chromedriver because chromedriver is not fully spec compliant.

Example sessions

Here are some example commands for running this demo.

Run one at a time with geckodriver:

geckodriver --port 4444 > /dev/null 2> /dev/null &
wd-tasty-demo --wd-remote-ends 'geckodriver: https://localhost:4444'

Run one at a time with geckodriver, but can it with all the logs:

geckodriver --port 4444 > /dev/null 2> /dev/null &
wd-tasty-demo --wd-remote-ends 'geckodriver: https://localhost:4444' --wd-verbosity silent

Run one at a time with chromedriver:

chromedriver --port=9515 &
wd-tasty-demo --wd-driver chromedriver --wd-response-format chromedriver --wd-remote-ends 'chromedriver: https://localhost:9515'

Run two at a time with geckodriver:

geckodriver --port 4444 > /dev/null 2> /dev/null &
wd-tasty-demo --wd-remote-ends 'geckodriver: https://localhost:4444' --num-threads 2