Skip to content
This repository has been archived by the owner on Jan 17, 2023. It is now read-only.

Latest commit

 

History

History
128 lines (88 loc) · 3.97 KB

testing.md

File metadata and controls

128 lines (88 loc) · 3.97 KB

👈 Back to README

Testing

We have several mechanisms for testing parts of Test Pilot. All of these tests must pass for Pull Requests to be accepted into the project, so it's very handy to know how to run them and write new ones as code changes.

Lint

We use the recommended mozilla-central lint rules. See the source for eslint-plugin-mozilla for details about the rules.

We also use the following ESLint plugin recommended rules:

  • eslint-plugin-import
  • eslint-plugin-flowtype
  • eslint-plugin-react

To lint the frontend, run npm run lint in the testpilot directory. To lint the add-on, run npm run lint in the addon directory.

To lint only one file in the frontend, run ESLint inside the testpilot directory:

./node_modules/.bin/eslint [path/to/file.js]

To lint only one file in the add-on, run eslint inside the addon directory:

./node_modules/.bin/eslint -c ../.eslintrc [path/to/file.js]

All tests

To quickly run all tests, including add-on tests, frontend tests, eslint checks, and flow types coverage reports, use npm run test:all.

Front-end client tests

Our front-end is based on React & Redux. We use a combination of Mocha, Chai, Sinon, & Enzyme to build tests for it. Reading the docs for each of those projects is the most helpful, but we have plenty of tests to review for inspiration in the frontend/test/app directory.

After completing the [Development Quickstart][quickstart], you can run the tests with the following command:

npm test

However, you can also run the tests on a file watcher, so that they get executed with every change you make:

npm run test:watch

It's useful to have this running as you develop, because then you can see test failures quickly and not have to remember to run the suite before submitting a Pull Request.

Add-on tests

Unit tests for the add-on are run via jpm as an npm script in the addon directory:

cd addon
npm install
npm test -- --binary=/Applications/Nightly.app/Contents/MacOS/firefox-bin

Look in the addon/test directory for examples of tests.

Integration tests

The tests expect the WebExtension to be built. If not you will receive an error stating that the add-on or WebExtension is not found.

Please follow the instructions here.

Test environment setup

The tests can be run on any modern version of Firefox, but to run the installation tests, you must use Firefox Nightly. Firefox must also be executable from the command line. After installing, you can run this command to see if it is:

firefox --version

Run the tests

  1. Install Tox.
  2. Download geckodriver v0.19.1 or later and ensure it's executable and in your path.
tox

This will run the integration tests as well as flake8.

Changing or adding element selectors

Selenium allows for multiple types of HTML/CSS selection methods. The documentation found here shows the different ways to use these available methods.

The pytest plugin that we use for running tests has a number of advanced command line options available. To see the options available, run pytest --help. The full documentation for the plugin can be found here.