Skip to content

Testing Guide for Bugfixes

Jump to bottom
jwzimmer edited this page Jun 2, 2016 · 5 revisions

Testing takes 2 forms in this repo – browser testing, and unit testing.

Browser Tests

We currently support the following browsers in scratch-www:

  • Chrome (Latest - 2 versions)
  • Firefox (Latest - 2 versions)
  • Safari (Latest - 2 versions)
  • Internet Explorer 10+
  • Microsoft Edge (Latest - 2 versions)

Browser Testing your own bugfix

When submitting a bugfix PR, make sure you test it on a browser to confirm it's working for you locally first. We recommend testing on at least 2-3 browsers before submitting a PR, if possible. The exceptions to this rule would be if:

  1. You do not have access to more than one browser
  2. The bugfix is browser-specific (i.e. addressing a problem that only occurs on one browser anyway)

Browser Testing another person's bugfix

One of the other things that you can do in this repo that is incredibly helpful is perform QA on other people's bug fixes locally (i.e. testing that a fix works for you, and that it doesn't adversely affect other parts of the website). If testing locally, we recommend testing on at least 3 browsers – all 5 if possible. The exceptions to this rule would be if the bugfix is browser-specific.

When testing in the browser, it's important to make sure that, in addition to the bug fix working properly, no other existing parts of the site that are adjacent to the bug fix are adversely affected by the code changes. If you're trying to figure out what parts of the site to take a look at, here's a handy dandy list of current areas of the website handled by this repo, broken down by unique user experience:

If a bug fix is for a specific area, make sure to test all of the functionality for that particular area as described in each of those sections. If you encounter bugs in one of these areas when testing a bug fix, make sure to leave a comment describing how to replicate the bug on the original Pull Request for the bug fix.

Functional Tests

Writing functional test scripts are incredibly helpful for us with bug fixes, as they can help to isolate problems that might arise during QA to specific areas of the website, especially between functional components and browser interactions.

When submitting a PR, check to see if it is possible to add a test for the fix. In general, test scripts should be writeable for everything functional. The only things that might, as a general rule, be exempt from test scripts are bugs related to device display and formatting.

Our Testing Infrastructure

We use tap as our testing library. All of our unit tests are contained within a top-level directory called test/. Within test, there are three subdirectories, each for a different type of test:

  • unit/ – test scripts for individual pieces of code that do one thing and one thing only (low level tests)
  • functional/ – test scripts for methods that may involve multiple units of code (mid level tests)
  • integration/ – test scripts for end-to-end functionality of a feature or component (high level tests)

Each separate test that is added should be contained to its own file within one of these directories. In order to run the test suite, you can use the following cli commands:

make test – builds the repo, and runs all tests (recommended)
make unit – runs all tests in the unit/ subdirectory
make functional – runs all tests in the functional/ subdirectory
make integration – runs all tests in the integration/ subdirectory

Refer to our Makefile to see how this is set up. If you would like to run just one test – or a subset of tests – you can do so using the tap bash script. For instance:

./node_modules/.bin/tap ./test/unit/trust_me_this_works.js
Clone this wiki locally