Developers: Writing Tests for Pyfa

Writing tests for Pyfa is fairly simple, but there are a few guidelines to follow.

---

Any rule can be broken, but there's a cost to breaking them.

If the cost is too high, the pull request will be rejected until the tests are rewritten.

Write more tests!

Currently nearly any bit of code outside of gui can have tests written for it. If you are submitting a PR, you are expected to submit a matching set of tests that covers the code in the PR. Sometimes this means you need to write broader tests, as your PR may only be for a small portion of a method.

Each test should be test one specific thing.

Lumping tests together should be avoided if at all possible. While this increases the time it may take to run tests, it makes tests more targeted and easier to track the root cause when they fail. A good indication is if your assert statements are all grouped in a single block, versus scattered through the test.

Example: Testing all attributes on a fit after adding a single module (which may be 87+ assert statements) is a good example of a test with a narrow focus. Testing how a fit changes over time when adding multiple modules would not be a narrow focus, even if it has less assert statements.

Get help

There are a number of helper fixtures already written. Use them to avoid recreating work and keeping tests more standardized. Helper fixtures can be invoked by importing them, then being called as a parameter in your function.

In order to initialize the database and configure Eos:

from _development.helpers import DBInMemory as DB

def test_example(DB):

This will initialize the database and create a DB dictionary that can be used later if needed. Example query to the database: DB['db'].getItem("Co-Processor II")

Gamedata and Savedddata also have helpers, and can be initialized using the same method. We can then call them to pull data. To get the All 0 character would be: char0 = Saveddata['Character'].getAll0()

Name your tests

Test names are important because it's often the only information you have to go on when a test fails. Tests methods must always start with test_, if you do not follow this convention the test will not run.

The rest of the method name is more flexible, and should be as descriptive as possible. A recommended convention is: test_<method name that you are testing against>_<what you are testing>

test_newtest is an example of a bad method name. test_getFitsWithShip_RifterFit is an example of a better method name. We know what function we're testing against (getFitsWithShip) and we know what we're doing with it (testing against RifterFit).

Location location location

Tests are broken into a folder structure to match their purpose. There are two primary locations tests will go: test_modules and test_smoketests.

test_modules is a folder structure that should mirror the existing folder structure in Pyfa. If you are testing a function that lives in eos.saveddata.fit.py, then the test would be test_modules.test_eos.test_saveddata.test_fit.py. All test folders should start with test_ to prevent shadowing of the existing folder structure. Any tests in test_modules should be as low level as possible, ideally targeting a specific method in a specific module.

test_smoketests is for more general tests that test a wide range of code, without a specific target. These tests will be used to ensure that very high level actions perform as expected. If you want to test a fit, with a booster, and a specific created character, then this is a good place for that sort of test.

Cleanup after yourself

All tests must clean up after themselves. Tests are not guaranteed to run in a particular order, so any changes made to the environment should be backed out before the test ends. If a complex set of steps is needed to build the environment, and you wish to run many tests on that environment, a class can be used to group the tests together so the environment doesn't have to be recreated.

Example test that saves a fit to the database, checks for the assert then removes it.

def test_getFitsWithShip_RifterFit(DB, RifterFit):
    DB['db'].save(RifterFit)
    assert Fit.getFitsWithShip(587)[0][1] == 'My Rifter Fit'
    DB['db'].remove(RifterFit)

Running unit-tests and smoke-tests

Switch to proper virtual environment

For cmd.exe: PyfaEnv\scripts\activate.bat
For bash: source <venv>/Scripts/activate

Install pytest

pip install pytest  

Switch to pyfa directory.

Run tests (any will do)

python -m pytest
py.test