Skip to content

Commit

Permalink
Improve readme and --help (#473)
Browse files Browse the repository at this point in the history
- Recommend installing locally. Fixes #249.
- Move more common flags first, both in the readme and in `--help`.
- Sync the descriptions between the readme and `--help`.
- Try to explain the difference between node-test-runner and elm-explorations/test.
- Document where to put your tests.
- Various other little tweaks to try to be more helpful.

PR for elm-explorations/test: elm-explorations/test#160

We should also update the repo description because it currently says:

> Runs elm-test suites from Node.js. Get it with npm install -g elm-test
  • Loading branch information
lydell authored Jan 20, 2021
1 parent fbb6f2a commit 995b916
Show file tree
Hide file tree
Showing 4 changed files with 158 additions and 97 deletions.
40 changes: 12 additions & 28 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,7 @@

Notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](http://keepachangelog.com/).
This project mirrors major Elm versions. So version 0.18.\* of this project will
be compatible with Elm 0.18.\*.
The format is based on [Keep a Changelog](http://keepachangelog.com/). This project mirrors major Elm versions. So version 0.18.\* of this project will be compatible with Elm 0.18.\*.

## 0.19.1-revision4 - 2020-09-21

Expand Down Expand Up @@ -101,8 +99,7 @@ be compatible with Elm 0.18.\*.

### Fixed

- All reporters were considered "machine readable", resulting in the error
stream being ignored (#161)
- All reporters were considered "machine readable", resulting in the error stream being ignored (#161)

### Added

Expand All @@ -117,8 +114,7 @@ be compatible with Elm 0.18.\*.

### Changed

- Imports in the `Example.elm` file are now sorted for compatibility with
`elm-format@exp`
- Imports in the `Example.elm` file are now sorted for compatibility with `elm-format@exp`

## 0.18.5 - 2017-06-11

Expand All @@ -128,8 +124,7 @@ be compatible with Elm 0.18.\*.

### Added

- If your project depends on `elm-lang/html`, `elm-test init` will add a
dependency to `eeue56/elm-html-test` so you can test your HTML. (#154)
- If your project depends on `elm-lang/html`, `elm-test init` will add a dependency to `eeue56/elm-html-test` so you can test your HTML. (#154)

## 0.18.4 - 2017-05-30

Expand All @@ -142,41 +137,30 @@ be compatible with Elm 0.18.\*.

### Added

- `--add-dependencies target/elm-package.json` flag to add any missing
dependencies _from_ the `elm-package.json` file in the current director _to_ the
`target/elm-package.json` file. (#28)
- `--add-dependencies target/elm-package.json` flag to add any missing dependencies _from_ the `elm-package.json` file in the current director _to_ the `target/elm-package.json` file. (#28)
- `Test.todo` to mark tests as _not yet implemented_. (#104)
- `--fuzz` flag to override the default fuzz count. (#77)
- `Test.only` and `Test.skip` to limit which tests will be executed.

### Changed

- `elm-test init` now adds all dependencies from the package `elm-package.json`
to the generated `tests/elm-package.json` file. (#68)
- You no longer write a `tests/Main.elm` file. Rather, you pass the paths to
your tests to the `elm-test` executable to run just those files, or you run
`elm-test` without arguments which will look for all elm files under `tests/`
and `test/`. (#72)
- `elm-test init` now adds all dependencies from the package `elm-package.json` to the generated `tests/elm-package.json` file. (#68)
- You no longer write a `tests/Main.elm` file. Rather, you pass the paths to your tests to the `elm-test` executable to run just those files, or you run `elm-test` without arguments which will look for all elm files under `tests/` and `test/`. (#72)
- All exposed values of type `Test` are executed by the test-runner.
- Duplicate titles/descriptions fail the test run. (#115)
- Empty describes are no longer allowed. (#95)

### Fixed

- Ignores `elm-stuff` (#100)
- Tests that throw a runtime exception fail with the exception message as
failure, rather than crashing the runner. (#69)
- Tests that throw a runtime exception fail with the exception message as failure, rather than crashing the runner. (#69)

#### Migrating from `0.18.2`

- Upgrade the runner `npm i -g elm-test`
- Remove `tests/Main.elm`
- Remove the dependency on `rtfeldman/node-test-runner` from
`tests/elm-package.json`
- Bump the dependency on `elm-community/elm-test` to `4.2.0 <= v < 5.0.0` in
`tests/elm-package.json`
- Ensure your test files expose each test you want to run, and that those values
are of type `Test`
- Make sure those tests aren't defined twice (for example: once as a top-level
value, and again in a `describe` block) or they will be executed twice.
- Remove the dependency on `rtfeldman/node-test-runner` from `tests/elm-package.json`
- Bump the dependency on `elm-community/elm-test` to `4.2.0 <= v < 5.0.0` in `tests/elm-package.json`
- Ensure your test files expose each test you want to run, and that those values are of type `Test`
- Make sure those tests aren't defined twice (for example: once as a top-level value, and again in a `describe` block) or they will be executed twice.
- run `elm-test` to execute your tests.
194 changes: 135 additions & 59 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,101 +1,175 @@
# node-test-runner [![Version](https://img.shields.io/npm/v/elm-test.svg)](https://www.npmjs.com/package/elm-test)

Runs [elm-test](https://package.elm-lang.org/packages/elm-explorations/test/latest) suites from Node.js
Runs [elm-explorations/test] suites in Node.js.

When people say “elm-test” they usually refer to either:

- This CLI tool for running tests.
- [elm-explorations/test] – an Elm package for defining tests that this CLI tool can run.

[elm-explorations/test]: https://package.elm-lang.org/packages/elm-explorations/test/latest

## Installation

```shell
npm install -g elm-test
```
npm install --save-dev elm-test
```

## Usage
## Quick start

```shell
elm-test install foo/bar # Install the foo/bar package to "test-dependencies"
elm-test init # `elm-test install elm-explorations/test` and create tests/Example.elm
elm-test # Run all exposed Test values in *.elm files in tests/
elm-test Foo.elm # Run all exposed Test values in Foo.elm
```
Install [elm-explorations/test] and create `tests/Example.elm`:

### Command Line Arguments
npx elm-test init

#### `install`
Run tests in the `tests/` folder:

Like `elm install`, except it installs to the `test-dependencies` field of your project's `elm.json` file instead of `dependencies`.
npx elm-test

```shell
elm-test install elm/regex
```
Run tests in one particular file:

#### `init`
npx elm-test tests/Example.elm

Runs `elm-test install elm-explorations/test` and then creates a `tests/Example.elm`
example test to get you started.
Run tests in files matching a [glob](https://github.com/isaacs/node-glob#glob-primer):

Afterwards, you can run `elm-test` with no arguments to try out the example.
npx elm-test "src/**/*Tests.elm"

#### `--compiler`
> Note: The double quotes are important! Without quotes, your shell might expand the globs for you. With quotes, elm-test expands the globs. This way the watcher can pick up new tests matching the globs, and it will work cross-platform.
The `--compiler` flag can be used to use a version of the Elm compiler that
has not been installed globally.
Run in watch mode:

```shell
npm install elm
elm-test --compiler ./node_modules/.bin/elm
```
npx elm-test --watch

#### `--seed`
## Where to put tests

Allow running the tests with a predefined seed, rather than a randomly generated seed. This is especially helpful when trying to reproduce a failing fuzz-test.
### Locating files containing tests

```shell
elm-test --seed=12345
```
There are 3 places you could put your tests:

#### `--fuzz`
1. In the `tests/` folder.

Define how many times a fuzzer should run. Defaults to `100`
This is the default and requires no extra setup.

```shell
elm-test --fuzz=500
```
2. In any source directory (`"source-directories"` in `elm.json` for applications, `src/` for packages) as separate files.

#### `--report`
A convention is to put test files next to the file it tests with a `Tests` suffix. For example, you could have `src/LoginForm.elm` and `src/LoginFormTests.elm`.

Specify which reporter to use for reporting your test results. Valid options are:
This requires telling elm-test which folders/files to run. Examples:

- `console` (default): pretty, human readable formatted output
- `json`: every event will be written to stdout as a json-encoded object
- `junit`: junit-compatible xml will be written to stdout
npx elm-test "src/**/*Tests.elm"
npx elm-test test/frontend/elm

```shell
elm-test --report=json
```
You might also need to configure your editor to understand that the `"test-dependencies"` in your `elm.json` are available in these files.

3. In already existing source files.

This allows testing internal functions without exposing them. (Be aware that testing implementation details can sometimes be counter-productive.)

This requires moving everything in `"test-dependencies"` in your `elm.json` into regular `"dependencies"`, so your project still compiles. This also helps your editor. Note that this approach isn’t suitable for packages, since you don’t want your package to unnecessarily depend on [elm-explorations/test].

You can mix all three variants if you want:

#### `--version`
npx elm-test tests "src/**/*Tests.elm" app

Displays the version of the current elm-test.
> In this example, `"src"` and `"app"` need to be in `"source-directories"` in `elm.json`.
```shell
$ elm-test --version
0.19.1
### Locating tests within files

For elm-test to find tests in your files you need to:

1. Create top-level values of the type [Test](https://package.elm-lang.org/packages/elm-explorations/test/latest/Test#Test). You can name the values anything – the only thing that matters is that their type is `Test`.
2. Expose them.

Example:

```elm
module LoginForm exposing (alreadyLoggedInTests, tests)

import Test exposing (Test)


tests : Test
tests =
-- ...


alreadyLoggedInTests : Test
alreadyLoggedInTests =
-- ...
```

#### `--watch`
Some prefer to expose a single `Test` value and group everything using [describe](https://package.elm-lang.org/packages/elm-explorations/test/latest/Test#describe). Some prefer to expose several `Test` values.

**Also check out the [elm-explorations/test quick-start](https://github.com/elm-explorations/test#quick-start) guide!**

## Command Line Arguments

These are the most common commands and flags. Run `elm-test --help` for an exhaustive list.

**Note:** Throughout this section, the `npx` prefix is omitted for brevity.

Starts the runner in watch mode. Upon changing any currently watched source
files (either in your your source-directories or in your tests'
source-directories), your tests will get rerun.
### install

```shell
elm-test --watch
Like `elm install`, except elm-test will install to `"test-dependencies"` in your `elm.json` instead of to `"dependencies"`.

elm-test install elm/regex

### init

Runs `elm-test install elm-explorations/test` and then creates a `tests/Example.elm` example test to get you started.

`elm-test init` requires an `elm.json` file up the directory tree, so you will need to run `elm init` first if you don’t already have one.

After initializing elm-test in your project, try out the example by running `elm-test` with no arguments.

elm init
elm-test init
elm-test

### --watch

Start the runner in watch mode. Your tests will automatically rerun whenever your project changes.

elm-test --watch

### --seed

Run with a specific fuzzer seed, rather than a randomly generated seed. This allows reproducing a failing fuzz-test. The command needed to reproduce (including the `--seed` flag) is printed after each test run. Copy, paste and run it!

elm-test --seed 336948560956134

### --fuzz

Define how many times each fuzz-test should run. Defaults to `100`.

elm-test --fuzz 500

### --report

Specify which format to use for reporting test results. Valid options are:

- `console` (default): pretty, human readable formatted output.
- `json`: newline-delimited json with an object for each event.
- `junit`: junit-compatible xml.

```
elm-test --report json
```

#### `--help`
### --compiler

Displays all the available options and commands.
If `elm` is _not_ in your `$PATH` when elm-test runs, or the Elm executable is called something other than `elm`, you can use this flag to point to your installation.

### Travis CI
elm-test --compiler /path/to/elm

To run a tool installed locally using `npm` you can use `npx`:

npx elm-test

`npx` adds the local `node_modules/.bin/` folder to `$PATH` when it executes the command passed to it. This means that if you have installed `elm` locally, `elm-test` will automatically find that local installation.

As mentioned in [Installation](#installation) we recommend installing elm-test locally in every project. This ensures all contributors and CI use the same version, to avoid nasty “works on my computer” issues.

## Travis CI

If you want to run your tests on Travis CI, [here's a good starter `.travis.yml`](https://docs.travis-ci.com/user/languages/elm/):

Expand All @@ -104,3 +178,5 @@ language: elm
elm:
- 0.19.1
```
Here is an example [`travis.yml`](https://github.com/rtfeldman/elm-css/blob/master/.travis.yml) configuration file for running tests in CI.
20 changes: 10 additions & 10 deletions lib/elm-test.js
Original file line number Diff line number Diff line change
Expand Up @@ -90,41 +90,41 @@ function main() {
.name('elm-test')
.usage('[options] [globs...]')
.description(examples)
.option('--watch', 'Run tests on file changes', false)
// For example `--seed` and `--fuzz` only make sense for the “tests” command
// and could be specified for that command only, but then they won’t show up
// in `--help`.
.option(
'--compiler <path>',
'Use a custom path to an Elm executable (default: elm)',
undefined
)
.addOption(
new Option('--seed <int>', 'Run with a previous fuzzer seed')
new Option('--seed <int>', 'Run with a specific fuzzer seed')
.default(Math.floor(Math.random() * 407199254740991) + 1000, 'random')
.argParser(parsePositiveInteger)
)
.option(
'--fuzz <int>',
'Run with each fuzz test performing this many iterations',
'Define how many times each fuzz-test should run',
parsePositiveInteger,
100
)
.addOption(
new Option(
'--report <format>',
'Print results to stdout in the given format'
'Specify which format to use for reporting test results'
)
.default('console')
.choices(Report.all)
)
.option('--watch', 'Run tests on file changes', false)
.option(
'--compiler <path>',
'Use a custom path to an Elm executable (default: elm)',
undefined
)
.version(packageInfo.version, '--version', 'Print version and exit')
.helpOption('-h, --help', 'Show help')
.addHelpCommand('help [command]', 'Show help');

program
.command('init')
.description('Create example tests')
.description('Install elm-explorations/test and create tests/Example.elm')
.action(() => {
const options = program.opts();
const pathToElmBinary = getPathToElmBinary(options.compiler);
Expand Down
1 change: 1 addition & 0 deletions prettier.config.js
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
module.exports = {
singleQuote: true,
proseWrap: 'never',
overrides: [
{
files: '*.js',
Expand Down

0 comments on commit 995b916

Please sign in to comment.