Transcript

transcript is a CLI tool for snapshot testing other CLI tools.

TODO: Video here.

Usage

Automatically record a shell session or type-out a transcript file by hand, then use the check command!

cat > demo.cmdt <<EOF
$ echo stdout
1 stdout

$ echo stderr 1>&2
2 stderr

# Non-zero exit codes.
$ false
? 1
EOF

transcript check ./demo.cmdt

Install

go get -u github.com/deref/transcript

NOTE: Transcript is not Go-specific. It is simply written in Go and Go provides a convenient distribution mechanism. If there is expressed interest, it may be re-packaged for various additional distribution channels.

Record

Initial authoring of tests is performed in an interactive shell.

To record an interactive session to a file, run:

transcript shell -o example.cmdt

The interactive shell supports standard readline behaviors and can be exited with ^d or exit like most other shells.

Check

To interpret a transcript file and validate that the results (stdio output and exit codes) have not changed, run the following:

transcript check example.cmdt

Check returns a non-zero exit code if any check failures or other errors occur.

Update

When the CLI tools under test are modified, the quickest way to update test files is to use the automated update process:

transcript update example.cmdt

This will interpret a command transcript file, but does not check any output or exit status expectations. Instead, the given file will be rewritten with the newly observed results.

Edit

While transcript files can be edited by hand, more advanced edits can be made using an interactive update session. The experience should be familiar to users of git rebase --interactive.

NOTE: Not yet implemented.

"Command Transcript" File Format

Transcript files represent recorded shell sessions.

.cmdt is the recommended file extension.

This format is intended to be human-editable, but sacrifices some ease of hand-authoring in exchange for added functionality. Users are expected to primarily use the transcript tool to create and update transcripts.

Structure

Cmdt files are line-oriented. Each line represents an instruction to the Transcript interpreter. Each instruction begins with an opcode, followed by a space. The remainder of an instruction line forms arguments to the operation specified by the opcode.

Operations

Operations with the following opcodes are supported:

# — comment

Comments may appear anywhere in a .cmdt file and are ignored by the interpreter.

A space is not required after the # opcode.

Blank lines are also treated as comments.

$ — command

Run a shell command.

Supports the subset of Bash syntax provided by mvdan/sh.

1, 2 — output

Match a line of output from a particular stdio stream of the previously run command.

The opcodes are named after the file descriptors of stdout (1) and stderr (2) respectively.

Output lines are matched exactly. More flexible matching may be configured by % directives in a future version of Transcript.

Transcript checking assumes that the interleaving of stdout and stderr lines is significant and that output lines are written atomically. The ordering of concurrent writes to both streams is undefined, which will lead to flakey tests. Incrementally written lines will be buffered, which may mask text interleaving issues that would affect users. Both of these shortcomings may be mitigated in the future.

? — exit-code

Exit code of the previously run command.

If omitted, the exit code defaults to 0.

% — directive

Reserved for future use by Transcript.

Go API

In addition to the transcript CLI, there is a Go API for users who wish to embed cmdt scripts in to their existing Go test suites.

import (
  _ "embed"

  "github.com/deref/transcript/cmdtest"
)

//go:embed test.cmdt
var fs embed.FS

func TestCLI(t *testing.T) {
  f, _ := fs.Open("test.cmdt")
  defer f.Close()
  cmdtest.Check(f)
}

NOTE: Assuming that ./test.cmdt uses the CLI tool you are developing, you must first build your tool and ensure it is on PATH.

There is also a CheckString function for small, inline tests. However, prefer to use .cmdt files so that the transcript tool can assist with updates and edits.