recorder: A Go repository from irfansharif

Recorder

Recorder is a handy little library that lets you record a set of operations and then replay from the recordings, which provides a handy way to mock out components being recorded. Operations are defined only by a 'command' and a corresponding 'output'; see grammar below. Recorders have very real ergonomic advantages:

Mocked data is easily auto-generated by simply "capturing the real thing"
Recordings are human readable, making it easy to git-diff behavioural changes
There's no need to litter the codebase with testable-but-clunky interfaces

This library is similar in spirit to cockroachdb/copyist. For real world usage, see cockroachdb/dev where it's used to mock out all attempts to shell out (through exec.Command). It also intercepts all OS operations (os.{Mkdir,Remove,Symlink}). (Also see example/.)

Users will typically want to embed a recorder into structs (think drivers) that oversee the sort of side-effect or I/O they'd like to capture and later playback from (as opposed to "doing the real thing" in tests). These side-effects can be arbitrary. If we're building a CLI that calls into the filesystem to filter for a set of files and writes out their contents to a zip file, the I/O could be the listing out for files, and the side-effects would include creating the zip file.

I/O could also be calling into anything that sits outside some package boundary. The recorder, if embedded into the package, lets us:

Record the set of outbound calls, and the relevant responses, while "doing the real thing"
Play back from an earlier recording, intercepting all outbound calls and effectively mocking out all dependencies the component has

Example

Let us try and mock out a globber. Broadly what it could look like is as follows:

type globber struct {
	*recorder.Recorder
}

// glob returns the names of all files matching the given pattern.
func (g *globber) glob(pattern string) []string {
	capture, _ := g.Next(pattern, func() (string, error) {
		matches, _ := filepath.Glob(pattern) // do the real thing
		capture := fmt.Sprintf("%s\n", strings.Join(matches, "\n"))
		return capture, nil
	})

	matches := strings.Split(strings.TrimSpace(output), "\n")
	return matches
}

All we had to do was define tiny bi-directional parsers to convert our input and output to the human-readable string form Recorders understand. Strung together. we can build tests that would plumb in Recorders with the right mode and play back from them when asked for. See example/ for this test pattern, where it behaves differently depending on whether -record is specified.

$ go test -run TestExample -record
PASS
ok      github.com/irfansharif/recorder/example 0.708s

$ cat testdata/recording
testdata/files/*
----
testdata/files/aaa
testdata/files/aab
testdata/files/aac

$ go test -run TestExample # also much faster!
PASS
ok      github.com/irfansharif/recorder/example 0.097s

When playing back from it, we wouldn't actually need to reach into the file-system. The results from an earlier run were already recorded; we'd just use that instead.

Once the recordings are captured, they can be edited and maintained by hand. An example of where we might want to do that is for recordings for commands that generate copious amounts of output (like fetching from some API). It suffices for us to trim the recording down by hand, and make sure we don't re-record over it (by inspecting the diffs during review). Recordings, like regular mocks, are expected to get checked in as fixtures.

Usage pseudo-code

func (d *driver) f(input interface{}) (interface{}, error) {
    command := // ... string representation of input
    capture, _ := d.Next(command, func() (string, error) {
        output, _ := // ... do the real thing f was supposed to
        capture := // ... string representation of output
        return capture, nil
    })

    output := // ... reconstruct output from its string representation
    return output, nil
}

Grammar

The printed form of an operation (the base unit of what can be recorded) is defined by the following grammar. This is what's used when generating/reading from recording files.

# comment
<command> \
<that wraps over onto the next line>
----
<output>

By default <output> cannot contain blank lines. This alternative syntax allows for it:

<command>
----
----
<output>

<more output>
----
----

Callers are free to use <output> to model external errors as well; it's all opaque to Recorders. The syntax was borrowed from cockroachdb/datadriven.

Contributing

To run fuzz tests:

# Get the pre-requisite binaries.
$ go get -u github.com/dvyukov/go-fuzz/go-fuzz \
  github.com/dvyukov/go-fuzz/go-fuzz-build

# Build the test program.
$ go-fuzz-build

# Run the fuzzer.
$ go-fuzz
2021/03/12 11:51:30 workers: 16, corpus: 2 (3s ago), crashers: 0, restarts: 1/0, execs: 0 (0/sec), cover: 0, uptime: 3s
2021/03/12 11:51:33 workers: 16, corpus: 96 (0s ago), crashers: 0, restarts: 1/0, execs: 0 (0/sec), cover: 279, uptime: 6s
2021/03/12 11:51:36 workers: 16, corpus: 110 (0s ago), crashers: 0, restarts: 1/64, execs: 7899 (878/sec), cover: 522, uptime: 9s
...

# Clean out fuzz-generated data.
$ git clean -fdx