Examples as tests

When writing a library, examples are often the best way to explain how it should be used. The problem with examples is that it is time consuming to keep them up to date, and non-working examples are a surefire way to annoy potential users.

For our framework Virgilio, I wanted to do better. Lets make our examples test cases, run them on every build, and we can be sure they always work. As a bonus, we'll get free tests. Nice. In theory this sounds great, but the harness test runners require will distract from what the examples aim to demonstrate. Wouldn't it be nice if we could write the example in a natural way and then parse it into a form that can be fed to mocha?

For this purpose specifically I wrote gulp-exemplary. It reads a very basic example format and outputs QUnit style test cases. For instance a console.log in the example, demonstrating the value of a variable at a certain point in the code, becomes an assertion in the test case:

console.log(typeof 42); //=> 'number'
// ... turns into ...
assert.deepEqual(typeof 42, 'number');

Start of line comments are interpreted as descriptions of test cases below. To support asynchronous code we're considering a test case done when the bottom most console.log (or assertion) in the test case is passed. It's all very simple, but so far it seems to cover our needs in the example writing department.

Virgilio has an examples directory which contains examples demonstrating every aspect of its functionality. Users can try them out, running them with the node command, using them as a starting point for experimenting with the framework. And because I test them as part of the build process, I know they'll work.