Using multiple reporters with Mocha when testing a Node.js application

Motivation

Using one reporter is enough in development, but it can be a real pain when running tests from a continous test server.

When using Mocha from the command line to test a Node.js application, having only one reporter output the tests results is usually the best solution. In this situation, developers just need to know at a glance which tests failed and which tests passed for the latest tests run, and they usually don’t need to save this information. Here’s a contrived example of how Mocha outputs tests results on the command line:

$ mocha -R spec
  mocha-sample
       should succeed for true == true
    1) should fail for true == false
  1 passing (21 ms)
  1 failing
  1) mocha-sample should fail for true == false:
     AssertionError: expected true to be false
$

Here we’re using a single reporter named spec. Having more reporters would usually not add any useful information, and it would probably confuse the user. However, there are other situations where having more than one reporter can be very useful.

For instance, let’s consider that we’re using a continuous testing server to run some tests using mocha. If we’re using Atlassian Bamboo, or any other continuous testing server that uses the XUnit XML file format to parse tests results, we’ll need to use the xunit-file reporter to display tests results on the continous testing server’s dashboard.

Now, what if we’d like to send our test results to a CouchDB database so that we can perform some analytics on the same tests results later on? We would need to output our tests results as a JSON file too. The problem is, Mocha doesn’t support this out of the box: we will need to run our tests twice if you want to output tests results in two different formats, and more generally as many times as the number of output formats we’ll need.

This is an issue for at least two reasons:

Existing solutions

Fortunately, several solutions exist that provide multiple reporters support in Mocha.

Mocha-multi, a Mocha reporters multiplexer

Glen Mailer’s mocha-multi is a Mocha reporter that provides support for multiple reporters. Although it’s described as “a bit of a hack to get multiple reporters working with mocha”, it works well.

Just install mocha-multi:

$ npm install mocha-multi

Don’t forget to use –save or –save-dev according to your needs and set the multi environment variable to the list of reporters you want to use, separated by a space:

$ ls
node_modules  package.json  test
$ multi='xunit=xunit.xml json=report.json' ./node_modules/mocha/bin/_mocha -R mocha-multi
$ ls
report.json  node_modules  package.json  test  xunit.xml
$

The value of the multi environment variable should follow a syntax that is easy to understand: ‘reporter-name=output reporter2-name=output …’, where output can be either - (a dash) for standard output and error, or a filename to output the reporter’s output to this file.

The fact that it’s implemented as a Mocha reporter is very interesting, since it doesn’t require changes to Mocha. It’s also very easy to install using npm and doesn’t require to wait for the next Mocha release to be used. However, the implementation seems to use a few hacks that could break interoperability with Mocha and other reporters in the future.

Multiple reporters in Mocha’s core

I recently submitted a pull-request that implements multiple reporters support within Mocha, without the need to use an external module. In order to use this solution, you’ll need to:

Here’s how it’s done:

$ ls
node_modules  package.json  test
$ cat package.json
{
  "name": "mocha-sample",
  "version": "0.0.0",
  "description": "Sample project using mocha",
  "main": "index.js",
  "scripts": {
    "test": "mocha"
  },
  "author": "Julien Gilli",
  "license": "BSD-2-Clause",
  "devDependencies": {
    "chai": "~1.8.1",
    "mocha": "git+https://github.com/misterdjules/mocha.git#multiple_reporters_support"
  }
}
$ ./node_modules/mocha/bin/_mocha -R xunit-file,mocha-json-file-reporter 2>&1 > /dev/null
$ ls
node_modules  package.json  report.json  test  xunit.xml
$

Here, we use two reporters: xunit-file and mocha-json-file-reporter.

One of the drawbacks of this solution, from which mocha-multi doesn’t suffer, is that it mixes output from reporters who output directly to the standard output or the standard error output. That’s the reason why, in our example above, we used two reporters that output their results to files, and not to standard error and output.

The fact that it’s implemented within Mocha requires approval from Mocha’s development team and will require users to upgrade Mocha if it ever gets merged upstream.

According to Mocha’s author @visionmedia, this pull-request could be merged “after the other issue to place all streams ontop of a streaming json thing, so they all just become consumers”. According to @travisjeffery, it means that either or both of issues #492 and #897 need to be fixed before moving on. Since it’s not clear how these issues should be fixed, I wouldn’t hold my breadth until it’s merged.

I will personally continue using mocha-multi and I will happily assist anyone willing to get my pull request merged upstream. I hope you find at least one of these two solutions helpful, and I invite you to post comments below if you have any question or comment.

Julien Gilli 14 December 2013
blog comments powered by Disqus