Using CasperJS, Drush and Jenkins to test Drupal

A few weeks ago, spurred on by some frustrating interactions with Behat and Javascript, I started looking into alternative frameworks for running behavioral tests against Drupal. A little bit of searching led me to this helpful post Behavorial Test For Custom Entity Using CasperJS, which outlines the basics of writing some simple CasperJS tests for a site.

I had never heard of CasperJS before, and if you haven't either you might want to go check out the CasperJS site then come back here. In short, CasperJS has a couple things going for it:

• It's very fast. I haven't run any benchmarking tests but it's clearly much faster than Behat + Selenium
• It's built on top of the excellent headless WebKit, PhantomJS
• Writing tests is easy; it's all in Javascript. Even if you are a JS novice, the documentation for the project is superb and can get you up and running quickly

On the other hand, Behat is a more mature framework and has the advantage of being written in PHP, plus, there is the outstanding Drupal Extension which provides Drush integration and simplifies content and user creation / deletion. For beginners you also have a library of tests to reference in the Drupal.org BDD tests project a.k.a. "Doobie".

With CasperJS, Drush is not available to us. But for our tests to really be useful, we'll need a way to access Drush during the test process, and we'll need a way to automatically create our own users and content and do other setup/teardown tasks. Clearly we don't want to write dozens of steps for submitting user/content creation forms; we want to use Drush. This could be done by placing a list of Drush commands in setup.sh and teardown.sh bash scripts, but this still doesn't solve the problem of accessing Drush in the middle of a test. So how can this be done?

Run Drush in the browser

Drush has a (probably not that well-known) command for running a web server on your machine: drush runserver. It turns out, extending the runserver command to expose Drush as a web service is really easy; instead of serving up a Drupal site with runserver, we will provide interaction with Drush.

EDIT: This is now a full-blown project, Drush REST API over on Drupal.org.

You can try it out for yourself. Grab this composer.json file and place it somewhere on your machine. Then switch to that directory and run composer install; this will download a customized Drush install into vendor (you can see and contribute to the pull request to Drush 7 here). Now run vendor/bin/drush runserver --drush-server; then visit https://localhost:8888/core-status --json and you'll see the output of Drush's core-status command in your browser, in JSON format, it should look something like:

With the drush runserver --drush-server command, any time you pass the --json option to Drush in the browser, you'll get JSON content back that can easily be parsed using Javascript. Commands that don't return output (like drush cc all) will return a JSON encoded success / error message depending on their results. For example, visiting https://localhost:8888/made-up-command will return a 500 error code along with:

This is great because in CasperJS, you can, for example, test creating some content then call drush watchdog-show --format=json to check the logs and see if an email was correctly generated and format - much like you can with Behat.

Even better, this means that now with we can include a setup.js file. CasperJS lets you specify --pre and --post options to include tests that run before and after your main test suite.

So we can do casperjs test --pre=setup.js --post=teardown.js /path/to/tests/dir. Then in our setup.js file, we can have code like:

Tip: give unique names for your test users based on the test that's being run (i.e. "testOnlineHelpAdmin" instead of "testAdmin"). Why? Because with Grunt and the grunt-casperjs plugin, you can run CasperJS tests asynchronously. Having different usernames per test will allow you to avoid collisions with CasperJS trying to simultaneously use the same user account for multiple test scenarios.

Finally, here's an example test - a few simple checks that only users with the Administrator role can access our Drupal site's "Online Help" is below:

Running the test with casperjs test tests/casperjs/online-help.js --pre=tests/casperjs/helpers/setup.js --post=tests/casperjs/helpers/teardown.js --uri=mysite.localhost then results in this output:

Test file: tests/casperjs/tests/online-help.js
# Restrict access to online help to Administrators only
PASS Anonymous user denied access to restricted node.
PASS Login form present on Access Denied page
PASS School Administrator user denied access to restricted node.
PASS Loaded login page.
PASS Admin can access Online Help
PASS Login form not present on handbook page

Putting it all together with Jenkins CI

Behavior driven development is a great software development process. You can spec out a feature, write the tests for it, then write the code to implement the feature. Or you can write tests against an existing set of features to catch regressions.

One thing I've learned, though, is that unless you have a policy for running tests, and enforce that policy, the tests won't get run; they'll eventually become out-dated, and then your tests are no longer useful.

This is where Jenkins comes in. While it's out of the scope of this post to write out how to set up Jenkins, this guide, Drupal Continuous Integration with Jenkins is really helpful and you can be up and running within a few hours. With a little bit of work, I configured a Jenkins instance so that on each commit that's pushed to our git repository, Jenkins will (1) import our @prod database and files to our @test instance on Acquia Dev Cloud (check out this gist for an example python script that will poll Acquia Cloud to see when the DB import is done, so you don't start your tests before the DB import task finishes!), (2) check out the git branch that was committed to in the @test environment, (3) run some setup tasks using Drush, (4) execute the CasperJS tests, and (5) run some teardown tasks. If any part of this fails, e-mails are sent out to the project manager and the committer with details about the failed build, so it can be fixed.

Questions? Ideas on how to do things differently? Please let us know in the comments.