One of the reasons that I prefer JsTestDriver over other JavaScript testing frameworks, is that the tests are made on real browsers, and allows me to test on all the major browsers that consumes my application, and verify compatibility issues with older versions of IE for example.
If you prefer to run a complete headless JavaScript testing server, perhaps JsTestDriver is not your best choice. I think CasperJS is a very good alternative to achieve that. It uses PhantomJS, a scriptable headless WebKit engine to run the tests.
In my case, I have a complete headless server running an instance of JsTestDriver, and Other server whit a couple of virtual machines, with common SO/Browsers combinations (Windows/IE7; Windows/IE8; Linux/Firefox...), which have actual browsers connected to the headless JsTestDriver server. But if you don't need to use real browsers, the CasperJS alternative may be a good choice.