Running Jasmine unit tests in your Visual Studio Online build

I couldn’t find thorough enough tutorial of running Jasmine tests in Visual Studio Online build as part of my continuous integration workflow. I struggled for hours to get this work and decided to write a step-by-step guide about how to make it work.

Install Chutzpah NuGet

Chutzpah is an open source JavaScript test runner which enables you to run unit tests using QUnit, Jasmine, Mocha, CoffeeScript and TypeScript. You can use it with Visual Studio’s built-in Test Explorer by installing Chutzpah Test Adapter for the Test Explorer extension.

Visual Studio Online (VSO) however needs the test adapter in your build sources and that’s why you need to install Chutzpah NuGet to your project:

PM> Install-Package Chutzpah  

After installing you should see Chutzpah.VS2012.TestAdapter.dll (amongst other files) in your packages/Chutzpah.[version]/tools folder. This is the file that we will need for VSO to run the tests using custom test adapter.

Note! I used version 4.1.0 of Chutzpah at the moment of writing.

Configure Chutzpah for your project

Here is a sample of the folder structure I have in my ASP.NET MVC 5 + AngularJS project.

Sample project folder structure

So I have written my application in TypeScript but my test file is pure JavaScript just to be more general with this tutorial.

In app/spec folder I have a file called chutzpah.json. This is where I configure my tests and references. Here are the contents of it:

{
    "Tests": [
        {
            "Path": "",
            "Include": "*/*Spec.js"
        }
    ],
    "References": [
        { "Path": "../../Scripts/angular.js" },
        { "Path": "../../Scripts/angular-mocks.js" },
        { "Path": "../../Scripts/lodash.js" },
        { "Path": "../application.js" },
        {
            "Path": "../controllers",
            "Include": "*.js"
        },
        {
            "Path": "../services",
            "Include": "*.js"
        },
        {
            "Path": "../directives",
            "Include": "*.js"
        }
    ]
}

So I’m simply telling chutzpah to look for tests in files that are in any subfolder and are ending “Spec.js”. I have referenced the relevant libraries and all of my controllers, services and directives and of course application.js.

Important! Do not add reference to jasmine.js and do not use /// <reference path="[anything].js" /> in your test files. This might cause Chutzpah not to find your tests without a reasonable error message.

Write Jasmine tests

I’ll just copy and paste my controller and test file for you to bootstrap your tests.

MyController.ts
module MyApplication {  
    "use strict";

    export class MyController {
        static $inject = [
            "$scope"
        ];

        constructor(private $scope) {
            $scope.vm = this;
        }

        checkSsn(ssn: string) {
            const ssnPattern = /^[0-9]{6}\-?[0-9A-Z]{4}$/;
            return ssnPattern.test(ssn);
        }
    }

    // register controller
    angular.module("app").controller("MyController", MyController);
}

If you are not familiar with TypeScript, no problem. The only relevant part here is the checkSsn-function to which I have written my unit tests for.

MyControllerSpec.js
"use strict";

describe("Controller: MyController", function () {

    beforeEach(function () {
        angular.module("ngAnimate", []);
        angular.module("ui.router", []);
        angular.module("ui.bootstrap", []);
        angular.module("ui.calendar", []);
        angular.module("toastr", []);
        angular.module("ngTable", []);
        angular.module("isteven-multi-select", []);
        angular.mock.module("app");
    });

    var myController, scope;

    beforeEach(angular.mock.inject(function ($rootScope) {
        scope = $rootScope.$new();
        myController = new MyApplication.MyController(scope);
    }));

    it("should validate 123456-1234 as valid SSN", function () {
        expect(myController.checkSsn("123456-1234")).toBeTruthy();
    });

    it("should validate 123456-123 as invalid SSN", function () {
        expect(myController.checkSsn("123456-123")).toBeFalsy();
    });

});

In the first beforeEach block I have mocked my app with its dependencies. In the second one I have injected $rootScope service and used it to initialize a new MyController.
Tests I wrote are testing if given values are valid Finnish social security numbers.

Run your tests

If you installed Chutzpah Test Adapter for the Test Explorer, you should see your tests in Visual Studio’s Test Explorer.

Visual Studio Online runs tests with vstest.console.exe. You might want to use it yourself too in case you run into weird problems (which I did). It’s annoying to run VSO builds over and over again to see if your tests run correctly.

Command line:

C:\>"C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" "C:\...\ProjectName\app\spec\MyControllerSpec.js" /TestAdapterPath:"C:\...\packages"  

VsTest will search recursively for *.TestAdapter.dll in given path and run the tests using it.

Visual Studio Online build definition

Go to Visual Studio Online and create a new build definition. Add new build step type of Visual Studio Test.

VSO Build definition

Give the path to your tests to the Test Assembly field. I used **\spec\**\*Spec.js to match all my test files.
Open the Advanced section and write $(Build.SourcesDirectory)\packages to Path to Custom Test Adapters.

Check that the mappings in Repository tab are correct and try running it.

VSO jasmine test run

Summary

For me the biggest challenge was finding the right Chutzpah configuration so that the test adapter finds my tests. In short: do not use /// <reference path="...js" /> in your test files and do not reference to jasmine.js in your chutzpah.json file.

The trick in the eyes of VSO is to install Chutzpah NuGet to your project and define a path to custom test adapter like this: $(Build.SourcesDirectory)\packages. VsTest will search recursively for Chutzpah test adapter dll and uses it to run your jasmine tests.