Automated Unit Testing Problem

DonnieHolmes · ‎10-10-2013

I've managed to get unit testing up and running in my project using a combination of mocha, chai, and sinon (our company standard JS testing libraries). This all works perfectly fine when launching unit tests from within a regular browser. However, things go awry when i try to run these tests from within a headless browser (PhantomJS using mocha-phantomjs). In this case, i get the following error:

ReferenceError: Can't find variable: require

Here's what I'm doing to run the tests:

<html>     <head>         <title>Unit tests</title>                                 <link href="../../node_modules/mocha/mocha.css" rel="stylesheet" type="text/css" />                             </head>     <body>         <script src="../../config/dojo.test.config.js"></script>          <script src="//js.arcgis.com/3.7compact/"></script>          <div id="mocha"></div>                 <script>             require(["../../node_modules/chai/chai.js",                     "../../node_modules/mocha/mocha.js",                     "../../node_modules/sinon/lib/sinon.js"], function (chai) {                         mocha.setup('bdd');                         mocha.reporter('html');                         expect = chai.expect;                         require(["test/unit/MapApp"], function () {                             if (window.mochaPhantomJS) {                                 mochaPhantomJS.run();                             } else {                                 mocha.run();                             }                         });                     });                     </script>     </body> </html>

Why are the testing libraries pulled in through a require? Originally I had chai, mocha, and sinon referenced in script tags, but I kept getting multipleDefine errors from the dojoLoader. The dojo documentation says this about the error:

[INDENT]AMD define was called referencing a module that has already been defined. The most common cause of this problem is loading modules via <script> elements in the HTML document. Use the loader; don't use <script> elements. The second most common cause is passing explicit module identifiers to define; don't do this either.[/INDENT]

After moving these scripts to a require call, everything worked fine in IE, FF, Chrome, etc.

It seems like something very fundamental is going wrong here since require is undefined when the code is run from phantomjs. Does anyone have any experience with running automated unit tests using this setup? Any suggestions from the ESRI folks? I'd be open to trying another solution that works w/ mocha. Thanks.

///Donnie

DonnieHolmes · ‎10-11-2013

Doh! I finally figured out what was going on here. I left out a key detail to this flow... I'm using VS2012 for my IDE, and I was accessing the test runner through my browser while I had the site running in debug mode.

I guess that it wasn't obvious to me from the mocha-phantomjs examples that I needed to access the runner through a web server and not simply through the local file system (which is what I was doing). When I point mocha-phantomjs to the test runner debug site everything works fine.

Thanks very much for the examples. I'm going to use those to get started on setting up my project to work with npm, grunt, and dojo builds.

View solution in original post

ReneRubalcava · ‎10-10-2013

Here is what I've used in my specrunner.html
I think you need to do the mocha set before the API loads, then when it's loaded, you can start mocha.

<div id="mocha"></div>
<div id="map"></div>
<script type='text/javascript' src='//js.arcgis.com/3.7'></script>
<script type="text/javascript" src="node_modules/mocha/mocha.js"></script>
<script type="text/javascript" src="node_modules/chai/chai.js"></script>
<script type='text/javascript' src='lib/sinon-1.7.3.js'></script>
<!-- source files -->
<script type="text/javascript" src="spec/main.js"></script>

<script>
    var dojoConfig = {
        isDebug: false,
        has: {'dojo-undef-api': true}
    };

    mocha.setup('bdd');
    mocha.ignoreLeaks();
</script>

The in main.js I'm able to use require() to setup my config.

(function () {

    /** packages and configuration stuff **/

    /**
     * Method same as with requirejs
     * http://www.bennadel.com/blog/2393-Writing-My-First-Unit-Tests-With-Jasmine-And-RequireJS.htm
     **/
    require([
        'require',
        'dojo/ready'
    ], function (require, ready) {
        ready(function () {
            require([
            ], function () {
                (window.mochaPhantomJS || window.mocha).run();
            });
        });
    });
})(this);

I got help setting this up from the AGRC guys repo.
https://github.com/agrc/AGRCJavaScriptProjectBoilerPlate/tree/master/src/app/tests

They even figured out how to build StubModule to stub our Dojo/Esri modules for testing, which is pretty cool.
https://github.com/agrc/StubModule

That's why in the dojoConfig, I set up

has: {'dojo-undef-api': true}

This allows the stubbing of Dojo modules.

When the tests run, I would still get a multipledefine error, but it was ignored and tests still worked. I run these using Grunt if that matters. Similar methods should work for Jasmine too.

ReneRubalcava · ‎10-10-2013

I go back and forth using mocha/chai/sinon or jasmine for testing with PhantomJS.
I've gotten that multiple define error when a test runs before, but just ignored it and it worked fine. I don't get the require not defined error though.

My specrunner.html looks like below, where ArcGIS JS API is first script tag and mocha/chai/sinon added just before these lines.

<script type="text/javascript" src="spec/main.js"></script>

<script>
    var dojoConfig = {
        isDebug: false,
        isJasmineTestRunner: true, // prevents parser in main.js from running
        has: {'dojo-undef-api': true}
    };

    mocha.setup('bdd');
    mocha.ignoreLeaks();
</script>

This didn't cause any issues that I could remember. In my main.js I use this for my require if it matters.

    /**
     * Method same as with requirejs
     * http://www.bennadel.com/blog/2393-Writing-My-First-Unit-Tests-With-Jasmine-And-RequireJS.htm
     **/
    require([
        'require',
        'dojo/ready'
    ], function (require, ready) {

        ready(function () {
            require([
                /* helper modules */
                'spec/js/helpers/sampleHelperSpec'
            ], function () {
                (window.mochaPhantomJS || window.mocha).run();
            });

        });

    });

Heck, I think I'll go back now and load all the libraries from the require statement and see what happens. Like I said, running my tests from Grunt, I ocassionally get the multipledefine error, but it doesn't seem to break my tests.

I got a lot of help from the AGRC repo as they use Jasmine with for headless testing with Grunt without much issue.
https://github.com/agrc/AGRCJavaScriptProjectBoilerPlate/tree/master/src/app/tests

Here's a blog post from Dave Bouwman using Jasmine for his headless testing of the API at Esri.
http://blog.davebouwman.com/2013/07/26/automated-headless-unit-tests-with-esri-js-api/

Maybe, and I can't say for sure, but looking at your require() statement.

require(["../../node_modules/chai/chai.js",
                    "../../node_modules/mocha/mocha.js",
                    "../../node_modules/sinon/lib/sinon.js"], function (chai) {
                        mocha.setup('bdd');
                        mocha.reporter('html');
                        expect = chai.expect;
                        require(["test/unit/MapApp"], function () {
                            if (window.mochaPhantomJS) {
                                mochaPhantomJS.run();
                            } else {
                                mocha.run();
                            }
                        });
                    });

You may want to move your unit test into a define() in another module required from you're config. This way you can load another context sensitive require() to load your tests. Context sensitive require() is useful for conditional loading of modules, like the tests. That's a big maybe, I can't say for sure.
http://dojotoolkit.org/reference-guide/1.9/loader/amd.html#context-sensitive-require

ReneRubalcava · ‎10-10-2013

I'm guessing your config is setting up your packages with a require(). You're loading that file before the AGSJS API in your script tags.

<script src="../../config/dojo.test.config.js"></script> 
<script src="//js.arcgis.com/3.7compact/"></script>

Try flipping the order.

Technically, there's no guarantee of order with script tags, but with so few being used on the page, I'd look at that possibility.

DonnieHolmes · ‎10-10-2013

Thanks for all the info Rene!

I tried to flip the order like you suggested and it broke (the working in-browser version) because it couldn't find a module that was listed in the config. I'm guessing that's because the config gets loaded after dojo and doesn't get applied?

For full disclosure, here are the contents of the rest of the files...

dojoConfig:

var dojoConfig = {
    paths: {
        'jscore': location.pathname.replace(/\/[^/]+$/, '') + '/../../node_modules/jscore',
        'widgets': location.pathname.replace(/\/[^/]+$/, '') + '/../../node_modules/widgets',
        'text': location.pathname.replace(/\/[^/]+$/, '') + '/../../node_modules/jscore/require/text',
        'styles': location.pathname.replace(/\/[^/]+$/, '') + '/../../node_modules/jscore/require/styles',
        'template': location.pathname.replace(/\/[^/]+$/, '') + '/../../node_modules/jscore/require/template',
        'mapapp': location.pathname.replace(/\/[^/]+$/, '') + '/../../src/mapapp',
        'test': location.pathname.replace(/\/[^/]+$/, '') + '/../../test'
    }
};

test module:

define(["mapapp/MapApp"], function (MapApp) {
    'use strict';
    describe("MapApp", function () {

        it("MapApp should be defined", function () {
            expect(MapApp).not.to.be.undefined;
        });
    });
});

Runs just fine in a browser, just not in PhantomJS...

DonnieHolmes · ‎10-11-2013

Doh! I finally figured out what was going on here. I left out a key detail to this flow... I'm using VS2012 for my IDE, and I was accessing the test runner through my browser while I had the site running in debug mode.

I guess that it wasn't obvious to me from the mocha-phantomjs examples that I needed to access the runner through a web server and not simply through the local file system (which is what I was doing). When I point mocha-phantomjs to the test runner debug site everything works fine.

Thanks very much for the examples. I'm going to use those to get started on setting up my project to work with npm, grunt, and dojo builds.