Concurrency limit

[wnr]

Is there any way of setting the concurrency (parallel) limit when testing with sauce? Because right now I'm unable to
test more than 3 browsers because I'm hitting my parallel cap of my open sauce account.

Have a look at https://github.com/axemclion/grunt-saucelabs which have the concurrency option.

[wnr]

@karma-runner will you have a look at this?

[rkistner]

In my project I simply define multiple grunt tasks, and run them sequentially:

karma: {
        sauce1: {
            configFile: 'karma-sauce.conf.js',
            browsers: ['SL_IE_9', 'SL_IE_10']
        },
        sauce2: {
            configFile: 'karma-sauce.conf.js',
            browsers: ['SL_Chrome', 'SL_Safari']
        }
}

I also use a solution similar to this answer to always run the tests in all browsers, even if tests in the first one fail.

Of course, this only works when using singleRun: true.

My full config: Gruntfile.js and karma.conf.js.

[wnr]

@rkistner thanks alot! That should solve my problems. Smart to use multiple grunt tasks, didn't think of that :)

[wnr]

@rkistner do you also have a solution to mark browser runs passed/failed perhaps?

[rkistner]

@wnr Unfotunately I don't. I'll log it as a separate issue.

[wnr]

@rkistner okay cool :)

[rkistner]

This was fixed by karma-runner/karma#57, which was released in karma 0.11.0 (npm install karma@canary).

No need to specify the concurrency limit - the first browsers start running the tests immediately, and the next batch are queued and run as soon as the first ones have finished.

[wnr]

Oh okay, cool :) Thanks a bunch

[kimmobrunfeldt]

Thanks @rkistner for sharing your setup! I ended up doing the same thing but also in a way that separate karma tasks are created dynamically of given sauce lab browser setup: karma.conf.js, saucelabs-browsers.js, Gruntfile.js check also sauce task.

[sgarbesi]

@rkistner Expansion using caolan's async library.

    grunt.registerTask('tests', function() {
        var done = this.async();
        var series = [];
        var success = true;
        var tasks = {};

        Object.keys(grunt.config.data.karma).forEach(function(key) {
            if (key === 'options') {
                return;
            }

            var task = 'karma:' + key;
            tasks[task] = 0;

            series.push(function(callback) {
                grunt.util.spawn({
                    args: [task],
                    grunt: true,

                    opts: {
                        stdio: 'inherit'
                    }
                }, function(error, result, code) {
                    tasks[task] = code;

                    if (code !== 0) {
                        success = false;
                    }

                    callback();
                });
            });
        });

        async.parallelLimit(series, 1, function() {
            done(success);
        });
    });

[sgarbesi]

@kimmobrunfeldt This doesn't actually flag the browser as having failed in saucelabs. Did you manage to do anything further to get that to work?

[joscha]

see #40 and karma-runner/karma#1646.