javascript - Why does SauceLabs say my QUnit Test Fails when its clearly passing? -

we trying use saucelabs verify our browser-based qunit tests passing in popular device/browser combinations ...

the tests pass when @ them in browser: https://ordenado.herokuapp.com/

but reason saucelabs telling "failed"...

see: https://saucelabs.com/tests/5b0f07813a7f4934bb44b07606ea2fd5

curl command used

for reference, used following curl command:

curl https://saucelabs.com/rest/v1/ordem/js-tests \ -x post \ -u ordem:secret_key \ -h 'content-type: application/json' \ --data '{     "platforms": [       ["windows 8.1", "internet explorer", "11"],       ["windows 8", "internet explorer", "10"],       ["windows 8.1", "firefox", "beta"],       ["windows 8", "firefox", "37"],       ["windows 7", "firefox", "32"],       ["os x 10.8", "safari", "6"],       ["os x 10.8", "chrome", "37"],       ["linux", "chrome", "30"],       ["linux", "firefox", "dev"],       ["os x 10.10","iphone", "7.0"],       ["os x 10.10","iphone", "8.2"],       ["os x 10.10","ipad", "7.0"],       ["os x 10.10","ipad", "8.2"]     ],     "url": "https://qunit.herokuapp.com/test/test.html?coverage=true",     "framework": "qunit",     "name":"ordem",     "public": "public",     "build": "build-007" }' 

also, else ever experience following error:

"the sauce vms failed start browser or device"

visit: https://saucelabs.com/u/ordem complete list of tests. click on of ones sauce claims "failed" , watch video see tests passing!!

any insight appreciated!

you should add several hooks report qunit test results saucelabs.

there an article how started qunit. there mentioned repository example following code snippet:

var log = []; qunit.done = function (test_results) {   var tests = log.map(function(details){     return {       name: details.name,       result: details.result,       expected: details.expected,       actual: details.actual,       source: details.source     }   });   test_results.tests = tests;    // delaying results bit cause in real-world   // scenario won't them   settimeout(function () { window.global_test_results = test_results; }, 2000); }; qunit.teststart(function(testdetails){   qunit.log = function(details){     if (!details.result) {      details.name = testdetails.name;      log.push(details);     }  } }); 

add code before tests , results reported properly