Page MenuHomePhabricator

QUnit test failures are missing from Jenkins short console log
Closed, ResolvedPublic

Description

Compare the short and full log for this failure:
https://integration.wikimedia.org/ci/job/mwext-MultimediaViewer-qunit/766/console
https://integration.wikimedia.org/ci/job/mwext-MultimediaViewer-qunit/766/consoleFull

The short log is useless, it starts just a few lines after the actual error message.

If this is due to only showing the last N characters, then maybe grepping for yellow/red color codes would be a better approach?

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 22 2014, 3:07 AM
bzimport set Reference to bz61843.
bzimport added a subscriber: Unknown Object (MLST).

Moving to CI component and adding Antoine to cc, since he in charge of Jenkins configuration (as far as I know).

We can potentially ask Zuul to point to the /consoleFull URL instead of /console. Another possibility is to make grunt less verbose and only display errors. Thoughts?

Showing errors only would be great, if it can be done reliably.

Krinkle updated the task description. (Show Details)
Krinkle set Security to None.
Krinkle removed a subscriber: Unknown Object (MLST).

The qunit Grunt task is run by Jenkins itself, not part of the test suite in MediaWiki core itself.

It's currently this verbose because the Jenkins job invokes grunt with the --verbose flag.

Difference:

$ grunt qunit --verbose
Initializing
Command-line options: --verbose

Registering Gruntfile tasks.

Initializing config...OK

Running tasks: qunit

Running "qunit" task

Running "qunit:all" (qunit) task
Verifying property qunit.all exists in config...OK
Files: tests/index.html -> all
Options: timeout=5000, inject="node_modules/grunt-contrib-qunit/phantomjs/bridge.js", urls=[], force=false, console, httpBase=false

Testing tests/index.html
Element - getDocument...OK
Element - onDOMEvent( "onfocusin" )...OK
OO.ui.Process - next...ERROR
>> Message: Steps can be added at the end
>> Actual: [
>>   4,
>>   1,
>>   2
>> ]
>> Expected: [
>>   0,
>>   1,
>>   2
>> ]
>> at lib/qunitjs/qunit.js:580
>>   at tests/Process.test.js:21
>>   at lib/qunitjs/qunit.js:190
>>   at lib/qunitjs/qunit.js:348
>>   at process (lib/qunitjs/qunit.js:1420)
>>   at lib/qunitjs/qunit.js:466

OO.ui.Process - first...OK
OO.ui.Process - execute (async)...OK
OO.ui.Process - execute (return false)...OK
OO.ui.Process - execute (async reject)...OK
OO.ui.Process - execute (wait)...OK
FlaggedElement - constructor...OK
FlaggedElement - getFlags...OK
FlaggedElement - hasFlag...OK
FlaggedElement - clearFlags...OK
FlaggedElement - setFlags...OK
Warning: 1/30 assertions failed (252ms) Use --force to continue.

Aborted due to warnings.

Without --verbose:

$ grunt qunit
Running "qunit:all" (qunit) task
Testing tests/index.html ..F..........
>> OO.ui.Process - next
>> Message: Steps can be added at the end
>> Actual: [
>>   4,
>>   1,
>>   2
>> ]
>> Expected: [
>>   0,
>>   1,
>>   2
>> ]
>> at lib/qunitjs/qunit.js:580
>>   at tests/Process.test.js:21
>>   at lib/qunitjs/qunit.js:190
>>   at lib/qunitjs/qunit.js:348
>>   at process (lib/qunitjs/qunit.js:1420)
>>   at lib/qunitjs/qunit.js:466

Warning: 1/30 assertions failed (253ms) Use --force to continue.

Aborted due to warnings.

Change 181356 had a related patch set uploaded (by Krinkle):
qunit macro: Invoke grunt without --verbose

https://gerrit.wikimedia.org/r/181356

Patch-For-Review

Krinkle renamed this task from QUnit test errors are missing in Jenkins short console log to QUnit test failures are missing from Jenkins short console log.Dec 21 2014, 4:54 AM

Change 181356 merged by jenkins-bot:
qunit macro: Invoke grunt without --verbose

https://gerrit.wikimedia.org/r/181356