Page MenuHomePhabricator

Jenkins triggering false failures occasionally on parsoid patches
Closed, ResolvedPublic

Description

In recent days, we've often run into this issue where a patch passes parser tests after submit, but fails after +2 in the gate-and-submit pipeline.

Looking at jenkins full console log for one the failing run for gerrit patch 106284 above (https://integration.wikimedia.org/ci/job/parsoidsvc-parsertests-run-harder/211/consoleFull), we see this:

17:59:40 + git checkout FETCH_HEAD
17:59:40 Warning: you are leaving 45 commits behind, not connected to

17:59:40 any of your branches:

That looks quite odd ... why would we be leaving behind 45 commits? That indicates a bad checkout somehow.

Look at full console log for a failing run for gerrit patch 107600 (https://integration.wikimedia.org/ci/job/parsoidsvc-parsertests-run-harder/286/consoleFull), we see this:

19:46:07 + git checkout FETCH_HEAD
19:46:07 Warning: you are leaving 2 commits behind, not connected to

19:46:07 any of your branches:

But, Looking at jenks full console log for the final passing run for gerrit patch 107600 (https://integration.wikimedia.org/ci/job/parsoidsvc-parsertests-run-harder/288/consoleFull), we see this:


19:51:12 + git checkout FETCH_HEAD
19:51:12 Warning: you are leaving 1 commit behind, not connected to

19:51:12 any of your branches:

This looks more reasonable.


Version: wmf-deployment
Severity: normal

Details

Reference
bz60106

Event Timeline

bzimport raised the priority of this task from to Needs Triage.Nov 22 2014, 2:57 AM
bzimport set Reference to bz60106.
bzimport added a subscriber: Unknown Object (MLST).

Change 117045 had a related patch set uploaded by Hashar:
Parsoid: uses ZUUL_COMMIT as a git refspec to build

https://gerrit.wikimedia.org/r/117045

Change 117045 merged by jenkins-bot:
Parsoid: uses ZUUL_COMMIT as a git refspec to build

https://gerrit.wikimedia.org/r/117045

I am pretty sure https://gerrit.wikimedia.org/r/#/c/117045/ fixed the issue, i.e. to use a branch specifier in the Jenkins Git plugin to discard its heuristic when finding which commit to build. That was hitting us previously (bug 46723) and the fix is fairly easy.

I have updated all the jobs:

parsoid-parse-tool-check
parsoid-parsertests-run
parsoid-parsertests-run-harder
parsoid-php-parsertests
parsoid-regressions
parsoid-roundtrip-test-check
parsoidsvc-php-parsertests