Page MenuHomePhabricator

Kraken unit-tests on Jenkins are failing
Closed, DeclinedPublic

Description

Currently, 5 unit-tests for Kraken fail when running it on Jenkins:

  1. org.wikimedia.analytics.kraken.pig.UserAgentClassifierTest.testIpad2: expected:<iPhone OS> but was:<iOS>
  2. org.wikimedia.analytics.dclassjni.DclassWrapperTest.testDclassWrapper: Model as expected expected:<A63[80]> but was:<A63[66]>
  3. org.wikimedia.analytics.dclassjni.DeviceClassificationTest.testDeviceClassification: Model as expected expected:<A63[80]> but was:<A63[66]>
  4. org.wikimedia.analytics.kraken.pig.ComparePageviewDefinitionsTest.testMobilePageviewCount: expected:<(1[4,16],18)> but was:<(1[,1],18)>
  5. org.wikimedia.analytics.kraken.pig.ComparePageviewDefinitionsTest.testDesktopPageviewCount: expected:<([1,5],2)> but was:<([0,0],2)>

Version: unspecified
Severity: normal

Details

Reference
bz54046

Event Timeline

bzimport raised the priority of this task from to Needs Triage.Nov 22 2014, 2:10 AM
bzimport set Reference to bz54046.
bzimport added a subscriber: Unknown Object (MLST).

While talking with the analytics team, I noticed analytics/kraken does not have any Jenkins job being triggered when a change is submitted. There is a manual job though http://integration.wikimedia.org/ci/job/Kraken/

I created a job in Jenkins using Jenkins Job Builder and added it in Zuul so it is now triggered. The configuration is:

https://gerrit.wikimedia.org/r/#/c/114622/2/analytics.yaml,unified

Aka it runs: maven clean package

I ran the job against a dummy commit ( https://gerrit.wikimedia.org/r/114531 ) and the jobs reports a lot of warnings and has some tests failures still.

Build page: http://integration.wikimedia.org/ci/job/analytics-kraken/6/

Tests results: http://integration.wikimedia.org/ci/job/analytics-kraken/6/testReport/

Console log: http://integration.wikimedia.org/ci/job/analytics-kraken/6/consoleFull

Fixing that is currently still not on our agenda :-(((

But since the errors and root causes are known, let me briefly chime in on
them (in case anybody else wants to fix the issues):

(In reply to Antoine "hashar" Musso from comment #2)

http://integration.wikimedia.org/ci/job/analytics-kraken/6/consoleFull

-----Begin:log snippet-----
21:44:07 Failed tests: testDclassWrapper(org.wikimedia.analytics.dclassjni.DclassWrapperTest): Model as expected expected:<A63[80]> but was:<A63[66]>
21:44:07 testDeviceClassification(org.wikimedia.analytics.dclassjni.DeviceClassificationTest): Model as expected expected:<A63[80]> but was:<A63[66]>
-----End:log snippet-----

Those two indicate that dclass is using a different wrong dtree file. We saw
those tests fail in the same way when trying to move from dtree v1.13 to v1.19.

-----Begin:log snippet-----
21:44:20 Tests in error:
21:44:20 testDesktopPageviewCount(org.wikimedia.analytics.kraken.pig.ComparePageviewDefinitionsTest)
21:44:20 testMobilePageviewCount(org.wikimedia.analytics.kraken.pig.ComparePageviewDefinitionsTest)
-----End:log snippet-----

Those two indicate that the testdata_* CSVs are not in place (see kraken's
README.md)

Fixing the above two items should make the tests pass.
(Using current HEAD

35cfc89027b0220fe9ff42a626332e9d32813610

the tests pass for me.)

kevinator claimed this task.
kevinator triaged this task as Low priority.
kevinator set Security to None.
kevinator removed a subscriber: drdee.

Closing since we're upgraded the cluster and use Refinery now... not Kraken.