Page MenuHomePhabricator

Fix selser result pass/failure detection in parserTests.js
Open, LowPublic

Description

When selser runs (html2wt) on modified HTML, we have to figure out whether the serialized wikitext is correct or not. However, there is no original wikitext to compare against since the DOM has been modified.

Currently, parserTests runs wt2wt on the same modified HTML and compares selser output with wt2wt output. wt2wt uses the regular full serializer. This pass/failure detection implicitly assumes that the output from wt2wt is correct. However, this is flawed -- it might actually be the case that selser output is correct and wt2wt is incorrect since wt2wt might be normalizing wikitext whereas selser wont (which is the whole point of selser, so to speak). This was not as much of an issue till a while back since wt2wt was normalizing less and parserTests used to run in rt-testing mode which used src-based rting for templates and the like. However, this is becoming more of a problem as we move towards greater normalization (ex: bug 51004), and when the regular serializer no longer run in rt-testing mode.

So, we need to figure out a more robust solution to this problem.


Version: unspecified
Severity: normal

Details

Reference
bz51718

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 22 2014, 1:52 AM
bzimport added a project: Parsoid-Tests.
bzimport set Reference to bz51718.

Change 85952 had a related patch set uploaded by Arlolra:
WIP: Remember the output of failing (blacklisted) tests

https://gerrit.wikimedia.org/r/85952

Change 85952 merged by jenkins-bot:
Remember the output of failing (blacklisted) tests

https://gerrit.wikimedia.org/r/85952

Subbu / Gabriel: Patch was merged month ago - is there more work left here, or can you close this ticket as RESOLVED FIXED?

We merged a different workable solution for now, but I still keep hoping for a smarter solution to turn up. Leaving this here in case someone has a smart idea for this when they are bored and idle and thinking about this. ;-)

weird. why is marcoil suddenly the default?

I think I'm the default for Parsoid/tests :)

Arlolra added a project: Parsoid.
Arlolra set Security to None.
ssastry lowered the priority of this task from Medium to Low.Feb 2 2015, 11:31 PM