Page MenuHomePhabricator
Create Task
Maniphest T64511

Mass rendering tests / visual diff
Closed, ResolvedPublic
Actions

Description

Once we have proper CSS hooked up as discussed in bug 51245 we should start automated visual comparisons between PHP parser HTML + CSS and Parsoid HTML + CSS. The easiest way to scale & surface this is likely testreduce.

Some promising tools mentioned in a recent wikitech thread titled 'CSS regressions':

Version: unspecified
Severity: normal
See Also:
https://bugzilla.wikimedia.org/show_bug.cgi?id=62633

Details

Reference
bz62511

Related Objects
Search...

StatusSubtypeAssignedTask
 OpenFeatureNoneT40891 MediaWiki API 2.0 (tracking)
 ResolvedJdlrobsonT95741 API: Inconsistent response for empty collections
 ResolvedssastryT64511 Mass rendering tests / visual diff
 ResolvedArlolraT53245 Link MediaWiki styles and create Parsoid-specific CSS styling to match MediaWiki's for differing DOM elements
 ResolvedArlolraT39902 RFC: Implement rendering of redlinks in Parsoid HTML as post-processor
 ResolvedEsandersT64803 VisualEditor: Extract red link API code from VE and put into Parsoid
 ResolvedCatropeT39901 VisualEditor: Links are not shown as redlinks when their target is blank (or as 'stub-alternate-colour' if appropriate)
 DeclinedNoneT101046 [EPIC] Use MCS as parser for main content in mobile web
 DeclinedNoneT78734 Support for mobile's section use cases in Parsoid HTML
 ResolvedssastryT114072 <section> tags for MediaWiki sections
 Resolved• bearNDT178705 Prepare MCS for Parsoid section tags
 Resolved• bearNDT178706 Improve section parsing in mobile-sections endpoint
 Resolved• bearNDT178707 Create script to compare old and new sectioning code
 Resolved• bearNDT178708 Update Wiktionary definition parsing to account for new (possibly nested) section tags
 OpenNoneT114194 MediaWiki should be able to support multiple parser engines
 Resolved• JhernandezT114279 Parsoid API responses contain unnecessary meta data bloating HTML response
 OpenNoneT78676 Store & load data-mw separately
 Resolved• marcoilT88290 Update data-mw encoding for <ref> tags to point to the HTML content in the <references /> output rather than duplicating it
 ResolvedCatropeT88650 Support data-mw.body.id for reference contents
 Resolved• marcoilT88660 Parsoid Cite: Render missing reflists
 ResolvedArlolraT76518 Errors in Parsoid v2 entry point
 ResolvedArlolraT54936 Move data-parsoid and possibly data-mw out of the DOM, add uids
 Resolved• GWickeT50483 Expose a parsoid public web API end point in the MediaWiki API
 Resolved• GWickeT89481 RESTBase beta release (revision storage / content API)
 Resolved• GWickeT1228 RESTbase deployment
 ResolvedfgiunchediT76986 RESTBase production hardware
 ResolvedfgiunchediT88805 rack and setup restbase production cluster in eqiad
 Resolved• CmjohnsonT83551 Split memcached in eqiad across multiple racks/rows
 Resolved• CmjohnsonT82259 mc1016 mgmt not working
 ResolvedJoeT89345 Missing memcached logs
 ResolvedfgiunchediT89639 restbase1006 faulty disk controller
 Resolved• GWickeT75955 RESTBase / Parsoid integration - waiting for parsoid deploy
 Resolved• JdouglasT88456 On-demand generation of HTML and data-parsoid
 Resolved• GWickeT88457 html2wt conversion support
 Resolved• GWickeT88588 Formalize the RESTBase v1 specification
 ResolvedArlolraT89411 Parsoid v2 API selser might not be working correctly
 Resolved• GWickeT76370 RESTBase performance testing
 Resolved• mobrovacT76165 Handle revision deletion and suppression in RESTBase
 Resolved• mobrovacT87520 Set up update jobs for RESTBase: code done and deployed, to be configured
 Resolved• mobrovacT88652 Integrate revisions into /page structure. Was: Create a /page/revision/{revision} entry point
 Resolved• mobrovacT87392 Add property & index on namespace in page_revisions
 Duplicate• JdouglasT78214 Documentation location and deployment process
 ResolvedAndrewT88850 Import Cassandra packages for Jessie
 Resolved• GWickeT89461 trebuchet puppet provider broken on systems without upstart
 ResolvedfgiunchediT89636 setup internal LVS for restbase eqiad servers
 ResolvedfgiunchediT78514 Detailed cassandra monitoring: metrics and dashboards done, need to set up alerts
 ResolvedfgiunchediT93886 Cassandra/CQL query interface monitoring
 ResolvedEevansT93884 graphs for Cassandra metrics
 ResolvedfgiunchediT97024 some cassandra metrics sent with invalid values
 Resolved• GWickeT78194 Public entry point for RESTBase
 Resolved• GWickeT89567 Load test production cluster
 ResolvedfgiunchediT89366 Access to restbase / cassandra cluster
 Resolved• mobrovacT91102 Update / maintain Beta Cluster restbase cluster: Up & working with VE
 Resolved• GWickeT91188 Need to change data model for >~300 wikis in single cassandra cluster
 Resolved• GWickeT128231 All known clients of Parsoid HTML (either directly hitting Parsoid or via RESTBase) should provide Accept: headers in their requests
 Resolved• AlexMonk-WMFT128233 VisualEditor: Add Accept: header to your HTML requests to RESTBase/Parsoid as per T124365
 Resolved• santhoshT128235 CX Server: Add Accept: header to your HTML requests to RESTBase/Parsoid as per T124365
 ResolvedssastryT128236 Flow: Add Accept: header to your HTML requests to RESTBase/Parsoid as per T124365
 Resolved• MhollowayT128237 Mobile Content Service: Add Accept: header to your HTML requests to RESTBase/Parsoid as per T124365
 ResolvedJackmcbarnT128238 EPH Gadget: Add Accept: header to your HTML requests to RESTBase/Parsoid as per T124365
 ResolvedRenxiaoyiT128239 Google: Add Accept: header to your HTML requests to RESTBase/Parsoid as per T124365
 ResolvedcscottT130685 OCG: Add Accept: header to your HTML requests to RESTBase/Parsoid as per T124365
 ResolvedKelsonT130686 Kiwix: Add Accept: header to your HTML requests to RESTBase/Parsoid as per T124365
 OpenNoneT114413 Support various conversions in Parsoid's pb2pb endpoint
 OpenNoneT130639 All known clients of Parsoid HTML that require data-mw should fetch data-mw separately (if using RESTBase) or process the data-mw blob in Parsoid's pagebundle API response (if using Parsoid directly)
 OpenNoneT88623 On Parsoid HTML read pages, VisualEditor should be able to load data-mw from a separate API call and zip it into the DOM
 ResolvedArlolraT130638 Add data-mw as a separate JSON blob in the pagebundle output of Parsoid's API
 ResolvedArlolraT91700 Store and unpack data-* attributes when serializing/parsing document fragments throughout the pipeline
 OpenNoneT130643 Content Translation should load data-mw from a separate API call alongside the body content
 DeclinedNoneT130687 OCG should load data-mw from a separate API call alongside the body content
 OpenNoneT130689 Google's services should load data-mw from a separate API call to RESTBase
 OpenNoneT124837 Update Flow for Parsoid changes re data-mw
 DeclinedNoneT125885 Switch Flow storage to store a page bundle blob instead of HTML
 DeclinedNoneT125886 Run migration script (HTML => wikitext => page bundle) in production
 DeclinedNoneT125888 Write migration script (HTML => wikitext => page bundle) in production
 OpenNoneT125889 Update code (extractors, converters) that uses data-mw to deal with separate components
 OpenNoneT125890 Figure out how to get page bundle from Flow VE
 InvalidNoneT194980 Triage
 InvalidNoneT130641 Allocate a storage bucket for Parsoid's data-mw attribute
DuplicateNoneT114282 Red links via Parsoid
 DuplicateNoneT114286 Resolving styling inconsistencies between HTML returned by Parsoid and HTML returned by PHP parser
 Resolved• JhernandezT113769 Spike: Scope the first version of JavaScript-based prototype on existing Parsoid service
 ResolvedJdlrobsonT113649 Demonstrate impact of loading just lead sections of a group of static pages
 DuplicatePeterT114216 Support CSV output from the WPT wrapper script
 Resolved• bmansurovT115073 [6 hours] Test a subset of pages with only a lead section [take 2]
 DeclinedNoneT145219 [EPIC] Cleanup MobileFormatter and MobileViewAPI where they overlap
 DuplicateNoneT109461 [EPIC]: Improvements to mobileview API
 Resolved• nrayT86210 MobileFrontend throws away warnings and other data from action=parse
 ResolvedphuedxT69054 API mobileview "protection" sometimes returns object, sometimes returns empty array in JSON output
 ResolvedAnomieT12887 API JSON formatter returns [] as an empty return value - inconsistent with {} for non-empty values
 ResolvedAnomieT76728 Clean up ApiResult and ApiFormatXml, create new formatversion
 ResolvedAnomieT85236 Better unit testing for ApiFormat*
 ResolvedEBernhardsonT90745 Flow ApiFlowModerateTopicTest::testModerateTopic() assumes ApiResult will pass through arbitrary objects
 ResolvedAnomieT98083 add checkbox to ApiSandbox to use formatversion=2
 ResolvedAnomieT98086 improve RELEASE-NOTES-1.25 for formatversion=2 change
 OpenNoneT212069 API action=wbgetentities does not handle formatversion=2
 DuplicateNoneT78479 API error messages should be localised
 ResolvedNoneT98528 mobileview "thumbwidth" parameter doesn't always result in return of desired image url for SVG files
 ResolvedphuedxT156408 Create interface between Skin and MobileFormatter
 DeclinedNoneT146396 Improve performance of mobile references API
 Resolved• PcheloloT133221 Support red links updates in change-propagation
 ResolvedArlolraT169975 Missing images render as broken img tags, not redlinks
 ResolvedMooeypooT55436 VisualEditor: Style Parsoid's <figure>s to look like MediaWiki's <div class="thumb">s rather than replacing them
 ResolvedCatropeT55505 VisualEditor: Provide a way for users to edit auto-numbered external links
 Resolved• GWickeT55432 Typeof cleanup and smart serialization of new language and interwiki links
 ResolvedArlolraT69540 Produce/preserve the metadata about additional ResourceLoader modules required by extension tags
 Resolved• marcoilT73490 Parsoid should set the prop parameter when calling API action=expandtemplates
 Resolved• marcoilT86902 Improve Parsoid's loading of CSS modules using ResourceLoader
 Resolved• GWickeT68287 Thumb CSS misses background color and border on caption

Event Timeline

• bzimport raised the priority of this task from to Medium.Nov 22 2014, 2:59 AM
• bzimport added a project: Parsoid-Tests.
• bzimport set Reference to bz62511.
• GWicke created this task.Mar 10 2014, 9:34 PM
ssastry added a comment.Jun 23 2014, 9:52 PM

(In reply to Gabriel Wicke from comment #0)

Once we have proper CSS hooked up as discussed in bug 51245 we should start
automated visual comparisons between PHP parser HTML + CSS and Parsoid HTML
+ CSS. The easiest way to scale & surface this is likely testreduce.

Some promising tools mentioned in a recent wikitech thread titled 'CSS
regressions':

JS code driven by PhantomCSS (JS) and Resemble.js

C++ code .. probably standalone.

Ruby code driven by phantomjs + ImageMagick

nodejs library and has a dependency on a C library (cairo). Not sure if it will work with https://www.npmjs.org/package/image-diff which is ImageMagick binding.

It looks like huddle's solution will fit neatly within our nodejs setup. But, worth experimenting with a couple solns to see what works best in terms of: performance, quality, ease of use.

ssastry added a comment.Jun 24 2014, 10:03 PM

(In reply to ssastry from comment #1)

(In reply to Gabriel Wicke from comment #0)

Once we have proper CSS hooked up as discussed in bug 51245 we should start
automated visual comparisons between PHP parser HTML + CSS and Parsoid HTML
+ CSS. The easiest way to scale & surface this is likely testreduce.

Some promising tools mentioned in a recent wikitech thread titled 'CSS
regressions':

JS code driven by PhantomCSS (JS) and Resemble.js

Downloaded phantomcss and started looking at the code. PhantomCSS seems like a wrapper to work with casper to acquire screenshots, etc. capserjs can be run in nodejs using spookyjs. I'll try experimenting with this tomorrow.

But, I'm now wondering whether the generated images might be too big and/or if we'll get too many trivial rendering diffs, and whether running a normalizing html diff on PHP and Parsoid HTML should be considered as well.

ssastry added a comment.Jul 22 2014, 9:27 PM

The repo @ https://github.com/subbuss/parsoid_visual_diffs is now ready for deploy.

ssastry added a comment.Aug 7 2014, 12:37 PM

Now deployed and mostly working ... still has some issues, but it is more in the realm of tweaking and bug fixing.

Krinkle mentioned this in T64633: Jenkins: Set up perceptual diffs (visual regression testing).Mar 6 2015, 7:14 PM
Arlolra closed subtask T53245: Link MediaWiki styles and create Parsoid-specific CSS styling to match MediaWiki's for differing DOM elements as Resolved.Apr 1 2021, 8:36 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL