Page MenuHomePhabricator

test2.wikipedia can't load wikidata.org's revision content from ES
Closed, ResolvedPublic

Description

test2.wikipedia.org is now configured to act as a client to wikidata.org. In order to show language links defined in the wikidata items, test2 needs to load revision content from wikidata.org's database. This is done in WikiPageEntityLookup::loadEntity(). In the $row parameter, this gets a row that contains fields from wikidata.org's revision table. It then calls Revision::getRevisionText( $row ) to fetch the revision's blob. But it fails for some unknown reason.

Example:

  • Request: https://test2.wikipedia.org/wiki/New_York_City?action=purge
  • LanguageLinkHandler::getEntityLinks() calls SiteLinkTable::getItemIdForLink(), and gets back the info that q60 is the item to look for.
  • it then calls WikiPageEntityLookup::getEntity() to retrieve the entity object.
  • WikiPageEntityLookup::getEntity queries wikidata.org's database to get the revision ID for item q60's latest revision.
  • it gets rev_id=631469, old_text=DB://cluster25/316188, and old_flags=utf-8,gzip,external
  • WikiPageEntityLookup::getEntity calls self::loadEntity()
  • WikiPageEntityLookup::loadEntity calls Revision::getRevisionText on the row with old_text=DB://cluster25/316188, and old_flags=utf-8,gzip,external.
  • Revision::getRevisionText returns false.

Chad tried this in eval.php, and it works. So, apparently, somehow fetching the revision blob from ES fails.


Version: wmf-deployment
Severity: major

Details

Reference
bz42825

Event Timeline

bzimport raised the priority of this task from to Needs Triage.Nov 22 2014, 12:55 AM
bzimport set Reference to bz42825.

Note: the above example applies to the mw1.21-wmf5 branch of the Wikibase modules, as of commit rEWBA24d8471add31.

To test for this bug, try the following:

Go to https://test2.wikipedia.org/wiki/Helium, open the edit page. There should be no (or just a few) interlanguage links.

The, click preview. More (>100) language links should be loaded from https://www.wikidata.org/wiki/Q560.

Note however that this only works as a test as long as the solution for bug 42187 isn't implemented. That will make it unnecessary to load the item on the client, for now. We *will* need that ability in a few weeks at the latest, though.

In eval.php, try this:

$row = new stdClass();
$row->old_text = "DB://cluster25/316188";
$row->old_flags = "utf-8,gzip,external";
Revision::getRevisionText( $row );

Apparently, when Chad tried this, it worked. Yet, if tried in a web request, it fails...

$ mwscript eval.php --wiki=test2wiki

var_dump( ExternalStore::fetchFromURL( 'DB://cluster25/316188' ) );

bool(false)
$

:-(

As brion remarked on IRC, the external store servers have a DB for each wiki. If you look at addWiki.php you can see how each cluster master has the new DB initialized for a new wiki. Also see storage/blobs.sql. In order to access text blobs saved from a foreign wiki, one would need to change getRevisionText() to pass down some optional wiki ID and change ExternalStore to except it. So if you know you are fetching a text for wikidatawiki that wiki ID could be passed in.

This is a case of some subtle global state (the wiki ID is assumed to be the current wiki ID).

(In reply to comment #3)

In eval.php, try this:

$row = new stdClass();
$row->old_text = "DB://cluster25/316188";
$row->old_flags = "utf-8,gzip,external";
Revision::getRevisionText( $row );

Apparently, when Chad tried this, it worked. Yet, if tried in a web request,
it
fails...

Works when run on wikidatawiki, not elsewhere.

fixed by Change I14a7ebb8: (bug 42948) $wiki parameter for getRevisionText().

Closed older resolved bugs as verified.