Page MenuHomePhabricator

Flow: Topic-titles not displaying in some topics (disappearing?)
Closed, ResolvedPublic

Description

screenshot of many topics without titles

A number of the topics at Enwiki are displaying without a title. They do not have any title-edits in their history, and were created with titles.

eg. page 2 has mostly empty ones: https://en.wikipedia.org/w/index.php?title=Wikipedia_talk:Flow/Developer_test_page&topiclist_offset-id=rp338uhkrre3jsht&topiclist_offset-dir=fwd&topiclist_limit=10&workflow=rojlgvjchz98pvno


Version: unspecified
Severity: normal

Attached:

Screenshot_from_2014-02-25_17:48:54.png (1×1 px, 131 KB)

Details

Reference
bz61939

Event Timeline

bzimport raised the priority of this task from to High.Nov 22 2014, 2:56 AM
bzimport set Reference to bz61939.
bzimport added a subscriber: Unknown Object (MLST).

Has this been observed anywhere other than that test page? If not, it's likely some bit of CSS or JS added by a user in the course of testing that's overriding Flow styles.

(In reply to Maryana Pinchuk from comment #2)

Has this been observed anywhere other than that test page?

Yes, but only in the second topic currently at https://en.wikipedia.org/wiki/Wikipedia_talk:WikiProject_Hampshire
specifically: https://en.wikipedia.org/w/index.php?title=Wikipedia_talk:Flow/Developer_test_page&workflow=rpgca5e9yyrhoqq5

The last ones here seem ok to me. Only the original one still seems to have the problem.

I've done some digging and I assume we failed to fetch the content from ExternalStorage, and cached the incorrect result.

$ mwscript eval.php --wiki enwiki:

> $uuid = Flow\Model\UUID::create( 'rp2pfb6ete6pq6fg' );
> $collection = Flow\Model\PostCollection::newFromId( $uuid );
> $revision = $collection->getLastRevision();
> var_dump( $revision );

$revision->content seems to have a false value here. Judging from AbstractRevision.php, the value should either be the full text or an empty string ''.

Since the content is stored in ExternalStorage, it should be fetched via RevisionStorage::mergeExternalContent, which will fetch the content via ExternalStore::batchFetchFromURLs, which can return false. So I'm pretty sure that's where the false comes from. And it looks like we're only caching the value after that.

I've looked at clearing the cache, but has less luck there. It looks like the key the revision is saved at should be enwiki:flow_revision:v4:descendant:rp2pfb6ete6pq6fg:3.0 (I've also tried :pk: instead of :descendant: and :2.0 instead of :3.0, but no luck). Trying to either fetch those keys from cache returns a false, and trying to delete them doesn't change anything. I may be assuming an incorrect cache key here, though. Clearing the cache would only be a stopgap for this particular instance though, and not properly fix the issue.

Any way; to properly fix this, we should (unless someone has a better idea):

  • Make it possible to let our backend storage know to our cache-layers to not cache certain data -> the proper fix IMO, but a fair bit of work

Inferior alternatives:

  • Cache the real DB return (prior to ExternalStore::batchFetchFromURLs) and do ExternalStore::batchFetchFromURLs afterwards -> probably the easiest fix, but would result in a lot more calls to resolve data from ExternalStore.
  • Implement an 2nd ExternalStore::batchFetchFromURLs in AbstractRevision, to retry fetching content in case $revision->content is false -> easy too, but duplicate code, and still unneccessary calls to ExternalStore for the duration of the corrupt cache

Oh, and also: should we still have occurences of $revision->content being false, we should probably make AbstractRevision::getContent return a 'We could not fetch this content' message instead of displaying nothing.

Change 119304 had a related patch set uploaded by Matthias Mullie:
(bug 61939) Topic-titles not displaying in some topics (disappearing?)

https://gerrit.wikimedia.org/r/119304

Dug into this with benny last night, fairly sure what happened is:

  • Some data models were not in the cache
  • Visiting a contributions page loaded cross-wiki data and cached it
  • ExternalStore uses per-wiki databases, but flow doesn't take that into account when querying
  • As such the data that got cached included blank content

Things we should do:

As matthias mentioned, we need to either remove the bad data from the result set or somehow mark that row as uncachable.

We should probably also look into having a flowdb created on cluster25 and cluster26 (the insert clusters for ES) and moving all our content over so we have a valid cross-wiki content storage.

The short-term issue looks to be fixed; moving the longer term solution stuff to Trello: https://trello.com/c/4Oel8VTK/150-improve-caching

Change 134089 had a related patch set uploaded by Matthias Mullie:
(bug 61939) Topic-titles not displaying in some topics (disappearing?)

https://gerrit.wikimedia.org/r/134089

Change 119304 abandoned by Matthias Mullie:
(bug 61939) Topic-titles not displaying in some topics (disappearing?)

Reason:
Up-to-date patch at https://gerrit.wikimedia.org/r/134089

https://gerrit.wikimedia.org/r/119304

Change 134089 merged by jenkins-bot:
(bug 61939) Topic-titles not displaying in some topics (disappearing?)

https://gerrit.wikimedia.org/r/134089