Page MenuHomePhabricator

Automate cache expiration for pages using DynamicPageList
Open, MediumPublicFeature

Description

In theory it should be possible to have pages using the DynamicPageList expire from cache after some predetermined age, which would reduce the need for manual purges (eg, as noted at bug 11617) while still letting pages be cached fairly sanely against heavy load.

A couple parts should be 'easy':

  1. Cache-Control headers -- the s-maxage could be reduced, allowing the page to expire from Squids.
  1. ParserCache entry could have an expiration time added to it, letting the system re-render.

I'm not entirely sure though how best to handle If-Modified-Since checks (from user agents or proxy caches). The page_touched timestamp wouldn't have changed, so the system as currently set up would come back with a 304 response from the page metadata long before reaching the parser cache.

It might be necessary to add a page_expiry field or something... or some kind of equivalent check hacked in as a hook.

Note that other extensions could use a similar mechanism.


Version: unspecified
Severity: enhancement

Details

Reference
bz11685

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 21 2014, 9:54 PM
bzimport set Reference to bz11685.
bzimport added a subscriber: Unknown Object (MLST).

Assign open DynamicPageList issues to new component maintainer.

  • Bug 32336 has been marked as a duplicate of this bug. ***

Any solution that will save me from doing a manual ?action=purge on every single
page where DPL is used would be really useful. The third-party DPL seems to
have a specific config entry for the parser cache, I am not sure if this is of any use:
http://www.mediawiki.org/wiki/Extension:DynamicPageList_(third-

The third-party DPL seems to
have a specific config entry for the parser cache, I am not sure if this is of
any use:
http://www.mediawiki.org/wiki/Extension:DynamicPageList_(third-

The third party dpl basically disables cache totally, which is not an option... (And then tries to re-invent its own cache, which is isn't that great an idea)

However, we could shorten the parser cache length with a call to $parser->getOutput()->updateCacheExpiry( foo );. I'm not sure what an appropriate value of foo would be (72 hours maybe?) [this was recently suggested by Aaron I'm told]

In long term, we could possibly record which page has dpls involving which categories in a table somewhere, and then invalidate pages when someone adds/removes things from that category.

Added default 24 hour cache expire time (configurable via $wgDLPMaxCacheTime) in r103382.

Leaving bug open, as I think in longer term we could do better then just early expiring the cache.

Awesome, thanks. Do I need to file a separate bug to request a config change on Meta or can this be handled as part of this ticket?

(In reply to comment #6)

Awesome, thanks. Do I need to file a separate bug to request a config change on
Meta or can this be handled as part of this ticket?

I'll tag the revision as 1.18wmf1 which will cause it to be deployed a whole lot sooner. (As a work around, putting something like {{CURRENTHOUR}} on a wiki page will cause cache to expire much faster for that page).

Lowering priority as per comment 5 (had been "high" for ~18months)

Aklapper changed the subtype of this task from "Task" to "Feature Request".Feb 4 2022, 12:24 PM
Aklapper removed subscribers: DarTar, wikibugs-l-list.