Page MenuHomePhabricator

Allow extensions to provide purge mechanisms for external CDNs (beyond Squid and HTTP PURGE protocol)
Closed, ResolvedPublic

Description

Author: carlb613

Description:
SquidPurgeClient.php seems hard-coded so content delivery network (CDN) nodes are presumed to be Squid, compatible with HTTP PURGE sent from a trusted IP address.

This works if the cache acts like a Squid (Varnish is configurable to do this) but is problematic when dealing with commercial content delivery networks.

Services like CloudFlare (or Amazon AWS) expect the origin server to invoke their specific proprietary API to signal that a page or file has been superseded with updated content.

HTTP PURGE likely doesn't even reach them as the long list of IPv4 / IPv6 ranges is normally handled with [[mw:extension:TrustedXFF]] or a proprietary module (like mod_cloudflare for Apache). Even if it were possible to list the entire CDN in $wgSquidServers, those servers would ignore a Squid-style PURGE message.

The end result is that plenty of outdated content is served as the CDN has no way to know a new version exists until the original version expires from cache.

There needs to be a way to replace the stock SquidPurgeClient with an extension which provides a function to purge CDN files, but I see no hooks in the code to allow this. There are [[mw:Manual:CloudFlare#HTTP_purge]] extensions for WordPress, Drupal and the like but nothing for MediaWiki as the hook to add such an extension does not exist.


Version: 1.23.0
Severity: normal

Details

Reference
bz62356

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 22 2014, 3:07 AM
bzimport set Reference to bz62356.
bzimport added a subscriber: Unknown Object (MLST).

Hmm, that's odd we have no hook there (Note that we do have a hook for purging thumbnails - LocalFilePurgeThumbnails, but nothing for general article purging)

Would it make sense to hook [[mw:Manual:Hooks/TitleSquidURLs]] to get the list of URLs to purge?

It's certainly not how the hook is intended to be used (it's supposed to provide the list so extensions can add URLs) but the worst that can happen should be that extra URLs added by some other extension (depending on order called) miss getting purged. As the hook has only existed since MW 1.22, the number of affected extensions should be small.

ori added a subscriber: aaron.

Such a hook should definitely be added.

Krinkle renamed this task from SquidPurgeClient.php provides no hooks for extensions to purge individual files from non-Squid, non-Varnish content delivery networks to Allow extensions to provide purge mechanisms for external CDNs (beyond Squid and HTTP PURGE protocol).Apr 11 2020, 5:49 PM
Krinkle subscribed.

I believe you can use the EventRelayer interface for this. It does not yet have much documentation, but the following should get you in the right direction.

For a custom CDN purger:

  • Enable $wgUseCDN so that CdnCacheUpdate runs. (Keep these off $wgCdnReboundPurgeDelay, $wgCdnServers, and $wgHTCPRouting).
  • Configure the cdn-url-purges channel $wgEventRelayerConfig with an implementation that purges your CDN in the way that it requires.

See also:

The cdn-url-purges channel will be notified with a list of urls to purge in the CDN. As a site administrator, you'd configure $wgEventRelayerConfig['cdn-url-purges']. The extension could provide an EventRelayer subclass that handles messages for $channel === 'cdn-url-purges' in the way that your CDN provider requires. For example:

$wgUseCDN = true;
$wgEventRelayerConfig['cdn-url-purges'] = [
  'class' => MediaWiki\Extension\MyCloudFare\EventRelayer::class,
];