Restarted bits cache on pmtpa. It did not help apparently. Maybe touch the javascript/css files ?

ssh deployment-bastion.pmtpa.wmflabs
cd /data/project/apache/common-local/

sudo su --login --shell /bin/bash mwdeploy

touch!

:-]

I have no idea how the js/css resources get invalidated. Someone who knows about how bits caching works should have a look at it.

Roan: Could you give us a tip on how to effectively/easily invalidate cached js/css on Beta Cluster when they're updated?

Sam, any clue how we invalidate the JS/CSS cache on the bits cache when doing production deployment? I am assuming it expires after 5 minutes automatically.

So far this bug has failed to give any clear definition of what kind of resources this is about.

Are we talking about static resources served directly from the filesystem? E.g. files referenced directly out of skins/ or wgExtensionAssetsPath? That should only happen for for images in browsers without support for data URI embedding (or images too large to embed).

Anything loaded by ResourceLoader is naturally invalidated by proper HTTP response headers from load.php, there's no WMF-specific configuration in Varnish that makes that work.

Unversioned resources (e.g. startup module) has a cache-age of 5 minutes, and all other resources are requested via the startup with a version/timestamp in the url, and those urls respond with long-term caching (30+ days) from load.php. Varnish just consumes these max-age headers for its caching, and forwards them to the browser for client-side caching as well.

There are a few extra finetune things in Wikimedia's Varnish configuration for bits, but that should be applied to bits in Beta labs as well via puppet. If not, those manifests should be refactored to work in labs too.

So yeah, what resources are we talking about, in what way are they not being invalidated, and how is it different from production?

If this is indeed the same issue as T90983, I think the priority should be bumped somewhat. It makes beta somewhat less than useful for, you know, actually verifying the code changes before they hit production. :)

/me pokes resident varnish expert @BBlack

I pasted some headers responses on T100542#1318887 (most probably be a duplicate).

This is all related to the same basic issue in production: T99096

Closing in favour of these two since there are no directly tangible or actionable issues left here.

• T90983: ResourceLoader debug urls should bypass cache when they change
• T99096: Make Varnish cache for /static/$wmfbranch/ expire when resources change within branch lifetime

Note that I suspect the original issue reported in this task may actually be an unrelated problem related to HTML caching. If said configuration variable changed data in mw.config or changed which modules are loaded, then the module queue will only affect logged-out users after the viewed page expired page.

This is mostly by design and require the application to match this design. There's various performant and workable ways around this. Let me know if you want me to elaborate or guide in that.