Page MenuHomePhabricator

TMH does not send cache purge for transcodes of deleted media
Closed, DeclinedPublic

Details

Reference
bz52531

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 22 2014, 1:52 AM
bzimport set Reference to bz52531.

jgerber wrote:

I get a 404 for both urls, possibly just a caching issue?

possibly.
I still get the file(s), though.

wget -S http://upload.wikimedia.org/wikipedia/commons/transcoded/7/72/Ainomamade.wav/Ainomamade.wav.ogg
--2013-08-05 13:26:14-- http://upload.wikimedia.org/wikipedia/commons/transcoded/7/72/Ainomamade.wav/Ainomamade.wav.ogg
Resolving upload.wikimedia.org (upload.wikimedia.org)... 91.198.174.234, 2620:0:862:ed1a::b
Connecting to upload.wikimedia.org (upload.wikimedia.org)|91.198.174.234|:80... connected.
HTTP request sent, awaiting response...

HTTP/1.1 200 OK
X-Object-Meta-Sha1base36: 9zt0cg135ly9twlrauxjxlc4cwp1h41
Last-Modified: Wed, 24 Jul 2013 04:34:52 GMT
Etag: 72bf7863a9dfa797968bccad76d09f40
X-Timestamp: 1374640492.41808
Content-Type: application/ogg
X-Varnish: 3099219985, 112751158 9111791, 1418575134
Via: 1.1 varnish, 1.1 varnish, 1.1 varnish
Content-Length: 690198
Accept-Ranges: bytes
Date: Mon, 05 Aug 2013 11:26:14 GMT
Age: 79269
Connection: keep-alive
X-Cache: cp1064 miss (0), cp3006 hit (10), cp3006 frontend miss (0)
Access-Control-Allow-Origin: *

Length: 690198 (674K) [application/ogg]
Saving to: `Ainomamade.wav.ogg'

100%[======================================>] 690,198 2.39M/s in 0.3s

2013-08-05 13:26:14 (2.39 MB/s) - `Ainomamade.wav.ogg' saved [690198/690198]

(In reply to comment #1)

I get a 404 for both urls, possibly just a caching issue?

Likely the "US vs. Europe" caching issue.

(In reply to comment #2)

wget -S
http://upload.wikimedia.org/wikipedia/commons/transcoded/7/72/Ainomamade.wav/
Ainomamade.wav.ogg

HTTP/1.1 404 Not Found (from Hong Kong).

Marco: How much time is there since deletion of the file?

Seems unlikely to be esams specific. It probably just wasn't in cache on the other varnish cache site (just a guess)

(In reply to comment #4)

Marco: How much time is there since deletion of the file?

Virus de Amor - Dejame PRIMICIA 2013.wav
deleted 13:27, 4 August 2013

Ainomamade.wav
deleted 13:24, 4 August 2013

Update:
https://upload.wikimedia.org/wikipedia/commons/transcoded/f/fa/Virus_de_Amor_-_Dejame_PRIMICIA_2013.wav/Virus_de_Amor_-_Dejame_PRIMICIA_2013.wav.ogg
gives 404
https://upload.wikimedia.org/wikipedia/commons/transcoded/7/72/Ainomamade.wav/Ainomamade.wav.ogg
still works

Update: Both of my links in comment #6 give 404. Which is the desired behaviour.

(In reply to comment #7)

Update: Both of my links in comment #6 give 404. Which is the desired
behaviour.

After 30 days, things get killed out of cache automatically (or something like that. 30 was pulled out of my hat). If newly deleted files don't have transcodes purged immediately, this is still a bug.

(In reply to comment #5)

Seems unlikely to be esams specific. It probably just wasn't in cache on the
other varnish cache site (just a guess)

Propably true. Therefore I am trying to reopen this bug.

Unsuccessful, I can only choose RESOLVED or VERIFIED.

(In reply to comment #10)

Unsuccessful, I can only choose RESOLVED or VERIFIED.

Ask andre for editbugs rights.

Can not reproduce as of now.


Further information from curl -v

  1. Prior to deletion

1.1. From US

< HTTP/1.1 200 OK
< Server: nginx/1.1.19
< Date: Tue, 01 Apr 2014 12:27:53 GMT
< Content-Type: application/ogg
< Content-Length: 97644
< Connection: keep-alive
< X-Object-Meta-Sha1base36: fvav42psfg3se67xqf6y8ju0f7erkzg
< Last-Modified: Sun, 30 Mar 2014 21:19:12 GMT
< Etag: f3450464e444ef4ede86db562ea35813
< X-Timestamp: 1396214352.61077
< X-Varnish: 537554959, 4058383635 3970472239, 3276074645
< Via: 1.1 varnish, 1.1 varnish, 1.1 varnish
< Accept-Ranges: bytes
< Age: 81364
< X-Cache: cp1051 miss (0), cp4006 hit (4), cp4015 frontend miss (0)
< Access-Control-Allow-Origin: *
< Access-Control-Expose-Headers: Age, Content-Length, Date, X-Cache, X-Varnish
<

1.2. From Europe

< HTTP/1.1 200 OK
< Server: nginx/1.1.19
< Date: Tue, 01 Apr 2014 12:27:44 GMT
< Content-Type: application/ogg
< Content-Length: 97644
< Connection: keep-alive
< X-Object-Meta-Sha1base36: fvav42psfg3se67xqf6y8ju0f7erkzg
< Last-Modified: Sun, 30 Mar 2014 21:19:12 GMT
< Etag: f3450464e444ef4ede86db562ea35813
< X-Timestamp: 1396214352.61077
< X-Varnish: 670705656, 2855528785 2855199773, 1400256059 1400187325
< Via: 1.1 varnish, 1.1 varnish, 1.1 varnish
< Accept-Ranges: bytes
< Age: 127
< X-Cache: cp1051 miss (0), cp3010 hit (1), cp3007 frontend hit (1)
< Access-Control-Allow-Origin: *
< Access-Control-Expose-Headers: Age, Content-Length, Date, X-Cache, X-Varnish
<

  1. After deletion

Though I saw a difference in the age of the two cached files (127 against 81k) both cached transcodes return a 404 for US and Europe right now.

Gilles raised the priority of this task from Medium to Unbreak Now!.Dec 4 2014, 10:24 AM
Gilles moved this task from Untriaged to Done on the Multimedia board.
Gilles lowered the priority of this task from Unbreak Now! to Medium.Dec 4 2014, 11:22 AM