Page MenuHomePhabricator

Maintenance script to permanently delete files
Closed, ResolvedPublic

Description

Some times the ops team gets requests to permanently delete files. What has traditionally happened, is an ops team member gets the request, usually asks for an "upload.wikimedia.org" URL instead of a File:[...] URL and uses the swift command-line to eradicate the file from the deleted container. I've also heard reports of cleaning up thumbs that weren't previously deleted by MediaWiki, although I haven't seen this myself.

This is tedious and manual work and not all ops people have Swift domain knowledge. The requests usually come in as urgent, so it'd be nice to not have to figure out details (e.g. this time: Ceph) while handling the request.

Therefore, it'd be nice if there was a maintenance script that:
a) Takes a File:[...] as input,
b) Verifies that the file is "deleted" on the wiki,
c) Deletes it from the -deleted.NN container on every file backend,
d) Cleans up possibly generated thumbs, just in case.
e) Optionally resend cache purges?


Version: 1.22.0
Severity: enhancement

Details

Reference
bz47990

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 22 2014, 1:31 AM
bzimport set Reference to bz47990.

Without checking, I believe there is a maintaince script already that will do (A) and (B) and remove any old revisions of the file as well. That could be expanded on/used in conjunction with another script.

FWIW, from the LCA perspective, this could be hugely helpful. I hate having to bother engineering over these, because I know they're a pain. It would be ideal if there were something that accepted input from people in the staff group, for instance, and executed that. Alternatively, just something that would make the process easier for engineering would be welcomed by me.

pb

(In reply to comment #2)

... It would be
ideal if there were something that accepted input from people in the staff
group, for instance, and executed that. Alternatively, just something that
would make the process easier for engineering would be welcomed by me.

If it's opened up, It might also be worth while putting some logging in it so it goes to a [private] log file, just so who did what, when and why is recorded.

Related URL: https://gerrit.wikimedia.org/r/62549 (Gerrit Change I2fcc2b2c3e92cd851c38f947de6756a982cf194d)