Page MenuHomePhabricator

Deleting files on en.wp: Unknown error occurred in storage backend "local-swift-eqiad"
Closed, ResolvedPublic

Description

When attempting to delete a few images today on the English Wikipedia, in irregular intervals I was presented with an error that stated, "Error deleting file: An unknown error occurred in storage backend "local-swift-eqiad"". So far I've encountered it about 4 times in the span of 2 hours.

There wasn't any commonality with the images that presented the error and the files weren't particularly large. (For instance, one file was only 76 KB.) For all the files where I encountered the issue, when I attempted to delete them again I was successful.


Version: wmf-deployment
Severity: normal

Details

Reference
bz73094

Event Timeline

bzimport raised the priority of this task from to High.Nov 22 2014, 3:54 AM
bzimport set Reference to bz73094.

Thanks for taking the time to report this!

Can you please provide links to some of the affected pages?
Do you have times when this happened (the more exact, the better)?

I'm not seeing anything suspicious on https://wikitech.wikimedia.org/wiki/Server_Admin_Log :-/

I was able to replicate the error. It occurred when I tried to delete this file:
https://en.wikipedia.org/wiki/File:Lachrymose_leech.png and the error occurred around 00:20-00:22 UTC Nov 7th.

I also got a different error when trying to delete https://en.wikipedia.org/w/index.php?title=File:LuigiTV.png

Error deleting file: An unknown error occurred in storage backend "local-multiwrite".

This occurred around 00:44:30 UTC Nov 7

gerritadmin wrote:

Change 173864 had a related patch set uploaded by Gilles:
Log more details about backend-fail-internal errors

https://gerrit.wikimedia.org/r/173864

gerritadmin wrote:

Change 173864 merged by jenkins-bot:
filebackend: Log more details about backend-fail-internal errors

https://gerrit.wikimedia.org/r/173864

gerritadmin wrote:

Change 174128 had a related patch set uploaded by Gilles:
filebackend: Log more details about backend-fail-internal errors

https://gerrit.wikimedia.org/r/174128

gerritadmin wrote:

Change 174128 merged by jenkins-bot:
filebackend: Log more details about backend-fail-internal errors

https://gerrit.wikimedia.org/r/174128

MarkTraceur set Security to None.

Removed Patch-For-Review because the merged patch was for debugging purposes.

Gilles moved this task from Batch Move 2019-03-06 to Done on the Multimedia board.

Ever since the extra tracking was put in place, not a single instance was caught. It was probably a partial outage around that particular time. Please reopen if it happens again.