Page MenuHomePhabricator

(optionally) store uploads in database blobs
Closed, DeclinedPublic

Description

Although there are good reasons to use the filesystem for uploaded files, there are also problems with it in
some situations:

  • Security -- world-writable directories on multi-user hosts
  • Security -- dangerous filetypes may slip through and execute scripts server-side
  • Inconvenient filesystem naming limitations (eg bug 362)
  • Stupid configurations -- PHP 'safe mode' and other such things that may make it difficult for some

installations to have a writable directory tree for uploads

So, it may be useful to optionally provide for storing uploads in BLOBs, as for instance Bugzilla does.

Issues to overcome:

  • Maximum file size limited by mysql max packet size [default should be big enough for most purposes]
  • Thumbnail generation may need to work via temporary files

Version: unspecified
Severity: enhancement

Details

Reference
bz363

Event Timeline

bzimport raised the priority of this task from to Lowest.Nov 21 2014, 6:54 PM
bzimport set Reference to bz363.
bzimport added a subscriber: Unknown Object (MLST).
  • Bug 5406 has been marked as a duplicate of this bug. ***

vertigosteve wrote:

Maybe running a second image database could be given as an option/alternative to
the current form?
Im sure MySQL can handle this too, but a native XML db would allow for
serialized connection to particular data. That would sort of fit in with the
m:Wikidata proposal.

vertigosteve wrote:

Now that I think about it, there are probably size issues with any native XML db
too. :)

docuswear wrote:

I am running in a replicated environment, and have MySQL running in a
multi-master mode. We would like to place all uploaded files in the database, so
we will not have to rsynch from location1 to location2, and vice-versa.

No progress since 2006. Is this even a good idea? CCing some Multimedia folks.

It seems like a valid idea, but performance is an unknown and I'm not sure how big the demand for it is. Imho someone who really needs it could build it and share it back upstream. The fact that this hasn't happened yet is probably a sign that people might not need it that badly.

Right now it would seem like spending a significant amount of time building something that only a handful of people might use. And low usage introduces its own issues, such as higher likeliness to break because the foundation wikis aren't running it (and so we're not paying attention to it working properly, beyond unit tests that probably won't have enough coverage).

I definitely agree with giving it Lowest priority...

This is pretty much a performance anti-pattern. If someone really wants it they could make a DBFileBackend subclass of FileBackendStore and point $wgLocalFileRepo at it (via the 'backend' field).

(In reply to Brion Vibber from comment #0)

Although there are good reasons to use the filesystem for uploaded files,
there are also problems with it in
some situations:

  • Security -- world-writable directories on multi-user hosts
  • Security -- dangerous filetypes may slip through and execute scripts

server-side

  • Inconvenient filesystem naming limitations (eg bug 362)
  • Stupid configurations -- PHP 'safe mode' and other such things that may

make it difficult for some
installations to have a writable directory tree for uploads

Since this report was filed, it is possible to use other backends like S3, Azure, Ceph, Swift, Gluster (Swift compat code). This handles 1 and 3-4.

https://gerrit.wikimedia.org/r/#/c/125573/ is able to handle encoding issues (3). https://gerrit.wikimedia.org/r/#/c/127460/ could also handle long names.