Page MenuHomePhabricator

OCG servers fill up their tmpfs
Closed, ResolvedPublic

Description

The tmpfs gets full because of a single page generated.

The latest offender, according to the authors.db file, is

https://en.wikipedia.org/wiki/Information_and_communication_technologies_for_development

please find out what happens as this is blocking production and requiring regular ops intervention.


Version: unspecified
Severity: blocker

Details

Reference
bz71647

Event Timeline

bzimport raised the priority of this task from to Needs Triage.Nov 22 2014, 3:45 AM
bzimport added a project: OCG-PDF-renderer.
bzimport set Reference to bz71647.

I should have added that it fills up because a 32 gb pdf file gets generated.

This happened again, for the same article, this morning. See the Server Admin Log for the timing.

I strongly suspect this has to do with the ocg service trying to embed

https://en.wikipedia.org/w/index.php?title=File:Wireless_Networking_in_the_Developing_World_(WNDW)_Third_Edition.pd

in the bundle/images/ dir I find one pdf file which is huge - it's a collection of scans in pdf form, which are ~ 188 MB each (per page).

As a stopgap solution, I added a <span class="noprint"> around the reference to the file, hoping this will stop the issues.

Change 165041 had a related patch set uploaded by Cscott:
Limit bundle sizes to 512MB (which should be very generous).

https://gerrit.wikimedia.org/r/165041

Change 165034 had a related patch set uploaded by Cscott:
Add options to limit bundle and image size.

https://gerrit.wikimedia.org/r/165034

Change 165041 merged by Cscott:
Limit bundle sizes to 512MB (which should be very generous).

https://gerrit.wikimedia.org/r/165041

Change 165034 merged by jenkins-bot:
Add options to limit bundle and image size.

https://gerrit.wikimedia.org/r/165034

Should be fixed with https://wikitech.wikimedia.org/wiki/OCG/Deployments#2014-10-6_.28afternoon.29

Can't test quite yet since a crashed entry for [[Information and communication technologies for development]] is still in the status queue.