Page MenuHomePhabricator

Allow to configure HTML fetcher max number of parallel requests
Closed, ResolvedPublic

Description

I have a pretty small server and each time I start the creation of a big book, this slows down the whole site. I want to be able to allow the OCG HTML fetcher to request only one page at a time.


Version: unspecified
Severity: enhancement

Details

Reference
bz71895

Event Timeline

bzimport raised the priority of this task from to Needs Triage.Nov 22 2014, 3:43 AM
bzimport set Reference to bz71895.
bzimport added a subscriber: Unknown Object (MLST).

gerritadmin wrote:

Change 167743 had a related patch set uploaded by Cscott:
Add ability to limit the number of simultaneous requests for small wikis.

https://gerrit.wikimedia.org/r/167743

gerritadmin wrote:

Change 167743 merged by jenkins-bot:
Add ability to limit the number of simultaneous requests for small wikis.

https://gerrit.wikimedia.org/r/167743

Kelson: --parallel-request-limit=1 should do what you want now.

@cscott
Thank you! I wanted to test it but unfortunately I didn't achieved to make it working. This seems to be a parameter to the "bundler", but I start the "service"!
How should I use this configuration parameter? I have tried to launch "mw-ocg-service.js --parallel-request-limit=1" but this fails.

Ah -- use the additionalArgs configuration option inside backend.bundler. eg in localsettings.js

"use strict";
module.exports = function(config) {
  config.backend.bundler.additionalArgs.push("--parallel-request-limit=1");
};