Page MenuHomePhabricator

Optimize multiple replace.py instances
Closed, DeclinedPublic

Description

Originally from: http://sourceforge.net/p/pywikipediabot/feature-requests/333/
Reported by: Anonymous user
Created on: 2013-03-12 21:11:04
Subject: Optimize multiple replace.py instances
Original description:
Since the biggest part of the time and I/O resources, in a replace.py based on a xml file, is reading and uncompressing the xml file itself. When this action is done, execute 10 or 50 regex is very similar. I think that the replace.py should be able to apply more -fix in the same time. Also, if the -save parameter is active, each -fix should go in a different file.

I think that passing an array to -fix and -save to be able to do this. This implementation is only an idea, obviusly ;\)

python version.py
Pywikipedia trunk/pywikipedia/ \(r11200, 2013/03/09, 11:43:52, OUTDATED\)
Python 2.7.3 \(default, Sep 26 2012, 21:51:14\)
\[GCC 4.7.2\]
config-settings:
use\_api = True
use\_api\_login = True
unicode test: ok


Version: core-(2.0)
Severity: enhancement
See Also:
https://sourceforge.net/p/pywikipediabot/feature-requests/333

Details

Reference
bz55013

Event Timeline

bzimport raised the priority of this task from to Needs Triage.Nov 22 2014, 2:14 AM
bzimport set Reference to bz55013.
bzimport added a subscriber: Unknown Object (????).

The idea is good, but much more command-line parameters are involved, practically each except -xml and -xmlstart.
Do you have any measurements on time consumption?

Xqt triaged this task as Low priority.Mar 17 2016, 6:02 AM
Xqt subscribed.
This comment was removed by Xqt.
Xqt raised the priority of this task from Low to Medium.Mar 17 2016, 6:03 AM

This might be a kind of async processing of multiple pages in one task. Probable applicable for BaseBot subclass.