Various bot commands end with the message:
Trying to find the reason in the code, grep does not find various pieces of that message. Thus I was not able to locate its origin in the code. :-(
Version: core-(2.0)
Severity: normal
URL: http://wiki.piratenpartei.de/wiki/index.php
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| Use cmtype to restrict page type for Cat.articles | pywikibot/core | master | +188 -44 |
I guess the method is site.categorymembers which launches the request. With Namespace Parameter the Maximum values to be returned is limited to 500 for bots and 50 for others. Are you running a bot or without bot flag when that error occurs. Could you give an example to reproduce that bug?
Seems this is a warning only due to $wgMiserMode Parameter and does not affect the results.
Change 136632 had a related patch set uploaded by Xqt:
(bug 65981) reduce limit if $wgMiserMode is set and generator is affected
(In reply to xqt from comment #1)
I guess the method is site.categorymembers which launches the request. With
Namespace Parameter the Maximum values to be returned is limited to 500 for
bots and 50 for others. Are you running a bot or without bot flag when that
error occurs. Could you give an example to reproduce that bug?
Yes, running a test on categories with -simulate and without bot flag.
The wiki is at http://wiki.piratenpartei.de/wiki/index.php and not available in git. It is fairly standard but has tons of local name spaces.
In my test, I do not refer to name spaces explicitly, but of course, category is one of them.
(In reply to xqt from comment #2)
Seems this is a warning only due to $wgMiserMode Parameter and does not
affect the results.
The bot terminates after the warning.
While I guess it might generally be able to continue, it does nothing and at least not what it is expected to do.
(In reply to Gerrit Notification Bot from comment #3)
Change 136632 had a related patch set uploaded by Xqt:
(bug 65981) reduce limit if $wgMiserMode is set and generator is affected
I tried the patch.
It seems not to solve the problem in one instance, but creates more output:
...
max
The "max" are new.
In another test, the patch seems to avoid the error - maybe because another error is hit earlier:
$ python pwb.py category -family:piratewiki -lang:de move -nodelete -inplace -from:ababa -to:abba -simulate -v
COMMAND: ['category', '-family:piratewiki', '-lang:de', 'move', '-nodelete', '-inplace', '-from:ababa', '-to:abba', '-simulate', '-v']
VERSION: ('[ssh] pywikibot-core.git (rPWBC3615274165f7, g3256, 2014/06/01, 13:13:55, OUTDATED)', 'b807b4f704dbfd042b2c1ae0b362a5b04093750e', (1, 22, u'.7'))
SYSTEM: ('Linux', 'tools-login', '3.2.0-59-virtual', '#90-Ubuntu SMP Tue Jan 7 23:02:33 UTC 2014', 'x86_64')
MODULES:
pywikibot/comms/http.py affc571 2014-06-01 00:29:06.175927 pywikibot/data/api.py 1a1ae93 2014-06-01 12:39:25.389090 pywikibot/textlib.py 43dc65b 2014-06-01 00:29:06.535937 pywikibot/i18n.py a7dd59b 2014-06-01 00:29:06.439934 pywikibot/comms/threadedhttp.py 5dd0840 2014-06-01 00:29:06.187927 pywikibot/date.py a1b85ad 2014-02-21 14:38:22 pywikibot/exceptions.py d9e869a 2014-06-01 00:29:06.291930 pywikibot/site.py 3d3f2cf 2014-06-01 00:29:06.515936 pywikibot/bot.py 2e7e859 2014-06-01 00:29:06.159927 pywikibot/throttle.py 1d8da7a 2014-06-01 00:29:06.543937 pywikibot/page.py 01efdf2 2014-06-01 00:29:06.475935 pywikibot/family.py 75ed448 2014-06-01 00:29:06.411933 pywikibot/plural.py 2b02046 2014-02-21 14:38:22 pywikibot/version.py 6d0d30d 2014-06-01 00:29:06.615939 pywikibot/userinterfaces/terminal_interface.py 0e8db29 2014-02-21 14:38:23 pywikibot/userinterfaces/terminal_interface_base.py 745aa26 2014-06-01 00:29:06.579938 pywikibot/config2.py 18b9ca6 2014-06-01 00:29:06.231929 pywikibot/pagegenerators.py 87472d0 2014-06-01 00:29:06.491935 pywikibot/userinterfaces/terminal_interface_unix.py 30e64ec 2014-06-01 00:29:06.591938 pywikibot/tools.py 31ca0ef 2014-06-01 00:29:06.567937 pywikibot/login.py 8f167ac 2014-06-01 00:29:06.455935 pywikibot/userinterfaces/transliteration.py 288348d 2014-06-01 00:29:06.607939
MESSAGES: unknown (not logged in)
Pywikibot ra774f3344f63c8a549779793c192b517e0866527
Python 2.7.3 (default, Feb 27 2014, 19:58:35)
[GCC 4.6.3]
Found 1 piratewiki:de processes running, including this one.
Password for user Purbo_T on piratewiki:de (no characters will be shown):
Logging in to piratewiki:de as Purbo_T
Should be logged in now
But here:
wiki.piratenpartei.de/Spezial:Kategorien/Ababa
you see that the category has one page, so here is something wrong.
Could not reproduce it one wikimedia site. Unfortunately I haven't an account at piratewiki. Anyway: could you give me more results starting of the beginning of your command line. I'll commit a new patch set with hardcoded Limit of 50. Could you try it.
Just a quick look at the patch showed me that this patch won't solve the problem. I hope I'm wrong. I will try to install this on a local wiki and try to test it if it works
Purodha Blissenbach, do you still see the problem you mentioned on 2014-06-01 13:02:28 UTC . That will be masking this bug, and should be raised as a separate bug if it is still occurring.
I believe the root of the problem is that category.py uses moves from the old category
Each of those filters on namespace.
It should do both using the same generator. I believe this is what Amir Ladsgroup was saying in the previous comment.
The alternative is to not use cmnamespace, but use cmtype
Tried again with the most recent version of 2014-09-06 00:00 UTC and still not working:
$ python pwb.py category -family:piratewiki -lang:de move -nodelete -inplace -from:ababa -to:abba -simulate -v
COMMAND: ['category', '-family:piratewiki', '-lang:de', 'move', '-nodelete', '-inplace', '-from:ababa', '-to:abba', '-simulate', '-v']
Found 1 piratewiki:de processes running, including this one.
VERSION: ('[ssh] pywikibot-core.git (rPWBC3615274165f7, g3256, 2014/06/01, 13:13:55, OUTDATED)', 'd3414c6abffdf7dcd0cb3d2bed35430030110a40', (1, 22, u'.9'))
SYSTEM: ('Linux', 'tools-login', '3.2.0-59-virtual', '#90-Ubuntu SMP Tue Jan 7 23:02:33 UTC 2014', 'x86_64')
MODULES:
pywikibot/comms/http.py affc571 2014-06-01 00:29:06.175927 pywikibot/data/api.py 1a1ae93 2014-06-01 12:39:25.389090 pywikibot/textlib.py 43dc65b 2014-06-01 00:29:06.535937 pywikibot/i18n.py a7dd59b 2014-06-01 00:29:06.439934 pywikibot/comms/threadedhttp.py 5dd0840 2014-06-01 00:29:06.187927 pywikibot/date.py a1b85ad 2014-02-21 14:38:22 pywikibot/exceptions.py d9e869a 2014-06-01 00:29:06.291930 pywikibot/site.py 3d3f2cf 2014-06-01 00:29:06.515936 pywikibot/bot.py 2e7e859 2014-06-01 00:29:06.159927 pywikibot/throttle.py 1d8da7a 2014-06-01 00:29:06.543937 pywikibot/page.py 01efdf2 2014-06-01 00:29:06.475935 pywikibot/family.py 75ed448 2014-06-01 00:29:06.411933 pywikibot/plural.py 2b02046 2014-02-21 14:38:22 pywikibot/version.py 6d0d30d 2014-06-01 00:29:06.615939 pywikibot/userinterfaces/terminal_interface.py 0e8db29 2014-02-21 14:38:23 pywikibot/userinterfaces/terminal_interface_base.py 745aa26 2014-06-01 00:29:06.579938 pywikibot/config2.py 18b9ca6 2014-06-01 00:29:06.231929 pywikibot/pagegenerators.py 87472d0 2014-06-01 00:29:06.491935 pywikibot/userinterfaces/terminal_interface_unix.py 30e64ec 2014-06-01 00:29:06.591938 pywikibot/tools.py 31ca0ef 2014-06-01 00:29:06.567937 pywikibot/login.py 8f167ac 2014-06-01 00:29:06.455935 pywikibot/userinterfaces/transliteration.py 288348d 2014-06-01 00:29:06.607939
MESSAGES: unknown (not logged in)
Pywikibot ra774f3344f63c8a549779793c192b517e0866527
Python 2.7.3 (default, Feb 27 2014, 19:58:35)
[GCC 4.6.3]
Password for user Purbo_T on piratewiki:de (no characters will be shown):
Logging in to piratewiki:de as Purbo_T
Should be logged in now
Change 158861 had a related patch set uploaded by XZise:
[FIX] Don't use wikibase and move pages only if valid
Change 158861 merged by jenkins-bot:
[FIX] Don't use wikibase and move pages only if valid
Still reproducible.
$ python pwb.py category -family:piratewiki -lang:de move -nodelete -inplace -from:ababa -to:abba -simulate -v
COMMAND: ['category', '-family:piratewiki', '-lang:de', 'move', '-nodelete', '-inplace', '-from:ababa', '-to:abba', '-simulate', '-v']
DATE: 2014-11-20 16:13:09.409555 UTC
VERSION: [ssh] pywikibot-core.git (60b788c, g4605, 2014/11/20, 22:48:28, OUTDATED)
SYSTEM: ('Linux', '...', '3.17.2-200.fc20.x86_64', '#1 SMP Tue Nov 4 18:04:56 UTC 2014', 'x86_64')
CONFIG FILE DIR: ..
PACKAGES:
_struct (/usr/lib64/python2.7/lib-dynload/_struct.so) = 0.2 backports (/usr/lib/python2.7/site-packages/backports/) = ?? cPickle (/usr/lib64/python2.7/lib-dynload/cPickle.so) = 1.71 ca_certs_locater (/usr/lib/python2.7/site-packages/ca_certs_locater/) = ?? distutils (/usr/lib64/python2.7/distutils/) = 2.7.5 email (/usr/lib64/python2.7/email/) = 4.0.3 google ([path unknown]) = ?? httplib2 (...) = 0.9 jaraco ([path unknown]) = ?? json (/usr/lib64/python2.7/json/) = 2.0.9 logging (/usr/lib64/python2.7/logging/) = 0.5.1.2 logilab (/usr/lib/python2.7/site-packages/logilab/) = ?? mpl_toolkits ([path unknown]) = ?? mwparserfromhell (/usr/lib/python2.7/site-packages/mwparserfromhell-0.3.3-py2.7-linux-x86_64.egg/mwparserfromhell/) = 0.3.3 parser (/usr/lib64/python2.7/lib-dynload/parsermodule.so) = 0.5 paste ([path unknown]) = ?? peak ([path unknown]) = ?? pickle (/usr/lib64/python2.7/pickle.pyc) = $Revision: 72223 $ pkg_resources (/usr/lib/python2.7/site-packages/pkg_resources.pyc) = ?? pywikibot ([path unknown]) = ?? re (/usr/lib64/python2.7/re.pyc) = 2.2.1 sphinxcontrib (/usr/lib/python2.7/site-packages/sphinxcontrib_napoleon-0.2.9-py2.7.egg/sphinxcontrib/) = ?? urllib (/usr/lib64/python2.7/urllib.pyc) = 1.17 urllib2 (/usr/lib64/python2.7/urllib2.pyc) = 2.7 zlib (/usr/lib64/python2.7/lib-dynload/zlibmodule.so) = 1.0 zope ([path unknown]) = ??
MODULES:
pywikibot/comms/http.py f3e93c8 2014-11-20 05:02:35.475199 pywikibot/data/api.py 8aa97b4 2014-11-20 05:02:35.488199 pywikibot/textlib.py 33003b4 2014-11-17 12:49:31.747458 pywikibot/i18n.py 23f218e 2014-11-17 18:36:28.172387 pywikibot/comms/threadedhttp.py 32e55bd 2014-11-17 12:49:31.664459 pywikibot/exceptions.py e711f48 2014-11-20 07:29:58.869259 pywikibot/site.py 8f260e8 2014-11-20 22:48:03.183176 pywikibot/bot.py 9fb6254 2014-11-20 22:24:46.860068 pywikibot/throttle.py 5cdc30d 2014-11-17 12:49:31.756458 pywikibot/page.py 2d71d8a 2014-11-20 07:29:58.918259 pywikibot/family.py 9daf228 2014-11-20 05:02:35.501199 pywikibot/plural.py 02a50e4 2014-11-15 21:56:35.098131 pywikibot/version.py e82f0c5 2014-11-17 12:49:31.789458 pywikibot/userinterfaces/terminal_interface.py b0e2743 2014-11-15 21:56:35.114131 pywikibot/date.py 350bdf7 2014-11-17 12:49:31.701459 pywikibot/userinterfaces/terminal_interface_base.py 5f37e69 2014-11-17 12:49:31.770458 pywikibot/config2.py bdb3e50 2014-11-20 07:29:58.850259 pywikibot/pagegenerators.py f2e8360 2014-11-20 05:02:35.529198 pywikibot/userinterfaces/terminal_interface_unix.py bb6384d 2014-11-17 12:49:31.784458 pywikibot/tools.py d71ad8a 2014-11-20 07:29:58.948258 pywikibot/diff.py 7773b7e 2014-11-17 12:49:31.702459 pywikibot/login.py db7be8f 2014-11-20 07:29:58.891259 pywikibot/userinterfaces/transliteration.py 1d8e217 2014-11-15 21:56:35.120131
PYWIKI REPO VERSION: 7d1d1c761b697b568b91e55746e57dfb73e939d3
Pywikibot re20d7c4e05f696821260ba612d13e1b672759e0a
Python 2.7.5 (default, Nov 3 2014, 14:26:24)
[GCC 4.8.3 20140911 (Red Hat 4.8.3-7)]
Found 1 piratewiki:de processes running, including this one.
Logging in to piratewiki:de as Jayvdb
Should be logged in now
The wiki has 106 namespaces.
http://wiki.piratenpartei.de/wiki/api.php?action=query&meta=siteinfo&siprop=namespaces
The provided patch is not related to this problem.
page.articles/etc should not be doing the following:
namespaces = [x for x in self.site.namespaces()
if x >= 0 and x != 14]They should be using cmtype, but we need to check which version cmtype was introduced.
cmtype was introduced in MW version 1.12.
I tested it on http://rnf.fisica.org.ar/api.php?action=query&list=categorymembers&cmtitle=Category:abc&cmtype=subcat which runs on MW 1.12 and
http://domiwiki.com/api.php?action=query&list=categorymembers&cmtitle=Category:abc&cmtype=subcat which is a MW 1.11 site.
Change 179071 had a related patch set uploaded (by John Vandenberg):
Use cmtype to restrict page type for Cat.articles
Change 179071 merged by jenkins-bot:
Use cmtype to restrict page type for Cat.articles
Change 136632 abandoned by Xqt:
(bug 65981) reduce limit if $wgMiserMode is set and generator is affected
Reason:
see 95c495d81069209d7c7de8cb90f0e3dfd0052b4e