Page MenuHomePhabricator

-catr: pagegenerator gets stuck in endless loop and crashes with a RuntimeError: maximum recursion depth exceeded in cmp
Open, LowPublic

Description

I'm running:

pwb.py claimit.py -lang:tl P31 Q55488 P17 Q928 -catr:Category:Mga_estasyon_ng_daangbakal_ng_Pilipinas

Processing [[tl:Estasyong daangbakal ng Abenidang E. Delos Santos]]
Processing [[tl:Estasyong daangbakal ng Laong Laan]]
Processing [[tl:Estasyong daangbakal ng Nichols]]
Processing [[tl:Estasyong daangbakal ng Santa Mesa]]
Processing [[tl:Estasyong Abad Santos ng LRT]]
Processing [[tl:Estasyong Anonas ng LRT]]
[[tl:Estasyong Anonas ng LRT]] doesn't have a wikidata item :(
Processing [[tl:Estasyong Baclaran ng LRT]]
Processing [[tl:Estasyong Betty Go-Belmonte ng LRT]]
Processing [[tl:Estasyong Blumentritt ng LRT]]
Processing [[tl:Estasyong Sentrong Araneta-Cubao ng LRT]]
[[tl:Estasyong Sentrong Araneta-Cubao ng LRT]] doesn't have a wikidata item :(
Processing [[tl:Estasyong Abenidang E.Delos Santos ng LRT]]
Processing [[tl:Estasyong Doroteo Jose ng LRT]]
[[tl:Estasyong Doroteo Jose ng LRT]] doesn't have a wikidata item :(
Processing [[tl:Estasyong Gilmore ng LRT]]
[[tl:Estasyong Gilmore ng LRT]] doesn't have a wikidata item :(
Processing [[tl:Estasyong J. Ruiz ng LRT]]
Processing [[tl:Estasyong Katipunan ng LRT]]
Processing [[tl:Estasyong Legarda ng LRT]]
Processing [[tl:Estasyong Quirino ng LRT]]
Processing [[tl:Estasyong Recto ng LRT]]
Processing [[tl:Estasyong Santolan ng LRT]]
[[tl:Estasyong Santolan ng LRT]] doesn't have a wikidata item :(
Processing [[tl:Estasyong Tayuman ng LRT]]
Processing [[tl:Estasyong V. Mapa ng LRT]]
Processing [[tl:Estasyong Ayala ng MRT]]
Processing [[tl:Estasyong Boni ng MRT]]
Processing [[tl:Estasyong Buendia ng MRT]]
Processing [[tl:Estasyong Sentrong Araneta-Cubao ng MRT]]
Processing [[tl:Estasyong Guadalupe ng MRT]]
Processing [[tl:Estasyong Abenida Hilaga ng MRT]]
Processing [[tl:Estasyong Kamuning ng MRT]]
Processing [[tl:Estasyong Magallanes ng MRT]]
Processing [[tl:Estasyong Ortigas ng MRT]]
Processing [[tl:Estasyong Abenida Quezon ng MRT]]
Processing [[tl:Estasyong Bulebar Shaw ng MRT]]
Processing [[tl:Estasyong Santolan ng MRT]]
Processing [[tl:Estasyong Abenida Taft ng MRT]]
Traceback (most recent call last):

File "C:\pywikibot\core\pwb.py", line 143, in <module>
  run_python_file(fn, argv, argvu)
File "C:\pywikibot\core\pwb.py", line 67, in run_python_file
  exec(compile(source, filename, "exec"), main_mod.__dict__)
File "C:\pywikibot\core\scripts\claimit.py", line 219, in <module>
  main()
File "C:\pywikibot\core\scripts\claimit.py", line 216, in main
  bot.run()
File "C:\pywikibot\core\scripts\claimit.py", line 111, in run
  for page in self.generator:
File "C:\pywikibot\core\pywikibot\pagegenerators.py", line 799, in DuplicateFi

lterPageGenerator

  for page in generator:
File "C:\pywikibot\core\pywikibot\pagegenerators.py", line 661, in Categorized

PageGenerator

  for a in category.articles(**kwargs):
File "C:\pywikibot\core\pywikibot\page.py", line 1879, in articles
  endsort=endsort

(.....)

File "C:\pywikibot\core\pywikibot\page.py", line 1879, in articles
  endsort=endsort
File "C:\pywikibot\core\pywikibot\page.py", line 1879, in articles
  endsort=endsort
File "C:\pywikibot\core\pywikibot\page.py", line 1879, in articles
  endsort=endsort
File "C:\pywikibot\core\pywikibot\page.py", line 1879, in articles
  endsort=endsort
File "C:\pywikibot\core\pywikibot\page.py", line 1879, in articles
  endsort=endsort
File "C:\pywikibot\core\pywikibot\page.py", line 1871, in articles
  for subcat in self.subcategories(step=step):
File "C:\pywikibot\core\pywikibot\page.py", line 1777, in subcategories
  total=total, content=content):
File "C:\pywikibot\core\pywikibot\site.py", line 1833, in categorymembers
  **cmargs)
File "C:\pywikibot\core\pywikibot\site.py", line 832, in _generator
  gen = gen_class(type_arg, site=self, **args)
File "C:\pywikibot\core\pywikibot\data\api.py", line 786, in __init__
  QueryGenerator.__init__(self, generator=generator, **kwargs)
File "C:\pywikibot\core\pywikibot\data\api.py", line 517, in __init__
  self.request = Request(**kwargs)
File "C:\pywikibot\core\pywikibot\data\api.py", line 137, in __init__
  self.update(**kwargs)
File "C:\Python27\lib\_abcoll.py", line 492, in update
  if isinstance(other, Mapping):
File "C:\Python27\lib\abc.py", line 141, in __instancecheck__
  subtype in cls._abc_negative_cache):
File "C:\Python27\lib\_weakrefset.py", line 73, in __contains__
  return wr in self.data

RuntimeError: maximum recursion depth exceeded in cmp
<type 'exceptions.RuntimeError'>
CRITICAL: Waiting for 1 network thread(s) to finish. Press ctrl-c to abort


Version: core-(2.0)
Severity: normal

Details

Reference
bz62964

Event Timeline

bzimport raised the priority of this task from to Low.Nov 22 2014, 2:54 AM
bzimport set Reference to bz62964.
bzimport added a subscriber: Unknown Object (????).

@Multichill, we could probably prevent this and similar by adding duplicate detection in the generator. But that has a cost on memory for very large bot runs. Any other ideas on how to fix this?

@Xqt wasn't this been solved recently?