Page MenuHomePhabricator

PreloadingItemGenerator crashes with NoPage exception
Closed, ResolvedPublic

Description

The PreloadingItemGenerator does what the name suggests: Preload Wikidata items so that these can be processed quickly.

When it encounters an item that has been deleted it crashes with a "pywikibot.exceptions.NoPage: Page [[wikidata:-1]] doesn't exist." exception.

It shouldn't crash. It should either:

  • Not preload the item and pass the empty item (and let downstream check)
  • Skip the item

Probably best to have the same behavior as with the PreloadingPageGenerator (or does that one crash too?)

Full trace:

Traceback (most recent call last):

File "C:\pywikibot\core\cc_to_sitelinks.py", line 131, in <module>
  main()
File "C:\pywikibot\core\cc_to_sitelinks.py", line 128, in main
  bot.run()
File "C:\pywikibot\core\cc_to_sitelinks.py", line 31, in run
  for item in self.generator:
File "C:\pywikibot\core\pywikibot\pagegenerators.py", line 890, in PreloadingI

temGenerator

  for i in site.preloaditempages(group, step):
File "C:\pywikibot\core\pywikibot\site.py", line 3625, in preloaditempages
  item.get()  # parses json and preloads the various properties
File "C:\pywikibot\core\pywikibot\page.py", line 2592, in get
  super(ItemPage, self).get(force=force, *args)
File "C:\pywikibot\core\pywikibot\page.py", line 2421, in get
  raise pywikibot.NoPage(self)

pywikibot.exceptions.NoPage: Page [[wikidata:-1]] doesn't exist.


Version: core-(2.0)
Severity: normal

Details

Reference
bz58976

Event Timeline

bzimport raised the priority of this task from to Needs Triage.Nov 22 2014, 2:27 AM
bzimport set Reference to bz58976.

Remarks for PreloadingPageenerator in compat which works without exception:

import wikipedia as wp
import pagegenerators
gen = [wp.Page('de', 'user:xqt/NonexistentPage')]
gen = pagegenerators.PreloadingGenerator(gen)
for page in gen:

print page

Getting 1 page from wikipedia:de...
[[de:Benutzer:Xqt/NonexistentPage]]

page.exists()

False

Same behaviour for core. It works without exception.

Change 103900 had a related patch set uploaded by Xqt:
(bug 58976) Do not get() the item straight after preloading it.

https://gerrit.wikimedia.org/r/103900

Change 103900 merged by jenkins-bot:
(bug 58976) Do not get() the item straight after preloading it.

https://gerrit.wikimedia.org/r/103900