Page MenuHomePhabricator

DRTRIGON-90 sequence of sometimes occuring "KeyError: u'parse'"
Closed, DeclinedPublic

Description

This issue was converted from https://jira.toolserver.org/browse/DRTRIGON-90.
Summary: sequence of sometimes occuring "KeyError: u'parse'"
Issue type: Sub-task - The sub-task of the issue
Priority: Minor
Status: In Progress
Assignee: drtrigon <dr.trigon@surfeu.ch>


From: drtrigon <dr.trigon@surfeu.ch>

Date: Sat, 09 Jul 2011 11:14:57

The error:

Traceback (most recent call last):

File "/home/drtrigon/pywikipedia/bot_control.py", line 225, in run
self.bot.main()
File "/home/drtrigon/pywikipedia/sum_disc.py", line 1307, in main
bot.run()
File "/home/drtrigon/pywikipedia/sum_disc.py", line 369, in run
self.getLatestRelevantNews()
File "/home/drtrigon/pywikipedia/sum_disc.py", line 747, in getLatestRelevantNews
entries = PageSections(page, self._param)
File "/home/drtrigon/pywikipedia/sum_disc.py", line 1138, in init
sections = page.getSections(minLevel=1)
File "/home/drtrigon/pywikipedia/dtbext/dtbext_wikipedia.py", line 50, in <lambda>
obj.dict[\'getSections\'] = lambda *args, kwds: Page.dict[\'getSections\'](obj, *args, kwds)
File "/home/drtrigon/pywikipedia/dtbext/dtbext_wikipedia.py", line 160, in getSections
r = result[u\'parse\'][u\'sections\']
KeyError: u'parse'

occurs from time to time - look at [1] and older changes. As temporary work-a-round the error raised is converted to

pywikibot.Error('Problem occured during data retrieval for sections in %s!' % self.title(asLink=True))

and the 'result' dict gets printed into the log file to be able to track the origin of this down.

In future there should be a final solution to this bug:

  • either keep the current code (without the print) which converts the error such that later the whole page gets processed in one part (fall back)
  • or add a loop with e.g. maximal 3 re-tries (like for low level url retrieval) and then use e.g. the same fall back like before

[1] http://de.wikipedia.org/w/index.php?title=Benutzer:DrTrigonBot/Status&direction=prev&oldid=90413407


Version: unspecified
Severity: minor

Details

Reference
bz59525

Event Timeline

bzimport raised the priority of this task from to Needs Triage.Nov 22 2014, 2:36 AM
bzimport set Reference to bz59525.

From: drtrigon <dr.trigon@surfeu.ch>

Date: Sat, 06 Aug 2011 13:01:29

With r135 the bot should notify SGE in case of errors and be re-runned by SGE. This could also solve that problem.


From: drtrigon <dr.trigon@surfeu.ch>

Date: Thu, 25 Aug 2011 14:48:42

Seams to occur rarely. Since creation of this ticket and introduction of work-a-round in code at '09 July' until today '25 Aug.' no such event was logged. You should find a debug output prior to a "Problem resolving section data" message, as you can see by looking into 'dtbext.wikipedia.getSections'.


From: drtrigon <dr.trigon@surfeu.ch>

Date: Fri, 23 Dec 2011 19:37:28

The code introduced in r158 to track down this issue is somehow useless since it triggers now a "Problem resolving section data, processing the whole page at once..." which is the most often triggered problem.

Thus the print in front of the raise was enhanced with the message "Query result:" - this string should be simpler to find in the log files in order to reproduce this bug.

Aklapper subscribed.

Declining this task as this project is not active anymore. See T300969 for more information.