_getUserDataOld call from low-level getUrl
Closed, ResolvedPublic
Actions

Assigned To

None

Authored By

	Legoktm
	Sep 24 2013, 10:29 PM

Description

Originally from: http://sourceforge.net/p/pywikipediabot/patches/608/
Reported by: valhallasw
Created on: 2013-04-13 20:51:59
Subject: _getUserDataOld call from low-level getUrl
Original description:
From http://lists.wikimedia.org/pipermail/pywikipedia-l/2012-October/007585.html :

I just wanted to put\(\) a simple page on a MediaWiki 1.16
instance, where I have to use screen scraping \(use\_api=False\).

There is something strange however:

There is an API call invoked by \_getBlocked:

/w/api.php?action=query&format=json&meta=userinfo&uiprop=blockinfo

Here's my backtrace:

File "pywikipedia/wikipedia.py", line 693, in get
expandtemplates = expandtemplates\)

File "pywikipedia/wikipedia.py", line 743, in \_getEditPage
return self.\_getEditPageOld\(get\_redirect, throttle, sysop, oldid, change\_edit\_time\)

File "pywikipedia/wikipedia.py", line 854, in \_getEditPageOld
text = self.site\(\).getUrl\(path, sysop = sysop\)

File "pywikipedia/wikipedia.py", line 5881, in getUrl
self.\_getUserDataOld\(text, sysop = sysop\)

File "pywikipedia/wikipedia.py", line 6016, in \_getUserDataOld
blocked = self.\_getBlock\(sysop = sysop\)

File "pywikipedia/wikipedia.py", line 5424, in \_getBlock
data = query.GetData\(params, self\)

File "pywikipedia/query.py", line 146, in GetData
jsontext = site.getUrl\( path, retry=True, sysop=sysop, data=data\)

getUrl\(\), which is also called from API, seems always
to call \_getUserDataOld\(text\) where text is ... API output
so it tries to do strange things on that and gives warnings
like

Note: this language does not allow global bots.

WARNING: Token not found on wikipedia:pl. You will not be able to edit any page.

which is nonsense since the analyzed text is not HTML - only API output.

If getUrl\(\) is supposed to be a low-level call, why call \_getUserDataOld\(\)
there?

http://www.mediawiki.org/wiki/Special:Code/pywikipedia/7461

has introduced this call there.

It's easily reproducable by this:

import wikipedia
import config
config.use\_api = False
wikipedia.verbose = True
s = wikipedia.getSite\("pl", "wikipedia"\)
p = wikipedia.Page\(s, u"User:Saper"\)
c = p.get\(\)
c += "<\!-- test -->"
p.put\(c, u"Testing wiki", botflag=False\)

//Saper

Version: compat-(1.0)
Severity: minor
See Also:
https://sourceforge.net/p/pywikipediabot/patches/608

Details

Reference: bz54548

	Subject	Repo	Branch	Lines +/-
	Fix T56548, regex failure in _getUserDataOld().	pywikibot/compat	master	+4 -6

Customize query in gerrit

Revisions and Commits

rPWBO pywikibot-compat
	rPWBO3e57f3132952 Fix T56548, regex failure in _getUserDataOld().

Event Timeline

• bzimport raised the priority of this task from to Low.Nov 22 2014, 2:10 AM

• bzimport added a project: Pywikibot-General.

• bzimport set Reference to bz54548.

• bzimport added a subscriber: Unknown Object (????).

Legoktm created this task.Sep 24 2013, 10:29 PM

reproduced, since it's in compat I set priority to minor

Aklapper added a project: Pywikibot-compat.Nov 27 2014, 4:01 PM

jayvdb removed a project: Pywikibot-General.Jun 3 2015, 3:09 AM

jayvdb set Security to None.

jayvdb added a subscriber: saper.

Xqt added a subscriber: Alexsh.Jun 3 2015, 6:53 AM

Find wpEditToken HTML in action=edit page, and line 7511 in current wikipedia.py in compat....regex not match so _getUserDataOld() cannot get edit token.
I fixed it in my local, but I don't have permission to push into gerrit.

@Alexsh: You may use https://tools.wmflabs.org/gerrit-patch-uploader/ or uploade the path here (or mail me). thx

Change 215587 had a related patch set uploaded (by Gerrit Patch Uploader):
Fix T56548, regex failure in _getUserDataOld().

https://gerrit.wikimedia.org/r/215587

gerritbot added a project: Patch-For-Review.Jun 3 2015, 9:22 AM

Alexsh added a comment.Jun 3 2015, 9:26 AM

This comment was removed by Alexsh.

Change 215587 had a related patch set uploaded (by Xqt):
Fix T56548, regex failure in _getUserDataOld().

https://gerrit.wikimedia.org/r/215587

Pywikibot has two versions: Compat and Core. This task was filed about the older version, called Pywikibot-compat, which is not under active development anymore. Hence I'm lowering the priority of this task to reflect the reality. Unfortunately, the Pywikibot team does not have the manpower to retest every single bug report / feature request against the (maintained) Pywikibot code base. Furthermore, the code base of Pywikibot-Compat has changed a lot compared to the code base of Pywikibot-Core so there is a chance that the problem described in this task might not exist anymore. Please help: Unfortunately manpower is limited and does not allow testing every single reported task again. If you have time and interest in Pywikibot, please upgrade to Pywikibot-Core and add a comment to this task if the problem in this task still happens in Pywikibot-Core (or directly edit the task by removing the Pywikibot-compat project and adding the Pywikibot project to this task). To learn more about Pywikibot and to get involved in its development, please check out https://www.mediawiki.org/wiki/Manual:Pywikibot/Development Thank you for your understanding.