TypeError: list indices must be integers, not str
Closed, ResolvedPublic
Actions

Assigned To

None

Authored By

	Multichill
	Jun 14 2014, 5:19 PM

Description

File "C:\pywikibot\coredev\rijksmonumenten_import.py", line 106, in run

  result = self.repo.editEntity(identification, data, summary=summary)
File "C:\pywikibot\coredev\pywikibot\site.py", line 566, in callee
  return fn(self, *args, **kwargs)
File "C:\pywikibot\coredev\pywikibot\site.py", line 3750, in editEntity
  data = req.submit()
File "C:\pywikibot\coredev\pywikibot\data\api.py", line 414, in submit
  message = result["error"]["messages"]["0"]["name"]

TypeError: list indices must be integers, not str
<type 'exceptions.TypeError'>
CRITICAL: Waiting for 1 network thread(s) to finish. Press ctrl-c to abort

We use ["0"] and that used to work. Maybe the upstream api was changed? This is a bit hard to reproduce.

Version: core-(2.0)
Severity: normal
Whiteboard: u=dev c=backend p=0

Details

Reference: bz66619

Event Timeline

• bzimport raised the priority of this task from to Medium.Nov 22 2014, 3:11 AM

• bzimport added a project: Pywikibot-Wikidata.

• bzimport set Reference to bz66619.

• bzimport added a subscriber: Unknown Object (????).

Multichill created this task.Jun 14 2014, 5:19 PM

Change 139794 had a related patch set uploaded by Xqt:
(bug 66619) Ignore TypeError for bug 46535

https://gerrit.wikimedia.org/r/139794

rijksmonumenten_import.py is not part of the framework. Could you give more information about values of editEntity(identification, data, summary=summary)?

Or place a print statement in api.py line 412:

print result["error"]

to get the whole error result

I haven't been able to reproduce this. Bot has been running for quite some time.

My assumption would be that the 'result["error"]["messages"]["0"]["name"]' causes the error because it's not four layers of dicts, but has a list in there somewhere.

Anyone a clue how to trigger an error to see if the format of error messages might have changed?

The relevant snippet that triggered tis bug:

data = {'labels':

    {monument.get('lang'):
     {'language': monument.get('lang'),
      'value': monumentName}
     }
}

identification = {}
summary = u'Creating new item with data from %s' % (monument.get('source'),)
pywikibot.output(summary)
result = self.repo.editEntity(identification, data, summary=summary)

Daniel probably broke it in https://gerrit.wikimedia.org/r/#/c/124323/

I asked him what to expect back in april, but never got a response http://lists.wikimedia.org/pipermail/wikidata-tech/2014-April/000473.html

The Wikidata technical documentation is a mess so I can't find what to expect.

Wikidata people, could you please provide us, your downstream users, with some decent documentation on what to expect?

Maarten, as you declined just to ignore the TypeError, I've uploaded a new path that should log the api request and its error content.

See also bug 45277

I don't know exactly what's happening here but it seems I have a very similar patch in review at https://gerrit.wikimedia.org/r/#/c/138660/

Please note that result["error"]["messages"] is not an array but an object (at least that's what I see in the JavaScript code my patch touches). I wonder why Python complains. Accessing the elements via ["0"] should work.

"0" is a string. If I would have a dict and a list:

somedict = { "0" : "bla", "1" : "more bla" }
somelist = ["bla", "more bla"]

I would access the "bla with somedict["0"] ("0" is a string here) and in the list I would do somelist[0] to access "bla.

somedict = { "0" : "bla", "1" : "more bla" }
somelist = ["bla", "more bla"]
print somedict["0"]

bla

print somelist["0"]

Traceback (most recent call last):

File "<stdin>", line 1, in <module>

TypeError: list indices must be integers, not str

print somelist[0]

bla

So if someone changed the output from a dict ("0") to list (0), that would explain this error.

In past we had a structure like the following for the error:

{u'error': {u'code': u'failed-save',

u'info': u'Another item (Q1151870) already has label "disambiguation" and description "Wikimedia disambiguation page" associated with language code en.',
u'messages': {u'0': {u'name': u'wikibase-error-label-not-unique-item',
                     u'parameters': [u'disambiguation',
                                     u'en',
                                     {},
                                     u'Wikimedia disambiguation page'],
                     u'type': u'error'},
              u'html': {u'*': u'<p>Another item (Q1151870) already has label "disambiguation" and description "Wikimedia disambiguation page" associated with language code en.\n</p>'}}},

u'servedby': u'mw1199'}

now result['error']['messages']['0']['name'] does not work anymore to get the u'wikibase-error-label-not-unique-item',

Guys, you realize somebody needs to post an example API request that contains an error message and shows the wrong structure we are talking about? As I said, from what I know there is nothing wrong with the API. At the moment this looks like a coding error in Pywikibot to me. Which is nothing the Wikidata team can fix.

Yup, someone changed the output of the api, bot just crashed with this message:

{u'servedby': u'mw1129', u'error': {u'messages': [{u'html': {u'*': u'Could not create a new page.\nIt already exists.'}, u'name': u'edit-already-exists', u'parameters': []}]}}

(In reply to Maarten Dammers from comment #12)

bot just crashed with this message:

Again: Example URL?

(In reply to Maarten Dammers from comment #12)

Yup, someone changed the output of the api, bot just crashed with this
message:

{u'servedby': u'mw1129', u'error': {u'messages': [{u'html': {u'*': u'Could
not create a new page.\nIt already exists.'}, u'name':
u'edit-already-exists', u'parameters': []}]}}

Has anyone seen this bug except with 'edit-already-exists'? It may be that one error is problematic, as it is a lower level part of the code which emits this error.

(In reply to Thiemo Mättig from comment #13)

(In reply to Maarten Dammers from comment #12)

bot just crashed with this message:

Again: Example URL?

Not possible. All POST and intermittent.

I uploaded a new PS to fix the API breaking change in gerrit 139794

I was thinking. Both cases might be mixed depending on what module is throwing the error. Maybe we should grab ['error']['messages'] and check if it's a list or a dict and depending on that, grab the message.

That way we don't care any more about the format, we just accept both formats.

Yes, have a look at my PS.

Ok, I had a chance to talk to the rest of the Wikidata team. The fact that "messages" was not an array was a bug. It should be an array (with numeric keys, obviously).

The problem is that there are many different places where error messages are build. We started unifying them by introducing Wikibase ApiErrorReporter but this may not have cleaned up all instances. If you find situations where "messages" still is an object (with string keys) please report them.

Change 152036 had a related patch set uploaded by XZise:
API change: error message is a list not a dict

https://gerrit.wikimedia.org/r/152036

Change 139794 merged by jenkins-bot:
(bug 66619) API change: error message is a list not a dict

https://gerrit.wikimedia.org/r/139794

Change 152036 abandoned by Xqt:
API change: error message is a list not a dict

Reason:
https://gerrit.wikimedia.org/r/#/c/139794/ is already merged.

https://gerrit.wikimedia.org/r/152036

Aklapper added a project: Pywikibot.Nov 27 2014, 4:15 PM