Page MenuHomePhabricator

Ask API should return [] for ordered lists and {} for maps that are not ordered
Closed, DeclinedPublic

Description

he api response format loses the sorting order.

This is because "results" is returned as an object instead of as a list.

Currently the response has this structure

{"query":

{"printrequests":[
   {"label":"","typeid":"_wpg","mode":2},                  
   {"label":"Allele1","typeid":"_str","mode":1},
   {"label":"Allele2","typeid":"_str","mode":1},
   {"label":"Magnitude","typeid":"_num","mode":1},
   {"label":"On chromosome","typeid":"_str","mode":1}],

 "results": {
    "Rs121964853(A;A)":
      {"printouts":{
          "Allele1":["A"],
          "Allele2":["A"],
          "Magnitude":[4],"On chromosome":["1"]
        },
      "fulltext":"Rs121964853(A;A)",
      "fullurl":"http:\/\/bots.snpedia.com\/index.php\/Rs121964853(A;A)"
      },

    "Rs121909520(G;G)":
      {"printouts":{
          "Allele1":["G"],
          "Allele2":["G"],
          "Magnitude":[4],
          "On chromosome"["1"]
       },
      "fulltext":"Rs121909520(G;G)",
      "fullurl":"http:\/\/bots.snpedia.com\/index.php\/Rs121909520(G;G)"},

...

I think a more correct structure would be

{"query":

{"printrequests":[
   {"label":"","typeid":"_wpg","mode":2},                  
   {"label":"Allele1","typeid":"_str","mode":1},
   {"label":"Allele2","typeid":"_str","mode":1},
   {"label":"Magnitude","typeid":"_num","mode":1},
   {"label":"On chromosome","typeid":"_str","mode":1}],

 "results": [
      {"printouts":{
          "Allele1":["A"],
          "Allele2":["A"],
          "Magnitude":[4],"On chromosome":["1"]
        },
      "fulltext":"Rs121964853(A;A)",
      "fullurl":"http:\/\/bots.snpedia.com\/index.php\/Rs121964853(A;A)"
      },

      {"printouts":{
          "Allele1":["G"],
          "Allele2":["G"],
          "Magnitude":[4],
          "On chromosome"["1"]
       },
      "fulltext":"Rs121909520(G;G)",
      "fullurl":"http:\/\/bots.snpedia.com\/index.php\/Rs121909520(G;G)"},

Version: REL1_20-branch
Severity: normal

Details

Reference
bz45303

Event Timeline

bzimport raised the priority of this task from to Needs Triage.Nov 22 2014, 1:18 AM
bzimport set Reference to bz45303.
bzimport added a subscriber: Unknown Object (MLST).

*** Bug 45298 has been marked as a duplicate of this bug. ***

This is because "results" is returned as an object instead of as a list.

In JS, maps are order dependent while arrays are not. Don't see how returning results as map would cause a sorting problem. It seems more likely something on the backend changed that causes sort to be no longer applied.

With these settings the API will return the executed SQL queries, so you can check if this is the case:

$wgDebugDumpSql = true;
$wgDebugToolbar = true;

Did the actual output of the API change from arrays to maps recently?

Unknown Object (User) added a comment.Feb 24 2013, 5:14 PM

Just to chip in here, the SMW\DISerializer did not change the output recently and it was always an results dictionary object (rather then the proposed change to an array).

The json response text is correctly ordered, but because the type is a {} instead of a [] all of the libraries which turn the response into an object do not preserve the ordering.

Sorry, I can confirm the comment above. I forgot that my JSON formatter extension was displaying an interpretation of the raw results. Indeed, the raw result from the API is indeed sorted. But once the JSON formatter goes through it the order is lost. The order is also lost when using simplejson in Python. I guess this really isn't a SMW issue, unless the return format is changed which seems unlikely and possibly a bad idea.

I'm going to close this as {} is the correct thing to return for order dependent collections. [] is a list, which is order independent in many languages such as JS.

I recognize the issue has been closed. But will continue to record some relevant notes on this topic.

http://search.cpan.org/~mlehmann/JSON-XS-2.33/XS.pm#JSON_->_PERL

http://www.ietf.org/rfc/rfc4627.txt

An object is an unordered collection of zero or more name/value

pairs, where a name is a string and a value is a string, number,
boolean, null, object, or array.

An array is an ordered sequence of zero or more values.

If that is the JSON spec, then we better adhere to it. The ordered vs unordered discussion came up in the Wikidata team a while back and there we figured that JavaScript treats lists as unordered, and that we thus should use maps if we want order to be retained. The JSON spec was not used in this discussion though. Guess it requires revisiting...

I said in that discussion that Javascript object preserve its ordering, but it must have been old information as the current doc for ECMAscript states that it does not preserve ordering. In our API we use key-value pairs as in the object notation ({}), where the value might be arrays ([]) and in some cases arrays of objects. Arrays are used if we must preserve order, and we make no assumption about order in objects other than what we might deduce from sorting the keys.

That said I think some browsers still preserve order in objects, but as long as it isn't consistent and according to the JSON-spec we can't use it.

See for example the answer about V8 in http://stackoverflow.com/questions/7214293/is-the-order-of-elements-in-a-json-list-maintained

That said I think some browsers still preserve order in objects, but as long
as
it isn't consistent and according to the JSON-spec we can't use it.

Some browsers explicitly re-order, and consider this a feature
https://code.google.com/p/chromium/issues/detail?id=883#c31

https://code.google.com/p/v8/issues/detail?id=164

Aklapper subscribed.

The Semantic MediaWiki developers requested in https://phabricator.wikimedia.org/T64114 to move their task tracking to https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues and to close remaining tasks in Wikimedia Phabricator. If you still face the problem reported in this task in a supported version of SMW, please feel free to transfer your report to https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues . We are sorry for the inconvenience.