Page MenuHomePhabricator

Ability to use count via the SMW Ask API
Closed, ResolvedPublic

Description

I would love to add some deeper semantic statistics to WikiApiary beyond what is reported in the smwinfo API method. This enhancement request could be seen as a request to add more to smwinfo, but I think it would be better (and more generic) as an enhancement to the Ask API.

I would specifically like to execute this query via the API:

{{#ask:[[Query size::+]]|format=count}}

by doing:

http://wikiapiary.com/w/api.php?action=ask&query=%5B%5BQuery%20size%3A%3A%2B%5D%5D%7Cformat%3Dcount&format=json

Right now this will result in a PHP exception:

PHP message: PHP Catchable fatal error: Argument 1 passed to ApiSMWQuery::addQueryResult() must be an instance of SMWQueryResult, string given, called in /SemanticMediaWiki/includes/api/ApiAsk.php on line 48 and defined in /SemanticMediaWiki/includes/api/ApiSMWQuery.php on line 82

The Ask API does allow parameters like sort and order to be passed, but it doesn't seem to handle format at all. For most cases format doesn't make sense (table, ul, array). However, aggregation formats methods like count, max, min, earliest, latest do make a lot of sense.

It would be really great to be able to do these queries via the API. I would then add this as extended semantic information to retrieve and graph in WikiAiary. :-)


Version: master
Severity: enhancement

Details

Reference
bz46458

Event Timeline

bzimport raised the priority of this task from to Needs Triage.Nov 22 2014, 1:40 AM
bzimport set Reference to bz46458.
bzimport added a subscriber: Unknown Object (MLST).

As a thought, maybe this could justify a new API method (e.g., action=smwcalc) that allows you to only specify conditions and request average, count, max, median, min, product and sum (all the current calculations). I would put earliest and latest in there too.

Unknown Object (User) added a comment.Mar 22 2013, 6:39 PM

(In reply to comment #0)

I would love to add some deeper semantic statistics to WikiApiary beyond what
is reported in the smwinfo API method. This enhancement request could be seen
as a request to add more to smwinfo, but I think it would be better (and more
generic) as an enhancement to the Ask API.

Generally, results from the API are format independent.

PHP message: PHP Catchable fatal error: Argument 1 passed to
ApiSMWQuery::addQueryResult() must be an instance of SMWQueryResult, string
given, called in /SemanticMediaWiki/includes/api/ApiAsk.php on line 48 and
defined in
/SemanticMediaWiki/includes/api/ApiSMWQuery.php on line 82

This certainly needs fixing because of the type hinting, it comes back with a fatal because getResult() doesn't return an SMWQueryResult object but instead outputs a "naked" number.

Unknown Object (User) added a comment.Mar 22 2013, 6:47 PM

(In reply to comment #1)

As a thought, maybe this could justify a new API method (e.g.,
action=smwcalc)

If some kind of statistics is needed it should go into ApiSMWInfo class (please, no additional action).

that allows you to only specify conditions and request average, count, max,
median, min, product and sum (all the current calculations). I would put
earliest and latest in there too.

Since we expect the API to be format neutral, any specific formatting rules (e.g. max is an application based formatting acting on a result set) to a result set should be done in a post-process.

Count is somewhat different because it is handled differently in the SQLStore together with the actual database query.

Unknown Object (User) added a comment.Apr 2 2013, 7:40 PM

[1] extends the smwinfo api to be able to access additional statistics such as querycount|querysize|conceptcount

[1] https://gerrit.wikimedia.org/r/#/c/57138

Thanks for adding these additional values to the SMWInfo API. I've started grabbing them from sites that are reporting it in WikiApiary.

http://wikiapiary.com/wiki/WikiApiary_talk:Operations/2013/April#Even_more_SMWInfo

Related URL: https://gerrit.wikimedia.org/r/59990 (Gerrit Change Ifb34d9b17a0b34082ee62245ecf5f49dc0b2eec1)

Unknown Object (User) added a comment.Apr 19 2013, 9:19 AM

The above change will add a "formatcount" parameter to ApiSMWInfo which will ouput
an array of used formats together with the count information.

JSON example

{

"info": {
    "formatcount": {
        "datatables": 44,
        "list": 27,
        "jqplotchart": 15,
        "eventcalendar": 15,
        "gallery": 15,
        "jqplotseries": 13,
        "count": 2,
        "sum": 2,
        "rdf": 1
    }
}

}

It is not yet merged with the master ...

That is awesome! Will this be variable to include formats that are added via extensions like Semantic Results Formats? I'm guessing yes since some of the items you list above are not built-in to SMW IIRC.

With your previous change and this one you now have everything I was requesting via my special collection page for WikiApiary except queries by size. See

http://wikiapiary.com/wiki/WikiApiary:Collect_Semantic_MediaWiki_usage_setup

The new querysize output that is in the master branch seems to be the sum of all query sizes. It would be great if similar to what you added with formatcount there could be a sizecount listing as well showing querycount at each size.

This is so awesome to add MWJames. I can't wait to incorporate this into WikiApiary so we can start seeing how deep SMW is incorporated into various wikis!

Just a general comment, any other items you all want to add to SMWInfo I will definitely commit to adding to WikiApiary and graphing over time and showing comparisons across all sites. So, if there are other stats that you all as the core SMW team would like to answer "I wonder how many wikis have queries that take more than x seconds?" for example, add it and I'll start collecting it.

WikiApiary couldn't exist without SMW and I want to create more special capabilities for the SMW community in it. I'd really like to be able to tell someone if their site is using overly burdensome queries.

https://gerrit.wikimedia.org/r/59990 (Gerrit Change Ifb34d9b17a0b34082ee62245ecf5f49dc0b2eec1) | change APPROVED and MERGED [by Jeroen De Dauw]