Nemo: Why is this under the "Other" component instead of "tools" (or "Tool labs tools" product) if it's hosted on tools.wmflabs.org/* ?

[Moving to "Tool Labs tools" as this is about a specific tool and not the general Wikimedia Tool Labs infrastructure. Petr and Yuvi are listed as maintainers on http://tools.wmflabs.org/ and hence CC'ed.]

If this requires fixing on an Labs Infrastructure level, please move to "Wikimedia Labs > tools". Thanks!

(I've no idea why I'm listed as maintainer)

(In reply to comment #2)

If this requires fixing on an Labs Infrastructure level, please move to
"Wikimedia Labs > tools". Thanks!

As I said, this is almost surely the same problem as in http://lists.wikimedia.org/pipermail/labs-l/2013-September/001572.html
A tool is 30 times slower on TL than on TS: until an alternative implementation/actionable solution is suggested to the maintainer, I consider this a bug in TL.

(In reply to comment #3)

(I've no idea why I'm listed as maintainer)

Ah, sorry. No idea how to get that fixed.

(In reply to comment #0)

This tool is a must have [...]

We already have [[m:Special:CentralAuth/Nemo_bis]]. If Special:CentralAuth is insufficient, I think we should focus on making it better rather than waste time porting tools between non-production hosts. If this is a must have, as you say.

(In reply to comment #6)

(In reply to comment #0)

This tool is a must have [...]

We already have [[m:Special:CentralAuth/Nemo_bis]]. If Special:CentralAuth is
insufficient, I think we should focus on making it better rather than waste
time porting tools between non-production hosts. If this is a must have, as
you
say.

Feel free to propose a redesign of the special page and/or to submit patches. Tools are not going to disappear.

The 26ms of overhead it takes to have a query result cannot be circumvented; it is the actual network roundtrip between the database replicas and where the tool labs is currently located (in different data centers).

Moving tool labs to eqiad (where the replicas live) is on the roadmap for 4Q 2013, but in the meantime there is - literally - nothing that can be done to improve this short of increasing the speed of light.

That said, tools can exploit parallelism of the databases to good effect to circumvent the problem (i.e.: selecting from several databases in one query), or simply make the queries asynchronously (the results will still take 26ms to get back, but if you do them simultaneously rather than sequentially that is not an issue anymore).

I'm closing as "WONTFIX" since there isn't a "Unfixable-in-the-current-setup-but-will-not-be-an-issue-once-labs-moved" resolution. :-)

(In reply to comment #8)

I'm closing as "WONTFIX" since there isn't a
"Unfixable-in-the-current-setup-but-will-not-be-an-issue-once-labs-moved"
resolution. :-)

I think --- is the best resolution for "should be fixed at some point but we'll need to check".

Setting lowest priority, then.

In Toolserver top50 with about 16k requests per day, for the records.
http://lists.wikimedia.org/pipermail/toolserver-l/2013-October/006337.html
That's about 20 % of September 2013 Tool Labs hits.
http://tools.wmflabs.org/awstats/cgi-bin/awstats.pl?month=09&year=2013&output=urldetail&config=tools.wmflabs.org&framename=index

Regardless of popularity, this cannot be made faster as it is currently written before labs move save by changing the laws of physics.

The tools makes ~1000 database queries serially; unless and until it is made to do them with at least some parallelism, it will remain highly dependent on latency.

I hadn't realized until recently that [[m:Special:CentralAuth/Nemo_bis]] now includes an edit count column. What additional functionality is needed before we can point the "sulinfo" interwiki prefix to "Special:CentralAuth"?

(In reply to comment #12)

Regardless of popularity, this cannot be made faster as it is currently
written
before labs move save by changing the laws of physics.

Platonides already pointed out on the mailing lists some solutions which require far less than breaking laws of physics.

(In reply to comment #13)

I hadn't realized until recently that [[m:Special:CentralAuth/Nemo_bis]] now
includes an edit count column. What additional functionality is needed before
we can point the "sulinfo" interwiki prefix to "Special:CentralAuth"?

This should read "sulutil" where it says "sulinfo". (I could've sworn it was "sulinfo"....)

I've formally proposed this change at [[m:Special:Permalink/6129569#sulutil]] if anyone following this bug is interested.

(In reply to comment #12)

Regardless of popularity, this cannot be made faster as it is currently
written before labs move save by changing the laws of physics.

I'm reminded of bug 55929.

The Toolserver tool doesn't seem to be doing so great... when I tried to access https://toolserver.org/~quentinv57/sulinfo/Nemo_bis earlier today, it took 9m30.186s for curl to finish. I just tried again and it took 15m47.230s. :-/

• Bug 58890 has been marked as a duplicate of this bug. ***

For the record, quentinv57's tool has been migrated to https://tools.wmflabs.org/quentinv57-tools/tools/sulinfo.php

(In reply to comment #19)

For the record, quentinv57's tool has been migrated to
https://tools.wmflabs.org/quentinv57-tools/tools/sulinfo.php

But it is also suffering from the latency issue.

(In reply to comment #20)

(In reply to comment #19)

For the record, quentinv57's tool has been migrated to
https://tools.wmflabs.org/quentinv57-tools/tools/sulinfo.php

But it is also suffering from the latency issue.

This tool is highly depended by ACC, so I was wondering when this can be resolved?

Sorry for constantly posting btw.

It's been highly optimized and now also moved to eqiad. Shouldn't take more that a few seconds to load SUL profiles. High profile users may take a bit longer but the max I got for such users was 5-6 seconds.

I'm now closing this as fixed.

Thanks!

$ curl https://tools.wmflabs.org/quentinv57-tools/tools/sulinfo.php?username=SieBot > /dev/null

% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                               Dload  Upload   Total   Spent    Left  Speed

100 100k 0 100k 0 0 33092 0 --:--:-- 0:00:03 --:--:-- 33109