Page MenuHomePhabricator

performance review of WikibaseQueryEngine
Closed, InvalidPublic

Description

We want to deploy simple query functionality for Wikidata. For this a performance review of WikibaseQueryEngine is needed. The code is at https://github.com/wmde/WikibaseQueryEngine.


Version: master
Severity: normal
Whiteboard: u=dev c=backend p=0

Details

Reference
bz67535

Event Timeline

bzimport raised the priority of this task from to Unbreak Now!.Nov 22 2014, 3:35 AM
bzimport set Reference to bz67535.
bzimport added a subscriber: Unknown Object (MLST).

Important dependency to also check out:
"doctrine/dbal": "~2.4"

I'm a bit worried about the update script - it looks like it wants to directly modify the schema. It'd make more sense in our environment to spit out the SQL that would modify it.

Also, not to be a tool bigot - but it'd be neat to implement an Elasticsearch backed version of this - it'd be simpler in many ways. Probably more complex in others.

Re update script: we are aware of this. The CLI can already dump the schema creation code for exactly that purpouse

Note there is a test in the performance group that is excluded by default. I used this to get a quick idea of how quick inserts are, at least when there is little data in the store. https://github.com/wmde/WikibaseQueryEngine/blob/master/tests/Integration/PerformanceTest.php

Poke for status - I'm really just looking for some general hint on Ori would be able to start the review. Or if I should bounce to Aaron?

Swapping over to Aaron because Ori is crushed under HHVM.

Some status on this - the tool is being rewritten to remove its dependency on external libraries. If possible it'd be cool to get started on the review before the rewrite. It _shouldn't_ be a huge rewrite - just replacing some of it.

the tool is being rewritten

It is not being "rewritten" - most of it will remain the same.

to remove its dependency on external libraries

That is being done, though does not involve rewriting the component.

Lydia_Pintscher removed a subscriber: Unknown Object (MLST).
Lydia_Pintscher removed a subscriber: Unknown Object (MLST).Dec 1 2014, 2:32 PM
Lydia_Pintscher lowered the priority of this task from Unbreak Now! to Low.Feb 19 2015, 11:49 AM
Lydia_Pintscher set Security to None.
Lydia_Pintscher claimed this task.

I'm closing this for now as it is unclear if we're going to deploy it ever on the WMF cluster.