Page MenuHomePhabricator

GlobalUsage not working with a simple one-database with multiple wikis setup (REL1.18)
Closed, DeclinedPublic

Description

Running REL1.18 (http://www.species-id.net/openmedia/Special:Version).

GlobalUsage is installed, the refresh logged many updates, directly checking the table confirms 38716 in openmedia.globalimageusage, but GlobalUsage does not find them.


Test example:
http://species-id.net/openmedia/File:Descurainia_sophia_0_Bl%C3%BCte_IMG_3492_Wohlers.jpg

used on:

http://offene-naturfuehrer.de/web/Sisymbrium_sophia_%E2%80%93_Gemeine_Besenrauke_%28JKI-Pflanzenportraits%29

Existing record in openmedia.globalimageusage table:

gil_wiki gil_page gil_page_namespace_id gil_page_namespace gil_page_title gil_to
onwiki 9589 0 Descurainia_sophia_–_Gemeine_Besenrauke_(JKI-Pflan... Descurainia_sophia_0_Blüte_IMG_3492_Wohlers.jpg

However,

http://species-id.net/o/index.php?title=Special%3AGlobalUsage&target=Descurainia_sophia_0_Blüte_IMG_3492_Wohlers.jpg

returns:

"File:Descurainia sophia 0 Blüte IMG 3492 Wohlers.jpg is not used on other wikis." Same for all other files tested so far.


PS: does GlobalUsage allow to be used like on commons.wikimedia.org, to display a usage at the bottom of each file page? Please mention on http://www.mediawiki.org/wiki/GlobalUsage (both if yes, only if configured, or no).


Version: REL1_18-branch
Severity: critical

Details

Reference
bz30571

Event Timeline

bzimport raised the priority of this task from to Low.Nov 21 2014, 11:54 PM
bzimport added projects: GlobalUsage, TestMe.
bzimport set Reference to bz30571.
bzimport added a subscriber: Unknown Object (MLST).

Bryan.TongMinh wrote:

Does this only happen on files with a special character like ü in the name, or on all files?

Happens with all files.

http://species-id.net/o/index.php?title=Special%3AGlobalUsage&target=Chenopodium_album_1a_im_Stoppelfeld_IMG_2597_Wohlers.JPG


Note: I assumed that all wikis would write their own images into a single global usage table. However, although

cd /var/www/v-species/o; sudo php ./maintenance/sql.php ./extensions/GlobalUsage/GlobalUsage.sql --conf ./LocalSettings.php

was executed only on the shared repository, all wiki checked seem to contain a
globalimagelinks table, and this table is filled with the records for the local images of each wiki.

Since GlobalUsage.sql was definitely not executed for all wikis, either refresh does this, or a general update.php script (which was executed, due to version changes) may do this?


The config for all wikis is:
require_once( "$IP/extensions/GlobalUsage/GlobalUsage.php" );
$wgGlobalUsageDatabase = "openmedia";

The documentation on mediawiki says: "In LocalSettings, set $wgGlobalUsageDatabase to the identifier of the wiki where the GlobalUsage data is located (usually the database name). It should be something understandable to wfGetDB. Example: $wgGlobalUsageDatabase = 'commonswiki';"

I interpret this that this should be the sql database name, i.e. the name in $wgDBname. Is this perhaps incorrect?

Bryan.TongMinh wrote:

(In reply to comment #2)

Note: I assumed that all wikis would write their own images into a single
global usage table. However, although

cd /var/www/v-species/o; sudo php ./maintenance/sql.php
./extensions/GlobalUsage/GlobalUsage.sql --conf ./LocalSettings.php

That is correct.

was executed only on the shared repository, all wiki checked seem to contain a
globalimagelinks table, and this table is filled with the records for the local
images of each wiki.

Since GlobalUsage.sql was definitely not executed for all wikis, either refresh
does this, or a general update.php script (which was executed, due to version
changes) may do this?


This is indeed due to update.php and probably because of an incomplete configuration.

The config for all wikis is:
require_once( "$IP/extensions/GlobalUsage/GlobalUsage.php" );
$wgGlobalUsageDatabase = "openmedia";

The documentation on mediawiki says: "In LocalSettings, set
$wgGlobalUsageDatabase to the identifier of the wiki where the GlobalUsage data
is located (usually the database name). It should be something understandable
to wfGetDB. Example: $wgGlobalUsageDatabase = 'commonswiki';"

I interpret this that this should be the sql database name, i.e. the name in
$wgDBname. Is this perhaps incorrect?

That is indeed incorrect. It should be the name given in a load balancer configuration [1]. I don't know myself how the load balancer works exactly or how to configure it, but perhaps GlobalUsage should be fixed to also work with a given database name.

[1] http://www.mediawiki.org/wiki/Manual:$wgLBFactoryConf

Question: So the extension GlobalUsage, at present cannot be used if you have a wiki farm with a single database server and no database load balancer?

I habe tentatively added to the extension page:

#: At present, this cannot be NOT a database name of the wiki, but must be a name given in a load balancer configuration: http://www.mediawiki.org/wiki/Manual:$wgLBFactoryConf -- it might be that on a setup without a multi-master-database Loadbalancer, the extension is not usable.

I have changed the bug title - can you make it work for $wgDBname relatively easy? I assume that most mediawikis work without $wgLBFactoryConf being set. It seems consistent that if $wgLBFactoryConf is not set, all wikis are databases on the same database host, so all other settings are simply re-used, except for the database one. This would work for us. For others, an additional variable $wgGlobalUsageDatabasePrefix would fix it (one database, tables have different prefixes).

Bryan.TongMinh wrote:

You can use LBFactory_Multi even if you use a single server, multiple databases setup. Unfortunately I don't have time currently to look into in more detail.

(In reply to comment #5)

You can use LBFactory_Multi even if you use a single server, multiple databases
setup.

Not really without an example. I tried to set it up by comparison with the WMF settings, but I am not sure how it should really be working:

$wgLBFactoryConf = array(
'class' => 'LBFactory_Multi',
'sectionsByDB' => array(
'enwiki' => 's1',
'ispiwiki' => 's1',
'k2n_handbook' => 's1',
'k2n_it' => 's1',
'k2n_sl' => 's1',
'k2n_wiki' => 's1',
'liaswiki' => 's1',
'malesiana_wiki' => 's1',
'metawiki' => 's1',
'nnvmwiki' => 's1',
'onwiki' => 's1',
'openmedia' => 's1',
'orowiki' => 's1',
'plantnet_terms_en' => 's1',
'plantnet_terms_fr' => 's1',
'plantnet_uses_en' => 's1',
'plantnet_uses_fr' => 's1',
'plazi_wiki' => 's1',
'pmedwiki' => 's1',
'rosawiki' => 's1',
'specialmedia' => 's1',
'specieswiki' => 's1',
'species_fr' => 's1',
'studienstiftungswiki' => 's1',
'testwiki' => 's1',
'testwiki2' => 's1',
'wbwiki' => 's1',
'zsm_entomology' => 's1'
),
'sectionLoads' => array('s1' => array( 'db1' => 100,),),
'serverTemplate' => array(

'dbname'      => $wgDBname,
'user'        => $wgDBuser,
'password'    => $wgDBpassword,
'type'        => 'mysql',
'flags'       => DBO_DEFAULT,
'max lag'     => 30,

),
'groupLoadsBySection' => array(),
'groupLoadsByDB' => array(),
'hostsByName' => array('db1' => 'localhost',),
'externalLoads' => array(),
'masterTemplateOverrides' => array(),
'externalTemplateOverrides' => array(),
'templateOverridesByCluster' => array(),
'readOnlyBySection' => array(),
);

On the positive side: after deleting all tables, and reinitializing I note:

  1. maintenance/update.php will still create the table globalimagelinks in any wiki, not just in the central repository.
  1. However, the refreshGlobalimagelinks commands, issued for all wikis, now write links only in the repository-globalimagelinks table.
  1. The images, present in the shared repo openmedia, used in offene-naturführer.de, still report that they are not used anywhere. phpmyadmin shows that records are present in the globalimagelink table, however.

Thus the original bug still seems to exist.

Gregor: Do you still face this problem?

(In reply to comment #0)

Running REL1.18 (http://www.species-id.net/openmedia/Special:Version).
http://species-id.net/o/index.php?title=Special%3AGlobalUsage&target=Descurainia_sophia_0_Blüte_IMG_3492_Wohlers.jpg

"You have requested an invalid special page."

Also, that wiki runs 1.18.1. Upgrading to 1.18.5 recommended.

(In reply to comment #7)

Gregor: Do you still face this problem?

Well global usage hasn't changed, so this still exists as an issue.

Arguably though it wouldn't be totally unreasonable to wontfix this and tell people to just set up $wgLBConfFactory. But it is also not unreasonable to make this work without extra effort. Hence lowering priority to low (perhaps should be lowest?)


For reference this is the config you would need if you had 3 wikis, with database names wikidb1, wikidb2, and wikidb3, each with a single master and no slaves. Below is what you would put in LocalSettings.php for wikidb1 (only thing that would change is $wgDBname parameter). If you use prefixes some of the varibles for db names take the form "dbname-prefix", but i can't remember which ones do that and which have separate variables off the top of my head.

$wgGlobalUsageDatabase = 'db name where the global usage db is';
$wgDBname = 'wikidb1';
$wgDBuser = 'your db username'; must be same for all 3 wikis in this simplified setup
$wgDBpassword = 'pass';
Same pass must be usable for all 3 wikis

// In order to get cross db access to work

$wgLBFactoryConf = array(

'class' => 'LBFactory_Multi',

This assumes all databses are on the same server
If the databases were on different servers, you'd
// have each database host be a different s.
'sectionsByDB' => array(

'wikidb1' => 's1', // Assumes db name for first wiki is wikidb1
'wikidb2' => 's1', // and so on.
'wikidb3' => 's1',

),

'sectionLoads' => array(

's1' => array(
    'localhost'  => 0, // All on section s1, which has single master, at localhost.
),

),

'serverTemplate' => array(

'dbname'      => $wgDBname,
'user'          => $wgDBuser,
'password'      => $wgDBpassword,
'type'          => 'mysql',
'flags'          => DBO_DEFAULT,
'max lag'      => 30,

),
);

Thanks for asking back and apologies for the delay in answering. Yes, we never could fix this under 1.18, and we did try using the wgLBConfFactory. But we are presently testing the move to 1.20 and retest global usage with the new code. I will update this bug as soon as I can.

(In reply to Gregor Hagedorn from comment #9)

Thanks for asking back and apologies for the delay in answering. Yes, we
never could fix this under 1.18, and we did try using the wgLBConfFactory.
But we are presently testing the move to 1.20 and retest global usage with
the new code. I will update this bug as soon as I can.

Gregor: How did this work out? And is this still a problem in a supported MediaWiki version?

(In reply to Gregor Hagedorn from comment #9)

Thanks for asking back and apologies for the delay in answering. Yes, we
never could fix this under 1.18, and we did try using the wgLBConfFactory.
But we are presently testing the move to 1.20 and retest global usage with
the new code. I will update this bug as soon as I can.

Gregor: How did this work out? And is this still a problem in a supported MediaWiki version?

Gregor: Could you answer comment 11, please?

Unfortunately closing this report as no further information has been provided.

Gregor: Please feel free to reopen this report if you can provide the information asked for in comment 10 - comment 12 and if this still happens. Thanks!

Gilles raised the priority of this task from Low to Unbreak Now!.Dec 4 2014, 10:12 AM
Gilles moved this task from Untriaged to Done on the Multimedia board.
Gilles lowered the priority of this task from Unbreak Now! to Low.Dec 4 2014, 11:22 AM