Page MenuHomePhabricator

ClearMessageBlobs loop should check slave lag
Closed, ResolvedPublic

Description

ClearMessageBlobs uses TRUNCATE TABLE in a tight loop:

foreach ( $wgConf->getLocalDatabases() as $wiki ) {
	$lb = wfGetLB( $wiki );
	$db = $lb->getConnection( DB_MASTER, array(), $wiki );
	$db->query( "TRUNCATE TABLE " . $db->tableName( 'msg_resource' ), __METHOD__ );
	$db->query( "TRUNCATE TABLE " . $db->tableName( 'msg_resource_links' ), __METHOD__ );
	$lb->reuseConnection( $db );
}

This results in a lot of replication slave lag on WMF production shards.

It should use a batched delete if possible, instead of truncate.

It should definitely check slave lag in the loop.

Details

Reference
bz58433

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 22 2014, 2:19 AM
bzimport set Reference to bz58433.
bzimport added a subscriber: Unknown Object (MLST).

Change 108838 had a related patch set uploaded by Reedy:
Add wfWaitForSlaves() call in ClearMessageBlobs loop

https://gerrit.wikimedia.org/r/108838

(In reply to comment #1)

Change 108838 had a related patch set uploaded by Reedy:
Add wfWaitForSlaves() call in ClearMessageBlobs loop

https://gerrit.wikimedia.org/r/108838

That's the "basic lazy" improvement

Batched deleting shouldn't be too much work to do it

Change 108838 merged by jenkins-bot:
Add wfWaitForSlaves() call in ClearMessageBlobs loop

https://gerrit.wikimedia.org/r/108838

Patch merged; resetting bug status

Krinkle claimed this task.
Krinkle updated the task description. (Show Details)
Krinkle set Security to None.
Krinkle removed a subscriber: Unknown Object (MLST).