Page MenuHomePhabricator

LocalisationUpdate not working again
Closed, ResolvedPublic

Description

Looking at the log - it looks like it hasm't worked at all for a week or so. This is very bad because at translation sprint we promised quick appearance of translations.


Version: unspecified
Severity: major
URL: https://wikitech.wikimedia.org/view/Server_admin_log

Details

Reference
bz32767

Event Timeline

bzimport raised the priority of this task from to High.Nov 22 2014, 12:08 AM
bzimport added a project: Deployments.
bzimport set Reference to bz32767.
bzimport added a subscriber: Unknown Object (MLST).

Looks like it's stuttering since November 9. We've had a lot of translators complaining in the past few days. Server admin log just shows "01:57 logmsgbot: LocalisationUpdate failed".

Yeah, I gotta fix this some time.

Roan, can you please provide some insight in when "some time" may be?

10(In reply to comment #1)

Looks like it's stuttering since November 9.

?

It's been running daily before and after that, It has been broken since the 22nd (Well 23rd, last complete was 22nd), But I don't see anything SAL wise that would standout to me as breaking it.

If you are referring to the daily fails at the same time when it was completing, That was the old/broken LU setup before Roan (and others?) got it up and running properly in puppet I believe.

Roan, can you please add a status update? Niklas just told me that last night's log indicates it still failed, despite your recent fixes.

In the log: svn: Can't open file '/home/wikipedia/l10n/trunk/phase3/.svn/lock': Permission denied

wrong language translatewiki message

i can confirm that the translatewiki message transfer is still buggy. In finnish wikipedia couple days ago some of the flagged revisions messages changed from finnish to english so i changed them locally to finnish.

One example of the changed texts is Revreview-reject-summary-cur.

Attached:

Järjestelmäviestit_–_Wikipedia_1324050451725.png (1×1 px, 123 KB)

(In reply to comment #8)

i can confirm that the translatewiki message transfer is still buggy. In
finnish wikipedia couple days ago some of the flagged revisions messages
changed from finnish to english so i changed them locally to finnish.

One example of the changed texts is Revreview-reject-summary-cur.

This is bug 33122.

(In reply to comment #7)

In the log: svn: Can't open file '/home/wikipedia/l10n/trunk/phase3/.svn/lock':
Permission denied

Yeah, something weird is going on here. It looks like something may still be trying to run the old script. I've put in some debugging code now, hopefully that'll provide some insight after tonight's run.

(In reply to comment #9)

Yeah, something weird is going on here. It looks like something may still be
trying to run the old script. I've put in some debugging code now, hopefully
that'll provide some insight after tonight's run.

So, turns out the old script isn't running anymore, what you were looking at is the old log file. The new log file is at /var/log/l10nupdatelog/l10nupdate.log . However, something keeps truncating the log file every morning at 06:40 UTC (I think it's logrotate -- I submitted a proposed fix in Gerrit), so I put in a cron job that snatches a copy of the log at 05:00 and copies it to /home/catrope/l10nupdate.log

The error message in that log file is something that will be fixed with the changes to the l10nupdate shell script that were already submitted and merged in Gerrit. However, there are some puppet errors on fenari right now, so the changes aren't being applied. I've submitted fixes for these errors in Gerrit, but those have to be reviewed, merged and deployed by someone in ops. Until that happens (hopefully on Monday), LU will keep failing.

This should be fixed now, can someone confirm?

Got word that trunk L10n updates have indeed appeared again in Wikimedia wikis.