Page MenuHomePhabricator

Huggle adding wrong level warnings when warning users
Closed, ResolvedPublic

Assigned To
Authored By
Josve05a
Jan 12 2014, 8:13 PM
Referenced Files
F12782: tst_testmain.exe
Nov 22 2014, 2:41 AM
F12781: tst_testmain.exe
Nov 22 2014, 2:41 AM
F12780: tst_testmain.exe
Nov 22 2014, 2:41 AM
F12779: tst_testmain.exe
Nov 22 2014, 2:41 AM
F12777: tst_testmain.exe
Nov 22 2014, 2:41 AM
F12778: Qt5Test.dll
Nov 22 2014, 2:41 AM

Description

Huggle 3.0 Build 796 adds a lvl 2 warning even if one exists already.

See [https://en.wikipedia.org/w/index.php?title=User_talk%3A71.108.77.231&diff=590405095&oldid=590404758 this edit].


Version: 3.0.0
Severity: normal

Details

Reference
bz59982

Event Timeline

bzimport raised the priority of this task from to Low.Nov 22 2014, 2:41 AM
bzimport added a project: Huggle.
bzimport set Reference to bz59982.

There may be several reasons for this, either:

  • huggle didn't recognize the lvl 2 warning, because there was no known pattern in the message
  • huggle ignored the lvl 2 warning, because it was old
  • MOST POSSIBLE huggle didn't know about the warning because it was sent between last check of talk page and revert, this can be actually fixed by extra TP re-parse before each user warning OR by fetching the edit token in same moment as first re-parse and reusing it later, in case it fail, it means page was edited and new token needs to be obtained as well as check for templates needs to be done again.
  • Bug 61933 has been marked as a duplicate of this bug. ***
  • Bug 62646 has been marked as a duplicate of this bug. ***

These warnings are giving basetimestamp so mediawiki should reject them unless huggle did parse the recent talk page version, it seems to be a bug in mediawiki?

copied from bug 61933:

se4598 2014-03-03 20:40:51 CET

Which language does your system have? and whats "February" in that language?

If it's not english, then likely that's the cause.
(The section was skipped because the date of the signature was considered invalid b/c parsed with a non english locale;
https://github.com/huggle/huggle3-qt-lx/blob/master/huggle/huggleparser.cpp#L307 following )

gladjonatan 2014-03-03 20:46:40 CET

My system runs with Swedish, and February in Swedish is februari or Februari. (It does not count as a name in Swedish, therefor not written with a capital letter. Except when in beginnings of sentences and stuff like that.)

  • end of copy

additional info: QDate::fromString() is used with MMMM, which considers all long month names, which are "... localized according to the system's default locale settings."

Now Huggle is breaking down totaly.

In this edit (https://en.wikipedia.org/w/index.php?title=User_talk%3ADustyF&diff=599622995&oldid=599622795 ) it added a lvl 1, when a lvl was issued 2 min earlier.

In this edit (https://en.wikipedia.org/w/index.php?title=User_talk%3AWikihelper123454&diff=599621716&oldid=599618528 ) it is adding a lvl 2, when a lvl was already issued ca. 20 min earlier.

On the same user talk page as above, this edit (https://en.wikipedia.org/w/index.php?title=User_talk:Wikihelper123454&diff=next&oldid=599621716 ) added another lvl2, making it 3 lvl 2 on the same talk page.

the third one is different from the others because here a lvl 2 warning was made despite you/huggle itself posted the previous lvl 2 warning. That's maybe a different cause.

I will give a list of url, what lvl HG3 posted, what lvl was already there, and how llong between:

*https://en.wikipedia.org/w/index.php?title=User_talk%3A71.11.159.0&diff=599622987&oldid=599622573
lvl 1
lvl 2
4 min

*https://en.wikipedia.org/w/index.php?title=User_talk%3ARyanhalo98&diff=599622024&oldid=599520539
lvl 1
lvl 1
18 h

*https://en.wikipedia.org/w/index.php?title=User_talk%3A204.129.176.11&diff=599621845&oldid=599619789
lvl 1
lvl 1
14 min

The list goes on, almost 9/10 of all my edits is this way since I updated to the recent build relesed.

I don't think there was any significant change since last release, so this would happen even if you used the previous version.

Problem here is that huggle doesn't see the talk page, before it post a message, it is mediawiki which merges the warning between the warnings of others. Only proper solution here would be to have mediawiki reject the message in case there was an edit to talk page so that huggle re-parse it, but I don't know if mediawiki can do that.

Other workaround which isn't reliable, would be just to re-parse the talk page everytime before the warning is sent. This has 2 issues:

It significantly worsen the performance and it isn't reliable because some user could in theory send a warning between the check and edit, there is also replication lag (reading page may return older version, while when you write to a page, you access the master sql server which contains the current version) I think this needs to be solved on mediawiki level

I am lowering priority because this should now happen less since I implemented expiration time, but until bug https://bugzilla.wikimedia.org/show_bug.cgi?id=62698 is fixed it may still happen

i know that it is fixed and will be in the next pre-built build. But I thought I might as well post this from the 1024 (or what ever buid it is, that is live right now).

https://en.wikipedia.org/w/index.php?title=User_talk%3A190.213.2.61&diff=600157662&oldid=600156817
In these 2 edits, Huggle 3 added 2 lvl 3 warnings after a level 4 already had been given.

As long as the next build is not relesed (for those who don't have the techincal skils to build it them selfs) these kinds of errors will happen.

It was already released, try latest version.

But it still may happen because the limit time is 40 seconds, that means if cached talk page is older than 40 seconds, it is auto-parsed again, however if someone send a warning within 40 seconds, it will be scored wrong again. This is something that must be fixed on mediawiki side, so that huggle can safely recognize edit conflicts. As long as mediawiki is not able to tell the client that edit conflict happened, and instead try to resolve it itself, which it now does, this will not be reliable.

As far as i can tell from glancing over my edits, it still seems that warnings levels are often incorrect. For example:

I am currently using build 1076 which is currently the latest released build. As far as i can tell Huggle only ups the warning level if i warned a user before. If another editor warned the editor (Irregardless of when, be in in 40 seconds or after 40 seconds) this isn't detected or adjusted for when placing the warning. Looking at the page history it will not display the "1", "2" and so icons for warning level either unless i warned a user before.

These icons are different bug, I don't like idea of parsing the talk page on every new edit just to have fancy icons, because that would slow huggle down a lot, instead I think we should fetch the information from IRC feed (if someone send a warning to user they usually have some summary like "Sending warning" etc which would increment the level.

I will make a unit test from every talk page which failed this, so keep posting them in this bug (everytime you see huggle issued wrong level, just post it here), so that we ensure new version would do that right

I've been testing the new 1096 revision for a little bit now and so far the results are promising. After manually checked the warnings levels for the past 25 or so reverts i can say that all of them seem to be spot on so far.

There is one thing i did notice, not sure if it is related:

Huggle did not place a warning though - instead this was displayed in the event log:
"wo 2. apr 16:25:12 2014 ERROR: Unable to deliver the message to 65.18.38.128: Unknown error: articleexists"

I haven't seen this one before so i'm not sure if this is related to the latest changes or if this is something separate altogether.

(In reply to gladjonatan from comment #20)

Using the latest build in this edit HG3 added a lvl 2, when it should have
added a lvl 1
https://en.wikipedia.org/w/index.php?title=User_talk%3A209.232.147.
121&diff=602447725&oldid=596077638

https://github.com/huggle/huggle3-qt-lx/commit/4e36234a5eee35b4dd22fd50a0d4e56a091e4f8b fixes it

(In reply to Excirial from comment #19)

I've been testing the new 1096 revision for a little bit now and so far the
results are promising. After manually checked the warnings levels for the
past 25 or so reverts i can say that all of them seem to be spot on so far.

There is one thing i did notice, not sure if it is related:

  • On 16:21 today this edit was reverted by another user:

https://en.wikipedia.org/w/index.
php?title=Diary_of_a_Wimpy_Kid&diff=602436176&oldid=602436142

  • On 16:23 the IP editor blanks his talk page:

https://en.wikipedia.org/w/index.php?title=User_talk%3A65.18.38.
128&diff=602436421&oldid=602436180

  • On 16:25 the editor vandalized again and this time i reverted:

https://en.wikipedia.org/w/index.
php?title=Diary_of_a_Wimpy_Kid&diff=602436675&oldid=602436662

Huggle did not place a warning though - instead this was displayed in the
event log:
"wo 2. apr 16:25:12 2014 ERROR: Unable to deliver the message to
65.18.38.128: Unknown error: articleexists"

I haven't seen this one before so i'm not sure if this is related to the
latest changes or if this is something separate altogether.

https://github.com/huggle/huggle3-qt-lx/commit/1225992b272b191afaa5ece3c1d648507c7ef9da

One exception i ran into is this IP editor. I reverted two edits made by this user:

However, Huggle immediately warned me the user already had a level 4 warning and asked if i wished to report them. Looking at the user talk page there are plenty of warnings, but none recent:

Another example of the above for completeness sake. I tried to revert this edit:

When doing so Huggle reported that the user already had a level 4 warning, and asked if i wished to report it. However, as before there is no recent level 4 warning at the talk page:

Did any of these happen using latest huggle? Please post huggle version together with new reports

(In reply to Excirial from comment #24)

Another example of the above for completeness sake. I tried to revert this
edit:

https://en.wikipedia.org/w/index.
php?title=Aron_Ralston&diff=602647300&oldid=602367072

When doing so Huggle reported that the user already had a level 4 warning,
and asked if i wished to report it. However, as before there is no recent
level 4 warning at the talk page:

I created unit test https://github.com/huggle/huggle3-qt-lx/commit/e82b6eb6d6ce8f62bb59dca28621e4b8c6537628 for this talk page, but it PASSED just fine, so this issue is likely already fixed in latest huggle

On build 1110:

I don't know how long in between the dates it has to be to "reset" the warning level, but...

It added a lvl 1, where a lvl 1 was added only 5 days ago.

(In reply to gladjonatan from comment #29)

On build 1110:

I don't know how long in between the dates it has to be to "reset" the
warning level, but...

https://en.wikipedia.org/w/index.php?title=User_talk:216.48.138.
103&oldid=603342430

It added a lvl 1, where a lvl 1 was added only 5 days ago.

yes it's 5 days on english wikipedia, by default it's 30 though on other wikis

(In reply to gladjonatan from comment #28)

On build 1110:

https://en.wikipedia.org/w/index.php?title=User_talk:
Nathangrice&oldid=603335334

It added a lvl 2, where a lvl 4 should have been added.

biog3 was not in definitions in config it's now fixed

It seems to be fixed now. (petan > to ping in the IRC-channel)

I added a unit test for this as well as for other reported pages, this seems to be fixed now

oops on Comment 35 is regarding Build 1282.

I see the same warning level issue in RC1:

I will create a unit test for these but I am pretty sure this was already fixed long time ago, it's just that you are not running latest version

(In reply to Peter Bena from comment #40)

I will create a unit test for these but I am pretty sure this was already
fixed long time ago, it's just that you are not running latest version

Yes, this was fixed a while ago - I am dead certain that this worked just fine in versions prior to RC1. However, ever since I installed RC1 (Build 1282) this issue seems to have resurfaced.

I made a test for every single of these talk pages and all passed, so please update your huggle, it works.

I can't reproduce it, both unit tests are just fine.

Can you find some errors in logs? Something like "unable to retrieve talk page of bla bla" or something like that.

Created attachment 15367
System logs for Comment 47

attachment file_59982.txt ignored as obsolete

Created attachment 15370
System logs for Comment 48

attachment file_59982.txt ignored as obsolete

I downgraded to bild 1242, with opes that it would wrok, but instead I found the same problem there.

https://en.wikipedia.org/w/index.php?title=User_talk:164.58.75.210&diff=608574839&oldid=608570883 (A lvl 1, where the users should have been reported)

typos in Cmment 50 : hopes*, work*

typos in Comment 50 : hopes*, work*

I didn't get a single wrong edit on rc3 so far on windows, IMHO the problem is resolved, I will check your configs though

I have wiped your configuration file on english wikipedia, please try now. In case you will have any wrong edit, paste also the logs from huggle. You can also start huggle in debug mode (level 2 or more -vv) the debug logs will be helpful here

I have tried again, and the problem is still there.

https://en.wikipedia.org/w/index.php?title=User_talk%3A62.171.192.105&diff=608823440&oldid=608823099

(added a lvl 1, should have been a lvl 3)

I will add system logs from debug mode in Comment 57.

Created attachment 15403
System logs for Comment 56

attachment file_59982.txt ignored as obsolete

Can you pick any other queue in menu?

I can't use IRC as provider, but I can change from default-queue to filtred edits, if that will help...

Created attachment 15404
Unit

Attached:

Created attachment 15405
test

Attached:

download test and unit files and place them to folder where huggle is installed, then open command line and navigate to this folder, execute tst_testmain and send me the output

Here you go (Disclaimer: This test was not executed from my regular system)

  • Start testing of HuggleTest *****

Config: Using QtTest library 5.1.1, Qt 5.1.1
PASS : HuggleTest::initTestCase()
FAIL! : HuggleTest::testCaseTalkPageParser0001() 'user->WarningLevel == level' returned FALSE. (level parsed was 0 should be 3!!)
..\test\tst_testmain.cpp(85) : failure location
FAIL! : HuggleTest::testCaseTalkPageParser0002() 'user->WarningLevel == level' returned FALSE. (level parsed was 0 should be 3!!)
..\test\tst_testmain.cpp(85) : failure location
PASS : HuggleTest::testCaseTalkPageParser0003()
PASS : HuggleTest::testCaseTalkPageParser0004()
FAIL! : HuggleTest::testCaseTalkPageParser0005() 'user->WarningLevel == level' returned FALSE. (level parsed was 0 should be 4!!)
..\test\tst_testmain.cpp(85) : failure location
FAIL! : HuggleTest::testCaseTalkPageParser0006() 'user->WarningLevel == level' returned FALSE. (level parsed was 0 should be 1!!)
..\test\tst_testmain.cpp(85) : failure location
FAIL! : HuggleTest::testCaseTalkPageParser0007() 'user->WarningLevel == level' returned FALSE. (level parsed was 0 should be 1!!)
..\test\tst_testmain.cpp(85) : failure location
FAIL! : HuggleTest::testCaseTalkPageParser0008() 'user->WarningLevel == level' returned FALSE. (level parsed was 0 should be 1!!)
..\test\tst_testmain.cpp(85) : failure location
FAIL! : HuggleTest::testCaseTalkPageParser0009() 'user->WarningLevel == level' returned FALSE. (level parsed was 0 should be 1!!)
..\test\tst_testmain.cpp(85) : failure location
FAIL! : HuggleTest::testCaseTalkPageParser0010() 'user->WarningLevel == level' returned FALSE. (level parsed was 0 should be 3!!)
..\test\tst_testmain.cpp(85) : failure location
FAIL! : HuggleTest::testCaseTalkPageParser0011() 'user->WarningLevel == level' returned FALSE. (level parsed was 0 should be 1!!)
..\test\tst_testmain.cpp(85) : failure location
FAIL! : HuggleTest::testCaseTalkPageParser0012() 'user->WarningLevel == level' returned FALSE. (level parsed was 0 should be 1!!)
..\test\tst_testmain.cpp(85) : failure location
FAIL! : HuggleTest::testCaseTalkPageParser0013() 'user->WarningLevel == level' returned FALSE. (level parsed was 0 should be 2!!)
..\test\tst_testmain.cpp(85) : failure location
FAIL! : HuggleTest::testCaseTalkPageParser0014() 'user->WarningLevel == level' returned FALSE. (level parsed was 0 should be 4!!)
..\test\tst_testmain.cpp(85) : failure location
PASS : HuggleTest::testCaseWikiUserCheckIP()
PASS : HuggleTest::testCaseTerminalParser()
PASS : HuggleTest::testCaseConfigurationParse_QL()
PASS : HuggleTest::testCaseScores()
PASS : HuggleTest::cleanupTestCase()
Totals: 8 passed, 12 failed, 0 skipped

  • Finished testing of HuggleTest *****
  • Start testing of HuggleTest *****

Config: Using QtTest library 5.1.1, Qt 5.1.1
PASS : HuggleTest::initTestCase()
FAIL! : HuggleTest::testCaseTalkPageParser0001() 'user->WarningLevel == level'
returned FALSE. (level parsed was 0 should be 3!!)
..\test\tst_testmain.cpp(85) : failure location
FAIL! : HuggleTest::testCaseTalkPageParser0002() 'user->WarningLevel == level'
returned FALSE. (level parsed was 0 should be 3!!)
..\test\tst_testmain.cpp(85) : failure location
PASS : HuggleTest::testCaseTalkPageParser0003()
PASS : HuggleTest::testCaseTalkPageParser0004()
FAIL! : HuggleTest::testCaseTalkPageParser0005() 'user->WarningLevel == level'
returned FALSE. (level parsed was 0 should be 4!!)
..\test\tst_testmain.cpp(85) : failure location
FAIL! : HuggleTest::testCaseTalkPageParser0006() 'user->WarningLevel == level'
returned FALSE. (level parsed was 0 should be 1!!)
..\test\tst_testmain.cpp(85) : failure location
FAIL! : HuggleTest::testCaseTalkPageParser0007() 'user->WarningLevel == level'
returned FALSE. (level parsed was 0 should be 1!!)
..\test\tst_testmain.cpp(85) : failure location
FAIL! : HuggleTest::testCaseTalkPageParser0008() 'user->WarningLevel == level'
returned FALSE. (level parsed was 0 should be 1!!)
..\test\tst_testmain.cpp(85) : failure location
FAIL! : HuggleTest::testCaseTalkPageParser0009() 'user->WarningLevel == level'
returned FALSE. (level parsed was 0 should be 1!!)
..\test\tst_testmain.cpp(85) : failure location
FAIL! : HuggleTest::testCaseTalkPageParser0010() 'user->WarningLevel == level'
returned FALSE. (level parsed was 0 should be 3!!)
..\test\tst_testmain.cpp(85) : failure location
FAIL! : HuggleTest::testCaseTalkPageParser0011() 'user->WarningLevel == level'
returned FALSE. (level parsed was 0 should be 1!!)
..\test\tst_testmain.cpp(85) : failure location
FAIL! : HuggleTest::testCaseTalkPageParser0012() 'user->WarningLevel == level'
returned FALSE. (level parsed was 0 should be 1!!)
..\test\tst_testmain.cpp(85) : failure location
FAIL! : HuggleTest::testCaseTalkPageParser0013() 'user->WarningLevel == level'
returned FALSE. (level parsed was 0 should be 2!!)
..\test\tst_testmain.cpp(85) : failure location
FAIL! : HuggleTest::testCaseTalkPageParser0014() 'user->WarningLevel == level'
returned FALSE. (level parsed was 0 should be 4!!)
..\test\tst_testmain.cpp(85) : failure location
PASS : HuggleTest::testCaseWikiUserCheckIP()
PASS : HuggleTest::testCaseTerminalParser()
PASS : HuggleTest::testCaseConfigurationParse_QL()
PASS : HuggleTest::testCaseScores()
PASS : HuggleTest::cleanupTestCase()
Totals: 8 passed, 12 failed, 0 skipped

  • Finished testing of HuggleTest *****

Just a thought - would it be an idea to use a regular expression instead of the current string based matching in order to detect the warnings on a user page? Something like this:

#include <QRegularExpression>

QRegularExpression re("<!-- Template:uw-(?<Template>.+?) -->.+\\d{2}:\\d{2}, (?<Date>\\d{1,2} \\w+ 20\\d{2}) \\x28UTC\\x29");

QRegularExpressionMatch match = re.match(page);

if (match.hasMatch()) {
    QString Template = match.captured(1);
    QString Date = match.captured(2);
}

I just tested the concept inside Huggle itself and it seems to work. The result for "Template" contains "Vandalism2" and the result for the Date contains "22 June 2009".

My guess is it could do something like this:

  • Loop trough all the matches for a user page.
  • Check if the date is in range. If not, ignore.
  • If the case is in range: Log the associated warning level if it is higher than the previous highest warning level.

The regex is a first draft so i don't expect it to work in all circumstances just yet, but that could be improved.

I don't like for these reasons:

  • regexes are horribly slow, this would be massive execution in loop over long text in short time
  • this solution isn't very flexible, it can he hardly customized enough to run on any possible wiki installation
  • it could fail just the same as it does now

but yes, we could use some other lightweight regexes just to make it easier for parser

Created attachment 15409
ut 2

Attached:

Created attachment 15410
Result of ut 2 (test 2)

attachment file_59982.txt ignored as obsolete

Created attachment 15411
ut 3

new one

Attached:

Created attachment 15412
Result of ut 3 (test 3)

attachment file_59982.txt ignored as obsolete

Fyi - i just ran test 2 and 3. The results i receive seems to be identical to the results posted above. Once exception: The logging in test 3 reports a localized date and time for me as well.

Created attachment 15414
ut 4

Attached:

Created attachment 15415
Result of ut 4 (test 4)

attachment file_59982.txt ignored as obsolete

Created attachment 15416
ut 5

Attached:

Created attachment 15417
Result of ut 5 (test 5)

attachment file_59982.txt ignored as obsolete

Using Huggle 3.0.2, Build 1616:

<!-- Template:uw-huggleafd1 --> isn't registered in Huggle config at https://en.wikipedia.org/wiki/Wikipedia:Huggle/Config#Warning