Page MenuHomePhabricator

DifferenceEngine grinds on pathological case
Closed, ResolvedPublic

Description

Author: wikiadmin

Description:
Generating diffs can take 20 to 30 seconds for some files not exceptionally long.

On hosted shared servers with max_execution_time set to 10, this will cause timeouts and HTTP 500's.

Related second bug that even with $wgFeedDiffCutoff = 0; diffs are generated for RSS feeds.
This causes the DifferenceEngine problem to kill the RSS feeds.


Version: 1.12.x
Severity: normal
OS: Linux
URL: http://wiki.xyrael.net/wiki/User:Renate/bug

Details

Reference
bz14867

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 21 2014, 10:11 PM
bzimport set Reference to bz14867.
bzimport added a subscriber: Unknown Object (MLST).

ayg wrote:

It's strongly recommended that decent-sized sites use the wikidiff2 PHP module instead of DifferenceEngine.php. Diffing is very CPU-intensive and doing it in PHP will take a long time. max_execution_time = 10 is very restrictive and will cause failure when parsing, diffing, or doing anything else requiring much CPU time. It's probably taking even longer than it otherwise would because the server is overloaded and/or has slow CPUs.

I doubt that there's anything here that's fixable on our side, unless you want this bug to be a generic "find a more efficient diff algorithm". I don't see any evidence presented that this case is particularly more pathological than diffs of any other 10 KB pages with a whole bunch of lines changed. I'm going to have to mark INVALID; either get wikidiff2 installed or get a faster/less restrictive host. If you want to reopen and post some patches, or make this a wishlist "make diffs faster" bug, go ahead.

(In reply to comment #1)

Nice and snappy for me.

http://test.wikipedia.org/w/index.php?title=User:OverlordQ/Sandbox&diff=61653&oldid=61652

Wikipedia doesn't use DifferenceEngine.php, so it's not really relevant. I'll point out that wikidiff2 does use the same algorithm as DifferenceEngine.php, though, so I don't think this case is failing because it's pathological. I think it's failing despite the fact that it's typical.

Keep this open -- the diff engine *ought* to behave sensibly for any input, either working quickly or degrading or failing gracefully.

Good behaviour on pathological test cases was the sole reason for writing wikidiff2. It's not a simple problem to solve in PHP.

  • This bug has been marked as a duplicate of bug 7601 ***