Page MenuHomePhabricator

ApiUploadTest is flaky
Closed, ResolvedPublicPRODUCTION ERROR

Description

Author: neilk

Description:
multiple runs of ApiUploadTest have different results. Sometimes they succeed, sometimes fail.

Oddly, printing debug information seems to make the errors more elusive.

Arguably ApiUploadTest is done wrong; it should open up a real HTTP connection rather than elaborately simulating an API request.


Version: unspecified
Severity: normal

Event Timeline

bzimport raised the priority of this task from to Low.Nov 21 2014, 11:24 PM
bzimport set Reference to bz26169.
bzimport added a subscriber: Unknown Object (MLST).

Is this actually in wikimedia-deployment, or on trunk?

I wouldn't trust any of the unit tests to run on deployment, lots of unmerged changes there.

the purpose of ApiUploadTest is not to test the actual upload so much as the code path handling the API upload. Requiring the test to use an actual web server would mean the tests couldn't be used by as many people.

Antoine is going to remove this test from the default group of tests. I'm unassigning this from him for now. Whoever takes this on probably needs to set up mock objects and other mechanisms to avoid this unreliability in testing.

Disabled with rSVN110790

ApiUploadTest is still in @group Broken in current master.

Krinkle renamed this task from ApiUploadTest is flaky -- inconsistent results to ApiUploadTest is flaky.Jan 12 2017, 10:00 PM

Change 379817 had a related patch set uploaded (by Aleksey Bekh-Ivanov (WMDE); owner: Aleksey Bekh-Ivanov (WMDE)):
[mediawiki/core@master] Make ApiUploadTest up to date so it passes

https://gerrit.wikimedia.org/r/379817

Change 379817 merged by jenkins-bot:
[mediawiki/core@master] Make ApiUploadTest up to date so it passes

https://gerrit.wikimedia.org/r/379817

Suggestions:

  • Get rid of RandomImageGenerator. Tests should be reproducible. Just use files from tests/phpunit/data/media instead.
  • In testUploadChunks(), don't download a multi-megabyte file from commons and then upload to the wiki, that's slow. Just override $wgMinUploadChunkSize and set the chunk size to 200 or something, then a tiny file from tests/phpunit/data/media can be uploaded in chunks.
  • Don't litter the wiki's actual upload directory with test files. Make a RepoGroup that points to a temporary directory.
  • Put teardown in tearDown() not in test methods after assertions are done. Destroy the whole temporary directory.
daniel subscribed.

Tagging CPT for fixing the test as suggested by time above. The fact that this test was disabled caused a bug to slip by which resulted in T231488: UploadFromChunks.php: Call to undefined method MediaWiki\FileBackend\FSFile\TempFSFileFactory::getTempFSFile().

mmodell changed the subtype of this task from "Task" to "Production Error".Aug 28 2019, 11:12 PM

Change 533283 had a related patch set uploaded (by Ppchelko; owner: Ppchelko):
[mediawiki/core@master] Rework ApiUploadTest.

https://gerrit.wikimedia.org/r/533283

Change 533283 merged by jenkins-bot:
[mediawiki/core@master] Rework ApiUploadTest

https://gerrit.wikimedia.org/r/533283