dnessett wrote:

Output of make safe

The output from the make safe run

Attached:

I'm guessing most of this is going to be postgres related issues

http://integration.mediawiki.org/ci/job/MediaWiki-phpunit/ is fine with SQLite, and MySQL locally for me is also fine

(In reply to comment #2)

I'm guessing most of this is going to be postgres related issues

Worth fixing, at least :) The skipped tests all look fine (skipping Sqlite tests due to missing support, etc). I'm kind of concerned about the "Aborted execution after 1 second" errors failures though.

dnessett wrote:

Acoording to http://www.gossamer-threads.com/lists/wiki/wikitech/267086, there are now only 2 tests failing in latest trunk for postgres based runs. However, I just ran make safe locally and get 1 error, 25 failures and 12 incomplete tests. I am attaching the output in a separate comment.

Same environment, except the MW revision is 108734

dnessett wrote:

Output of 2nd make safe run

Attached:

dnessett wrote:

(In reply to comment #4)

Acoording to http://www.gossamer-threads.com/lists/wiki/wikitech/267086, there
are now only 2 tests failing in latest trunk for postgres based runs. However,
I just ran make safe locally and get 1 error, 25 failures and 12 incomplete
tests. I am attaching the output in a separate comment.

Same environment, except the MW revision is 108734

Ignore this. I forgot to run update. After doing so, I only get 2 failures.

I am considering this bug fixed. Jenkins is now running tests against a postgreSQL backend so we get IRC notification like :

Project MediaWiki-postgres-phpunit build #23: STILL FAILING in 3 min 43 sec: http://integration.mediawiki.org/ci/job/MediaWiki-postgres-phpunit/23/

That is enough to remember everyone to have a look at it :-b

Thanks for the bug report. You made me add the Jenkins job!

dnessett wrote:

I thought I would look at the one remaining failure in the make safe phpunit tests for postgres. I did a svn up and then ran update. When I ran make safe, 24 failures showed up. I am attaching the make safe output in a separate comment. The revision is: 108821

dnessett wrote:

make safe failures for MW r108821

Attached:

dnessett wrote:

I found the problem causing ApiQueryTest::testTitlesGetNormalized to fail, but I don't know exactly how to fix it.

ApiQuery:execute() calls:
$this->mPageSet->execute() (line 254) calls:
$this->initFromTitles( $params['titles'] ) (line 303) calls:
$linkBatch = $this->processTitlesArray( $titles ) (line 414) calls:
$unconvertedTitle = $titleObj->getPrefixedText() (line 678) calls:
str_replace( '_', ' ', $s ) (line 1127)

In the test case, $wgMetaNamespace is left at its default set in DefaultSettings.php (false) and so is set to the value of $wgSitename, which in the test case is "MW_latest". The underscore is being replaced by a blank character, so the test fails when comparing the 'to' array entry expected and that returned.

One way to fix this is to change the test by replacing all spaces in the returned 'to' entry namespace prefix with underscores, but I suspect this just covers over a bug. So, somewhere in the code, there is an assumption made that $wgMetaNamespace is a string without spaces/underscores. This assumption should be removed.

Just for the record, I checked and underscores are legal in namespace prefixes (http://www.mediawiki.org/wiki/Manual:Title.php).

dnessett wrote:

Things are getting worse. The make safe tests now throw an MW exception (attached Make Safe r109424.txt). Surprisingly, this actually gives a hint to what may be going wrong.

I instrumented TestSuit.php (attached TestSuite.php.modified) to print out $wgLanguageCode on each test run. When I run make safe, its value changes to "zh" on the 429th ParserTest (attached Full Safe Tests Run.txt) and stays that way for the rest of the run. I couldn't figure out why until I looked at the incomplete tests. The 20th incomplete test changes the language to "zh" and then fails after 1 second. So, my guess is there is no proper unwinding of incomplete tests and they leave globals in a non-initialized state.

I then looked at the first incomplete test. It is the first test run (ArticleTablesTest::testbug14404). It times out after 1 second. So, there is the distinct possibility that globals are being corrupted by the first test and this problem propagates through the whole run.

This is semi-confirmed when I ran only the parser tests, i.e., ./phpunit.php --group Parser (attached Parser Tests Only Run.txt). All tests pass and $wgLanguageCode remains at "en" through out the run.

I don't how to address the problem of tests aborting and corrupting globals, but I seem to recall phpunit has an option that saves and restores there values. Perhaps that option isn't being used because it would lengthen the time it takes to run the tests.

dnessett wrote:

TestSuite.php with embedded debug statements

Attached:

dnessett wrote:

Make Safe Output from r109424

Attached:

dnessett wrote:

Make safe run with debug information created by TestSuite.php.modified

Attached:

dnessett wrote:

Parser Tests only with debug information from TestSuite.php.modified

Attached:

(In reply to comment #11)

I instrumented TestSuit.php (attached TestSuite.php.modified) to print out
$wgLanguageCode on each test run. When I run make safe, its value changes to
"zh" on the 429th ParserTest (attached Full Safe Tests Run.txt) and stays that
way for the rest of the run. I couldn't figure out why until I looked at the
incomplete tests. The 20th incomplete test changes the language to "zh"...

Tests changing the language should extend MediaWikiLangTestCase (which saves & restores it), not MediaWikiTestCase.

and then fails after 1 second. So, my guess is there is no proper unwinding of
incomplete tests and they leave globals in a non-initialized state.

What I wonder is, what makes your tests to abort after 1 second?
I have never seen that myself.

dnessett wrote:

(In reply to comment #16)

(In reply to comment #11)

I instrumented TestSuit.php (attached TestSuite.php.modified) to print out
$wgLanguageCode on each test run. When I run make safe, its value changes to
"zh" on the 429th ParserTest (attached Full Safe Tests Run.txt) and stays that
way for the rest of the run. I couldn't figure out why until I looked at the
incomplete tests. The 20th incomplete test changes the language to "zh"...

Tests changing the language should extend MediaWikiLangTestCase (which saves &
restores it), not MediaWikiTestCase.

The test that changes the language is the ParserTest, which runs a whole bunch of sub-tests of the parser functionality. Most of these do not change the language. I still don't quite understand how the parser tests are run within the PHPUnit framework, so I don't know if one PHPUnit test runs all parser tests or each test runs under a separate instance of MediaWikiTestCase. In the latter case, it makes sense to run those tests that change the language out of MediaWikiLangTestCase. In the former case, it may or may not make sense

and then fails after 1 second. So, my guess is there is no proper unwinding of
incomplete tests and they leave globals in a non-initialized state.

What I wonder is, what makes your tests to abort after 1 second?
I have never seen that myself.

I was hoping someone would indicate what might cause such timeouts and suggest a way to fix the problem. I have no idea why they are aborting after 1 second.

The timeout messages comes from PHPUnit. It is most probably caused because PHP_Invoker is installed ( https://github.com/sebastianbergmann/php-invoker ) which make PHPUnit think our tests are small and should run in less than a second.

By looking at the code of PHPUnit, this behavior only happens while in strict mode (by looking at PHPUnit/Framework/TestResult.php. Can you try editing MediaWiki file "tests/phpunit/suite.xml", in the very first few lines you should have something looking like:
strict="true"

replace it by:

strict="false"

That should disable strict node and the whole timeout system. If that solve the issue, we would have to find a way to disable that system entirely.

TIP: to show tests names, you can run the suite using --tap :

cd tests/phpunit
php phpunit.php --tap
TAP version 13

ok 1 - ArticleTablesTest::testbug14404
ok 2 - ArticleTest::testImplementsGetMagic
ok 3 - ArticleTest::testImplementsSetMagic
ok 4 - ArticleTest::testImplementsCallMagic
ok 5 - ArticleTest::testGetOrSetOnNewProperty
ok 6 - ArticleTest::testStaticFunctions
ok 7 - ArticleTest::testWikiPageFactory
ok 8 - BlockTest::testInitializerFunctionsReturnCorrectBlock
ok 9 - BlockTest::testBug26425BlockTimestampDefaultsToTime
ok 10 - BlockTest::testBug29116LoadWithEmptyIp with data set #0 (NULL)
ok 11 - BlockTest::testBug29116LoadWithEmptyIp with data set #1 ('')
ok 12 - BlockTest::testBug29116LoadWithEmptyIp with data set #2 (false)
ok 13 - BlockTest::testBug29116NewFromTargetWithEmptyIp with data set #0 (NULL)
ok 14 - BlockTest::testBug29116NewFromTargetWithEmptyIp with data set #1 ('')

dnessett wrote:

Run safe with strict equals false in suite.xml

Attached:

dnessett wrote:

(In reply to comment #18)

The timeout messages comes from PHPUnit. It is most probably caused because
PHP_Invoker is installed ( https://github.com/sebastianbergmann/php-invoker )
which make PHPUnit think our tests are small and should run in less than a
second.

By looking at the code of PHPUnit, this behavior only happens while in strict
mode (by looking at PHPUnit/Framework/TestResult.php. Can you try editing
MediaWiki file "tests/phpunit/suite.xml", in the very first few lines you
should have something looking like:
strict="true"

replace it by:

strict="false"

That should disable strict node and the whole timeout system. If that solve the
issue, we would have to find a way to disable that system entirely.

That did the trick. There is now only 1 failure. That is easily fixed if the correct tactic is to normalize the expected output on the assert (see Comment 10). If so, someone speak up and I will make the necessary change.

I have attached the output of make safe with strict="false" in suite.mxl.

Opened bug 34037 to make our suite works when PHP_Invoker is installed.

Array (

'from' => 'Project:articleA'

• 'to' => 'MW_latest:ArticleA'

+ 'to' => 'MW latest:ArticleA'
)

This test use $wgMetaNamespace which contains space/underscore on your project. The ApiQueryTest::testTitlesGetNormalized() test assumed it to be normalized and thus failed whenever that is no the case.

r110298 fix that issue (which has nothing to do with postgres anyway).

Thanks Dan for all the bug reporting you did so far on this bug. Since this last issue is solved, I am marking it as fixed. You probably want to open a new bug report if you have anymore issues.

Migrating from the old tracking task to a tag for PostgreSQL-related tasks.