Page MenuHomePhabricator

Review LaTeXML branch of Extension:Math for deployment
Closed, ResolvedPublic

Description

"<physikerwelt> Hi, I have created a new version of the Math extension that allows displaying mathml using the latexml converter instead of rendered images. Furthermore I have created a test wiki that contains all pages of the English Wikipedia Project that contain math. Now I’d be interested to test this feature inside the Wikimedia framework. Do you have a proposal how to do that."

See https://gerrit.wikimedia.org/r/gitweb?p=mediawiki%2Fextensions%2FMath.git;a=shortlog;h=refs%2Fheads%2FLaTeXML


Version: master
Severity: normal

Details

Reference
bz43222

Event Timeline

bzimport raised the priority of this task from to High.Nov 22 2014, 12:53 AM
bzimport added a project: Math.
bzimport set Reference to bz43222.

The author wrote a paper about this change, "Making Math Searchable in Wikipedia": http://wiki.physikerwelt.de/images/text_math_search.pdf

Created attachment 11739
Math parser tests on patch set 21

This attachment is the result of running:

php tests/parserTests.php --file=extensions/Math/mathParserTests.txt > mathParserPatchSet21.txt 2>&1

on patch set 21 of https://gerrit.wikimedia.org/r/#/c/30177/ in my Vagrant install.

Attached:

All tests pass! With the installation of 'texlive-lang-greek' and 'texlive-fonts-recommended', I was able to get 12/13 tests to pass. The remaining test failure was due to the srset attribute getting added to img tags, so I updated the expected output.

I documented the dependencies in README.

Merged. Thanks very much physikerwelt for your patience and fixes and thanks Matt for the reviews. This is now in the deployment queue.

What about the nullLockManager I asked about?

I'm not totally comfortable with the change from nullLockManager to fsLockManager (https://gerrit.wikimedia.org/r/#/c/30177/20..24/MathTexvc.php).

In theory it should be okay on the WMF cluster, since it looks like it only applies if wgMathFileBackend is not set.

In mediawiki-config, there is:

$wgMathFileBackend = $wmgMathFileBackend;

in CommonSettings.php

and:

'wmgMathFileBackend' => array(
'default' => 'global-multiwrite'
),

in LocalSetting.php.

However, it could affect other installations.

Created attachment 11755
Math parser tests on patch set 20

This is for background on the lock manager discussion.

Attached:

There's a thread on this on wikitech-l under "NullLockManager and the math extension"

(In reply to comment #9)

There's a thread on this on wikitech-l under "NullLockManager and the math
extension"

http://lists.wikimedia.org/pipermail/wikitech-l/2013-February/066265.html

I've recommended that physikerwelt squash some of the commits on the branch, so there won't be quite so many to review.

physik wrote:

Thanks a lot Matthew. I want to do some meaningful unit tests for the module. I would be quite happy if you could provide some input for that task. Is there a general stategy at the WMF to structured tests. In the end we should come up with a mutually orthogonal set of tests, some integration tests in addition and furthermore tests that check for the expected bugs.
Thus I'd be happy to collect a list of required test at
http://www.mediawiki.org/wiki/Extension:Math/Roadmap
After that the time is ready for merge.

Generally we have parser tests, which you've already seen, PHPUnit tests (can be used for unit or integration tests on server), QUnit (JavaScript unit tests), and Cucumber for integration tests (testing the overall user experience through the web browser).

Probably, PHPUnit would be most important for your stuff, but if you want to add some QUnit or Cucumber, that would be a bonus, particularly for the MathJax stuff.

See https://www.mediawiki.org/wiki/PHPUnit, https://www.mediawiki.org/wiki/QUnit and https://www.mediawiki.org/wiki/Cucumber

I've done a lot of QUnit here, and some PHPUnit, so I can definitely help with those. I'm working on a Cucumber test right now, but I haven't committed anything yet.

physik wrote:

(In reply to comment #14)
Thanks for these hints. However I was aware of the technical aspects.
Maybe to be more precise. Is there a prototpe extension that comes up with good unit tests. Especially the database stuff is something that is not trival to test.

Generally we have parser tests, which you've already seen, PHPUnit tests (can
be used for unit or integration tests on server), QUnit (JavaScript unit
tests), and Cucumber for integration tests (testing the overall user
experience
through the web browser).

Probably, PHPUnit would be most important for your stuff, but if you want to
add some QUnit or Cucumber, that would be a bonus, particularly for the
MathJax
stuff.

See https://www.mediawiki.org/wiki/PHPUnit,
https://www.mediawiki.org/wiki/QUnit and
https://www.mediawiki.org/wiki/Cucumber

I've done a lot of QUnit here, and some PHPUnit, so I can definitely help
with
those. I'm working on a Cucumber test right now, but I haven't committed
anything yet.

(In reply to comment #15)

(In reply to comment #14)
Thanks for these hints. However I was aware of the technical aspects.
Maybe to be more precise. Is there a prototpe extension that comes up with
good
unit tests. Especially the database stuff is something that is not trival to
test.

There probably are extensions with good database tests, but I can't pinpoint one offhand. However, if you go into the test directory of MediaWiki core, you can do:

git grep db

That will give you an indication of how databases are used for core tests. Extension tests generally subclass the same class, MediaWikiTestCase, which provides some of the database functionality you'll need.

physik wrote:

Thanks.
I tried to figure out how to set up the database tables for testing. As far as I did understand there must be a way to clone the table structure with a prefix.
However if I tied to do this, I get a warning the table "_prefix_math" does already exist.
See
https://gerrit.wikimedia.org/r/#/c/49477/1/tests/MathRendererTest.php

physik wrote:

Unfortunately there are some bugs in the master version. Is there a way to get them fixed faster than the via the normal review process.
See https://gerrit.wikimedia.org/r/#/q/status:open+project:mediawiki/extensions/Math,n,z

physik wrote:

It would be grate to be able to test LaTeXML in the context of the WMF.
see
https://wikitech.wikimedia.org/wiki/New_Project_Request/Math

I think it is very important to do tests there before committing something to the main branch.

physik wrote:

This is now adressed by

physik wrote:

mylyn/context/zip

Attached:

physik wrote:

(In reply to comment #21)
Sorry wrong link:
This is now adressed by
https://gerrit.wikimedia.org/r/#/c/61987/

(In reply to comment #23 by physikerwelt)

This is now adressed by
https://gerrit.wikimedia.org/r/#/c/61987/

Patch merged on 2013-05-26. What's the next steps?

physik wrote:

Since mathml is not supported by manny browseres
This change is also important to reach a wide audience
https://gerrit.wikimedia.org/r/#/c/69298/

There is still an open LaTeXML branch (https://git.wikimedia.org/log/mediawiki%2Fextensions%2FMath.git/refs%2Fheads%2FLaTeXML), with recent commits

However, I'm not sure if this bug is still helpful. Commits to master are being reviewed on a case-by-case basis.

physik wrote:

Since LaTeXML is merged and there are some users that use it actively this bug can be closed. Unfortunately I can not do that;-)

To avoid confusion I deleted the LaTeXML branch and renamed it to dev.

Just as information: The next step towards MathML are the deployment of the LaTeXML package. operations/debs/latexml

Marked fixed. :)

Do you have editbugs yet? If not, you can ask on IRC for someone to give it to you.