While it would be nice to make them translatable, I'm going to take this out of the blockers for initial experimental enabling (bug 31406).

(Started a thread on mathjax-users asking if people would be interested in menu localization)

https://groups.google.com/d/topic/mathjax-users/YNKXMCq9Bi0/discussion

Updating priority and importance. We shouldn't push un-i18n-ed components to non-English language wikis -- or even consider adding it when it has no i18n.

(In reply to comment #4)

Updating priority and importance. We shouldn't push un-i18n-ed components to
non-English language wikis -- or even consider adding it when it has no i18n.

Maybe it can be disabled before it's i18n-ed...

It's now enabled site-wide :/

Updated priority. Appears to be a major oversight that un-i18n-able software was deployed side wide. Also added a few CCs.

I'd say this blocks a nice full roll-out to default rendering (still a ways off), but it obviously doesn't block experimental opt-in usage as we have already turned it on.

Yep, agree w/ Brion. No reason to not let people test it as a pref. (The menu structure is also fairly opaque in normal usage; I hadn't discovered it until seeing this bug.)

Dropping priority to "high". "Highest" priority should be reserved for bugs that someone needs to drop what they're doing and address immediately (possibly on the weekend if necessary).

Any progress?

Any progress? Looks like the upstream thread is very dead.

Maybe MathJax can use https://github.com/wikimedia/jquery.i18n

Just to note that Aditya (added to CC) started to try doing this at the Bangalore DevCamp using the jquery.i18n that Wikimedia's Language Engineering team developed. He already has some results.

Hi everyone.

Just to keep you updated from the MathJax side. The next MathJax release (planned for the winter) will include proper UI localization.

(In reply to comment #14)

Just to keep you updated from the MathJax side. The next MathJax release
(planned for the winter) will include proper UI localization.

Great!
Can you link the instructions or the process you're going to follow?
I hope it doesn't involve emailing po files or similar pains.

First we have to work on our code a bit. Once localization is possible, we'd love input on how to crowd source. But no, definitely not emailing po files.

(In reply to comment #16)

First we have to work on our code a bit. Once localization is possible, we'd
love input on how to crowd source. But no, definitely not emailing po files.

Good, so it's much better for you to plan in advance rather than to spend tons of time fixing stuff later.
In particular, if you follow i18n good practices and use a format supported by [[mw:Extension:Translate]] (there are several) then it will be easy to add it to translatewiki.net.

(In reply to comment #14)

Hi everyone.

Just to keep you updated from the MathJax side. The next MathJax release
(planned for the winter) will include proper UI localization.

(In reply to comment #13)

Just to note that Aditya (added to CC) started to try doing this at the
Bangalore DevCamp using the jquery.i18n that Wikimedia's Language Engineering
team developed. He already has some results.

Are these two messages at almost the same time referring to the same efforts, or different efforts at i18n-ing the code?

Different efforts.

I'm speaking for the MathJax team. I only heard about attempts to localize on twitter and here so I thought I should comment and clarify our own plans.

As you (Siebrand) pointed out, the thread on the MathJax dev-list has been cold since we've been focusing on a bug fix release over the last few months.

(In reply to comment #19)

Different efforts.

I'm speaking for the MathJax team. I only heard about attempts to localize on
twitter and here so I thought I should comment and clarify our own plans.

As you (Siebrand) pointed out, the thread on the MathJax dev-list has been cold
since we've been focusing on a bug fix release over the last few months.

AFAIK the DevCamp Bangalore participants are planning on creating a pull request with an i18n solution for MathJax today (India time). You're of course free to discard that in favour of something still to be developed.

We'll definitely take a look but I can't make any promises.

So, here's the update:

Apparently, it's already possible to use menu-i18n.js as it is to localize the menu. We translated the menu to a few languages and it works. To achieve this we just need to change our code, but we would be happy to send you the translations that we made to a few languages already. I'm just unsure to which folder in GitHub to add them.

However, translating the menu alone is not enough. Messages like "Typesetting math: 45%" cannot be properly localized with the current code for at least two reasons:

1. Most importantly, they are concatenated on the fly. This will not work in languages with a different word order.

2. The digits in the percentage are not localized. This is relevant for Arabic, Persian, Marathi and some other languages.

Our jquery.i18n library has a ready-made solution for both problems - it has a flexible parameter system that allows any word order and it has automatic digit localization. So to have complete localization you should either use jquery.i18n or come up with your own parameter functions.

menu-i18n is an experimental extension made by Thomas Leathrum when we started to talk about localization. It was useful to give a prototype to base the discussion on. However, this extension has limits and we found many issues to address including your point 1). I don't think we thought about 2), although we also mentioned use of commas/dots in numbers. I'm sure Wikipedia folks has a good experience with localization and we certainly want to take benefits from your efforts and make the interface as convenient as possible for localization folks.

As Peter said, we are going to have a developer meeting today to plan MathJax's future developments now that MathJax 2.1 is released. Localization will be one of the topic. Personally, I would be in favor of using your extension or at least relying on it as much as possible. There is one thing that I would like to ask. Your extension is called "jquery.i18n" so I guess it requires jquery at the moment. Does that mean that MathJax has to include jquery or call a jquery installation on another server? Perhaps it does not rely so heavily on jquery and we could replace the use of jquery by MathJax's own API?

Thanks.

For the record, a summary of our previous discussions is available on our dev wiki:

https://github.com/mathjax/MathJax/wiki/Mathjax-v2.x-potential-feature%3A-localization

First I'd like to thank you again for providing another solution to localization in MathJax. I just read the code of jquery.i18n and speaking for myself I don't think it is really worth trying to integrate it to MathJax. The source code is small and there are many things that I think could be simplified if we implement something similar in MathJax. For example, I'm not sure whether redefining the split function is essential? Or whether you really need jquery to implement such a library? It seems that you only use it to load the file asynchronously but MathJax already has an API to do that and for this and the callback function in i18n.properties, we would rather use our own API to synchronize with MathJax's code, put feedback to our UI and author's listeners etc. Also, I think only the "map" mode is really important? We want to allow users to easily share the translation files and so the possibility to add Javascript code in the translation files with the "vars" mode does not sound robust nor secure. There are other weird things like why using parameters[i].split('='), keeping the first part and putting back the other parts together, instead of just parameters[i].split('=', 1)? And for multi-line values, why are these parts from the first line are only put back *after* the subsequent lines? Similarly in i18n.prop, it seems you are really giving you a lot of pain to implement a parser from the value to an array... just to finish with a typo "value.lengh == 1" that fortunately is without consequences. Mozilla uses a different format for *.properties files [1] where the parameters are just given by %1$S, %2$S etc and the percent can be represented by %S%%. That really just seems enough to me as the important point is to be able to pass the arguments and handle different orders in different languages, as you mentioned in comment 22. Replacing these placeholders will simply be a matter of calling the string's "replace" method with the right regexp expression and substitution function. I guess the Javascript's native regexp system will be more reliable and efficient (perhaps even more than just constructing the return value from the computed array).

I suspect we will just use JSON files to do the mapping from a key to a value and so parseData could just be replaced by our asynchronous file loading API while i18n.prop could just be replaced by a call to the standard "replace" method with appropriate parameters as I indicate above. I was not able to find the MathJax extension/changeset to use the jquery.i18n library. Is it public, still work in progress? I can't see the automatic digit localization in jquery.i18n, so perhaps it is implemented somewhere else? Is there anything important point to consider you'd like to mention besides what you indicate in comment 22?

Of course, we'd definitely want your translation files and I agree with comment 17 that it is important to have a format supported by Wikimedia and that follows i18n good practices. It seems that you have a JsonFFS script which "implements a message format where messages are encoded as key-value pairs in JSON objects". Would that make sense to you if we use this format? Or do you prefer to provide *.properties files and let us do the conversion?

Thanks.

[1] https://developer.mozilla.org/en-US/docs/XUL_Tutorial/Property_Files
[2] https://gerrit.wikimedia.org/r/gitweb?p=mediawiki/extensions/Translate.git;a=tree;f=ffs;hb=HEAD

Hi, some questions for the i18n experts:

1. Does the translatewiki interface provide localization notes (like comments from the original translation file) to help translators to figure out the context of a strings to translate (e.g. to indicate whether "Post" is a noun or a verb, or explain technical terms)? And what is the best way to insert such localization notes in the translation files?

2. We would like MathJax to support plural forms. It seems that among the formats supported by Mediawiki, only YAML and PO have this feature implemented (and the way plural form is handled in YAML seems slightly more restrictive than in PO files). In particular that does not seem available in the json format used in the jquery.i18n extension mentioned and which is also the closest to MathJax's internal structure. What has been the experience with translators on that issue and what would you recommend?

3. Key-based format seems more robust to avoid breaking the translation when something like a small typo is fixed or when one has to export the translation to other formats. Most formats seem to take this approach but not PO files. Again, what do you recommend/prefer here?

Thanks.

1. Yes it does. We usually store them under "qqq" language code. This can be exported alongside other translations (but not displayed in the software) or can be edited directly from translatewiki.net

2. It does. It uses the inline syntax like some of our other formats. See https://github.com/wikimedia/jquery.i18n/wiki/Specification#wiki-Plural_Gender_Grammar_support

3. I recommend key based formats. I have a blog posts coming up on my take on Gettext.

https://github.com/wikimedia/jquery.i18n/wiki/Specification#wiki-API is perhaps a better link, gives an example how inline plural syntax looks. Internally it uses the plural logic from CLDR - CLDR plural rules are also used in the Ruby on Rails YAML format.

(Mmmh, either the jquery.i18n code has changed a lot or I looked into the wrong repository in comment 25. I remember there were only a 2 or 3 files that had not been changed for one year...)

I was reading the FFS list ( https://gerrit.wikimedia.org/r/gitweb?p=mediawiki/extensions/Translate.git;a=tree;f=ffs;hb=HEAD ) and a grep for "plural" only returned YAML and PO. But I realize that the plural rules and gender are just indicated in the localizable string itself for the JSON format used in jquery.i18n. Is there a precise spec for this format, at least for the basic features? If not, perhaps examples that can be used to "guess" the format? In any case, having some translation files for the languages you translated (as indicated in comment 22) would be very helpful.

What kind of spec are you looking for? For developers or translators? As a dev you pass the info (number for plural) when calling the message. As a translator you use {{PLURAL:$1|uga|chaga}} with as many forms as it supported for given language.

Should we move this discussion to irc/email/wiki?

Thanks, yes I understood that but I'm looking for a spec that will gives exact syntax (like a grammar, regexp, or even informal description) if we are to implement a similar interface, so that we are sure that the files are compatible. For example, the wiki pages uses lowercase "plural" but comment 30 uses uppercase "PLURAL", so I guess this is case-insensitive? I suppose there are other exceptions of this kind.

I'll continue to discuss with the MathJax team and perhaps we could reopen the discussion on the MathJax dev mailing list later if Bugzilla is not the right place.

In MediaWiki and jquery.i18n the magic words are indeed case sensitive. Also, rather than using explicit keywords from cldr, we just rely on the order of the forms. If there are fewer forms than specified by the rules, we use the last provided form to fill the missing forms.

Maybe looking at the tests https://github.com/wikimedia/jquery.i18n/blob/master/test/jquery.i18n.test.js helps.

The keywords are case *in*sensitive (I think it was just a typo above).

This will be fixed upstream in the upcoming MathJax v2.2 release. The beta version has been released today cf. http://www.mathjax.org/mathjax-v2-2-beta-now-available/.

We've asked to be hosted on TranslateWiki, otherwise we'll put something on Transifex. We're not using jquery.i18n for several technical reasons but our format is very close to that and we're happy to provide multiple file formats.

This is now fixed upstream in MathJax v2.2 http://www.mathjax.org/mathjax-v2-2-now-available/

Peter, you might want to more info on the translations:

• How should people use the two new localizations (German, French)?
• Where should people go to work on adding more translations?
• What technology is used (most relevant for this bug).

MathJax 2.2: https://gerrit.wikimedia.org/r/64522

• You can right click a formula and select a translation from the menu. I haven't been able to find a config based way to 'preselect' a language, based on the information that we already have about users. Peter, can you clarify if I missed something ?

• I've heard they are talking to translatewiki.net https://groups.google.com/forum/?fromgroups#!topic/mathjax-users/FA5BNgw7AuM

• https://github.com/mathjax/MathJax/tree/v2.2-beta/unpacked/localization/en

Hm, sorry about that. I had replied, but it didn't make it through. This is what I wrote two days ago.

Matthew,

Thanks, I mostly wanted to let everyone know that the v2.2 is out of beta.

Our documentation at docs.mathjax.org is usually the right place to
find answers but since you asked, let me try.

1. Not sure what you're after. Page authors set a preferred language

via their MathJax configuration, page visitors can override this via
the MathJax menu, page authors can prevent them from doing so by
removing the corresponding menu entry. Again, our documentation has
more details. The mediawiki math-extension people probably want to
take a look.

2. We're still working on getting MathJax hosted at TranslateWiki and

Transifex. Our file format is very similar to the jquery.i18n format
and we'll convert it to other formats where needed. Again, see our
documentation (which for now will point to our developer wiki
https://github.com/mathjax/MathJax/wiki/Quick-guide-to-translating-MathJax).

3. Again, not sure what's meant. I'm guessing you're asking if we're

using jquery.i18n? In that case, the answer is no. It was not powerful
enough. In addition, we don't use jquery, so using it would have
significantly added to the MathJax load. Maybe I should mention that
MathJax is very modular with its own extension system, so technically
the answer would be: it's a MathJax extension. But I'm guessing that
won't help.

Hope this helps,
Peter.

(In reply to comment #38)

1. Not sure what you're after. Page authors set a preferred language

via their MathJax configuration, page visitors can override this via
the MathJax menu, page authors can prevent them from doing so by
removing the corresponding menu entry.

Yes, that is what I was trying to find (either directly or through a link) in the announcement. It seems from http://docs.mathjax.org/en/latest/localization.html it is passed in the URL.

3. Again, not sure what's meant. I'm guessing you're asking if we're

using jquery.i18n? In that case, the answer is no. It was not powerful
enough.

Yes, this was before I looked at the implementation, so I was wondering if you were using a pre-existing JavaScript localization library.

Related URL: https://gerrit.wikimedia.org/r/67001 (Gerrit Change Ibf43a3e44ea2bfdbd9fddc01c71c72ba097ff1b7)

After a bit of grepping I found the relevant entry points to configure the language I think, MathJax.Localization.locale and MathJax.Hub.config.menuSettings.locale

Related URL: https://gerrit.wikimedia.org/r/67009 (Gerrit Change Ie7530d3a904fb39945ba8b8d261852c2b8cb666b)

Related URL: https://gerrit.wikimedia.org/r/67014 (Gerrit Change I4e21cc5cdcb0d7a31a97b3b136d725f67a53f75c)

(In reply to comment #41)

After a bit of grepping I found the relevant entry points to configure the
language I think, MathJax.Localization.locale and
MathJax.Hub.config.menuSettings.locale

menuSettings.locale is actually the key one. The rest follow from there, but if menuSettings.locale is omitted, it won't work. See the last change above.

This is fixed.

The language selection is still limited. In the meantime, I encourage people to submit pull requests upstream (https://github.com/mathjax/MathJax/pulls) to add languages. Currently, they have German, English, and French.

As noted above, they are looking at using a translation site like TranslateWiki in the future.

Either way, when a new translation hits upstream master, you can file a bug to make sure our extension gets updated.

Until the project is added to translatewiki.net and pull requests are needed from translators, this can't be considered fixed.
All the discussion is in this report, which has always been marked upstream, so I don't see why split it.

Well sort of depends on your interpretation of 'translatable' I guess. Anyways, it definitely where my 'assign' stops :D

I'd second Derk-Jan. The "upstream" tag seems wrong to me (although technically correct).

I do understand that the real bug is "MathJax doesn't have enough translations", but that's something that the MediaWiki community can actually help fix -- and in a real sense we (MathJax) cannot fix this on our own since we don't have the resources to hire an army of translators.

So perhaps it should be closed and another bug opened that calls for translations.

(In reply to comment #48)

So perhaps it should be closed and another bug opened that calls for
translations.

No. This is a MediaWiki extensions and MediaWiki extensions are translated at translatewiki.net. You won't get help by MediaWiki translators outside of it and from a Wikimedia perspective the issue is not solved until the translation doesn't live there. I don't see a reason to separate the two things now.

Just to clarify: MathJax is not a MediaWiki extension, it's a third party library used by the math-extension. Either way, the MathJax team has no power over getting MathJax accepted into TranslateWiki, so help is needed on that end. We'd be thrilled to be there, but since it's by invitation only, all we can do is ask.

I've created an upstream issue, https://github.com/mathjax/MathJax/issues/493, specifically about coordinating what needs to be done to get MathJax on TranslateWiki.

MathJax is now on TranslateWiki. Translators are encouraged to participate at https://translatewiki.net/wiki/Special:Translate/out-mathjax-0-all

Thanks to everyone at WMF, TranslateWiki and MathJax for making this happen!

I hope we can contribute to the project's success with translatewiki.net.

Please use plain "translatewiki.net" in your communication (TWN as short form if you have to). Other forms more easily confused with other online translation platforms and wikis.

It seems part of my original pt-BR translations were not imported to translatewiki.net, since it shows "61% completion" at
https://translatewiki.net/w/i.php?title=Special:MessageGroupStats&language=en&group=out-mathjax-0-all#sortable:3=desc

For example, I had to create
https://translatewiki.net/w/i.php?title=MathJax:Mathmenu-OriginalMathML/pt-br&diff=5016014
again, but this was already in the MathMenu.js attachment from bug 49267.

(In reply to comment #55)

It seems part of my original pt-BR translations were not imported to
translatewiki.net, since it shows "61% completion" at
https://translatewiki.net/w/i.php?title=Special:
MessageGroupStats&language=en&group=out-mathjax-0-all#sortable:3=desc

For example, I had to create
https://translatewiki.net/w/i.php?title=MathJax:Mathmenu-OriginalMathML/pt-
br&diff=5016014
again, but this was already in the MathMenu.js attachment from bug 49267.

Should be resolved. I ran "php sync-group.php --lang=* --group=out-math* --noask"

(In reply to comment #56)

Should be resolved. I ran "php sync-group.php --lang=* --group=out-math*
--noask"

Thanks!