Page MenuHomePhabricator

Language names blurred and difficult to read using the hardcoded "Autonym" font on Windows computers
Closed, ResolvedPublic

Assigned To
None
Authored By
Danny_B
Oct 29 2013, 9:25 PM
Referenced Files
F12132: webfontkit-20131121-182104-fixed.zip
Nov 22 2014, 2:20 AM
F12129: win8-autonym.png
Nov 22 2014, 2:19 AM
F12131: win8-sidebar.png
Nov 22 2014, 2:19 AM
F12128: text_rendering_problem_Commons.png
Nov 22 2014, 2:19 AM
F12127: text_rendering_problem.png
Nov 22 2014, 2:19 AM
F12122: Blurred_Autonym_font.png
Nov 22 2014, 2:19 AM
F12126: font-autonym-arial-sample.png
Nov 22 2014, 2:19 AM
F12125: Autonym-FF22-Opera12-IE9.png
Nov 22 2014, 2:19 AM

Description

Blurred Autonym font

Interwiki names blurred and legible with difficulty (using the hardcoded "Autonym" font)

Firefox 25 / WinXPSP3 / standard antialiasing (= no ClearType)


Version: unspecified
Severity: major
OS: Windows 7
Platform: PC
See Also:
https://bugzilla.wikimedia.org/show_bug.cgi?id=40874
https://bugzilla.wikimedia.org/show_bug.cgi?id=56552
https://github.com/santhoshtr/AutonymFont/issues/38
https://github.com/santhoshtr/AutonymFont/issues/50

Attached:

Blurred_Autonym_font.png (260×180 px, 4 KB)

Details

Reference
bz56346

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 22 2014, 2:19 AM
bzimport set Reference to bz56346.
bzimport added a subscriber: Unknown Object (MLST).

Cannot reproduce with Win8, FF25 and 1024x768 on https://cs.wikipedia.org/wiki/ on crossbrowsertesting.com (only offering FF23 for XPSP3).

I am getting very similar messed up rendering on Opera 12 on Windows XP with ClearType rendering enabled.

Created attachment 13642
Screenshot from en.wp

Attached:

2013-11-01_00_32_11-Orrefors_-_Wikipedia,_the_free_encyclopedia_-_Opera.png (414×169 px, 5 KB)

Temporarily it can be fixed by adding in personal css:
#p-lang div.body ul { font-family: sans-serif; }

mr.heat wrote:

Autonym glyphs are missing pixels

Please tell me if I should post this as a separate issue.

I'm using Opera 12 on Windows 7 with "Clear Type" rendering turned off (on purpose, because I get headaches from the red and blue blurry shadows it adds around the glyphs). Unfortunately the rendering of Autonym is a mess (sorry to say that, it's the best word to describe what I see, see my screenshot). This makes the situation a lot worse compared to the situation before where a few glyphs may be missing or displayed as rectangles. Now some pixels are completely missing. An "o" looks like an "u" and "m" looks like "rri", for example.

Proposed solution: Either fix the font or simply don't use it on systems that don't have a problem with displaying the glyphs in the interlanguage links. If I disable Autonym (using the web developers tools) everything is back to normal, no glyphs are missing and no rectangles are displayed. In other words: Don't "fix" things that aren't broken. Thanks.

Attached:

Autonym-rendering-desaster.png (592×378 px, 11 KB)

I'm guessing there's not hinting information in the font; Windows' TrueType font renderer is notorious for relying on good hinting for on-screen resolutions, while OS X and Linux tend to ignore it and optimize their rendering without hinting.

I've gone ahead and filed an entry on the Autonym github issue tracker and copied the images (they're viewable inline there, for easier reference): https://github.com/santhoshtr/AutonymFont/issues/38

mr.heat wrote:

(In reply to comment #6)

Suggested reading:
http://www.smashingmagazine.com/2012/04/24/a-closer-look-at-font-rendering/

See my "Subpixel rendering is a lie" comment there.

(In reply to comment #6)

Suggested reading:
http://www.smashingmagazine.com/2012/04/24/a-closer-look-at-font-rendering/

TL;DR: Windows is bad. Danny, do you use Windows?

(Strike that, answer is in comment 0; setting Platform.)

Just for the record:

The same issue is with the <languages/> box.

(-> changing the summary to be more general)

michael wrote:

The language names on en.wiktionary.org’s home page render correctly on Safari 6.1/Mac, Firefox 25/Mac, and Chrome 30/Mac (on OS X Mountain Lion).

They also renders correctly on iPad Safari (iOS 5), except that Burmese and Khmer characters are broken anyway.

But they render correctly on the Mac with the Autonym font disabled. Every language name renders every character with the font spec set to “sans-serif.” Developers, please stop trying to “fix” an OS that doesn’t need help. Just serve this font to systems that need it, and do a reasonable amount of testing before releasing.

michael wrote:

The problem seems to be “missing pixels.” Only one of three screenshots shows anything “blurred” that isn’t also occurring in the default font. Might both be caused by lack of hinting, but might be two different bugs.

mr.heat wrote:

Firefox/Opera 12/IE 9 with ClearType enabled

Here is an other screenshot with ClearType enabled. From left to right:

  • Firefox with hardware acceleration disabled
  • Opera 12
  • IE 9

Same bug in Firefox and Opera, some pixels are completely missing. Enabling or disabling ClearType has no effect on this bug, obviously.

IE 9 uses hardware acceleration by default. Because of this the font looks different. But it's also distorted. The glyphs are more narrow and more blurred compared to the default font (seen in the upper half of the screenshot). Some glyphs are distorted. Look at the "g" from "English" and the "S" from "Suomi".

Enabling ClearType *and* hardware acceleration in Firefox makes it the same as IE 9. Enabling hardware acceleration *without* ClearType makes it worse. A lot worse. That's why I have no other choice than do disable both.

Why don't you use the default fonts for simple words like "English" that don't contain any problematic characters?

Attached:

Autonym-FF22-Opera12-IE9.png (352×530 px, 22 KB)

I also experience this bug, with FF 24.0 on Windows 7.
Isn't this a regression, since it worked before?

Created attachment 13681
Comparison Autonym Arial

I made a sample to show the problem: Two times "Nederlands" is shown, first with Autonym and then with Arial. The font size (height) is the same, even though Arial is slightly wider. Arial is better readable, just compare the first "e" of "Nederlands" or the lower-case "n": they seem to have holes.

Attached:

font-autonym-arial-sample.png (324×612 px, 26 KB)

(In reply to comment #15)

Isn't this a regression, since it worked before?

Technically no because it's a new feature, no fonts have been served for interwikis in the last few weeks to save on bandwidth etc.

As suggested by TMg (comment #6), is it not possible to let the browser choose the Autonym font only if there are missing glyphs? I’m not very knowledgeable about the details, but I know it is the global goal of the CSS list of fonts in font-family; is something like " font-family: sans-serif, "Autonym"; " would achieve the goal by only searching missing glyphs in Autonym?

mr.heat wrote:

(In reply to comment #18)

something like " font-family: sans-serif, "Autonym"; "

Unfortunately this is not how it works. My suggestion was to apply the font only to language names with otherwise missing glyphs. For example, the list in the sidebar looks like this:

<ul>
<li class="interwiki-en"><a lang="en" hreflang="en">English</a></li>
<li class="interwiki-fr"><a lang="fr" hreflang="fr">Français</a></li>
<li class="interwiki-ja"><a lang="ja" hreflang="ja">日本語</a></li>
</ul>

Remove the current rule:

#p-lang ul { font-family: "Autonym", sans-serif; }

Instead use a rule set like this for all non-latin language names:

#p-lang a[lang=ja], #p-lang a[lang=th] { ... }

Selectors like this work in all browsers including very old ones. Problem solved.

At least exclude the Wikidata "Edit links" item from the rule by applying it only to items that are in an other language (this may be an other issue, please tell me if I should split this):

#p-lang a[lang] { ... }

Besides, what was the problem in the first place? Don't you think all users that are able and interested in a specific language have the proper fonts installed? In think missing glyphs or rectangles are only visible to people that aren't interested in these languages anyway. Again, I'm very sorry to say that but this looks like a fix without a real world problem to me. I'm aware that it may help a few people but the current way the font is used creates more problems than it solves.

michael wrote:

#p-lang a[lang=ja], #p-lang a[lang=th] { ... }

Selectors like this work in all browsers including very old ones. Problem
solved.

Doesn’t work in MSIE 6. Sadly, millions use it and I think we still intend to support it. See http://www.ie6countdown.com/

Is there any official or community statement on which browsers we do support?

Don't you think all users
that are able and interested in a specific language have the proper fonts
installed?

No. I think the point is to make wikis accessible to people who can’t or won’t install software on their machines. This means tens of millions who rely on library or work PCs, or the latest mobile devices, or almost anything in between. Not only techies need to read their own language.

That said, it would be nice if the font spec only applied to the necessary language names, whichever those may be.

Michael: here https://www.mediawiki.org/wiki/VisualEditor/Target_browser_matrix (warning, old data) I read:
"The unofficial rule for Wikimedia's browser support is that we try to support reading (and not necessarily other functions) for all browsers used by at least 0.1% of page hits based on the WikiStats data."

If I read http://stats.wikimedia.org/wikimedia/squids/SquidReportClients.htm correctly, MSIE 6.0 does not meet that requirement.

(Yes, that's VE, which is not going to work with IE 6, but I guess the "less than 0,1%"-rule is generic? It's also mentioned at https://www.mediawiki.org/wiki/Compatibility#Grade_B .) :)

michael wrote:

Thanks for the enlightening links, all.

I see that MSIE 6 accounts for 0.83% of traffic, well over the 0.1% threshold, and thus remains supported as a grade B browser, for reading, but not necessarily having all features working. The same applies to MSIE 7 (1.47%) and 8 (4.71%).

All these guidelines carry out-of-date warnings, and don’t account for the conflicted grey area wherein MSIE 6 and 7 are supported for reading, but not for the ULS which provides fonts that may be necessary for reading.

I think the gist is that if you are using MSIE 6, you are more-or-less on your own, and may need to install some fonts. Editors and developers don’t want to abandon MSIE 6 users, but no one will invest time and energy into testing MSIE 6 and 7, so the realistic support level is “good luck, mate.”

Also good to know that WikiMedia is, or was, supposed to work without CSS3, without JavaScript, and in text-only browsers.

One more question: does the ULS try to work in MSIE 6 and 7, or is it disabled in these browsers?

michael wrote:

(In reply to comment #24)

I see that MSIE 6 accounts for 0.83% of traffic, well over the 0.1%
threshold,

Holy Hannah, MSIE 5.5 also qualifies with 0.11%!

IE6? OK. Let's be serious. Majority of other users INCLUDING users of IE6 are served with a bad font. Users that are interested in certain language will have glyphs for the font they are interested. So using a bad font just creates a bad experience for all users.

BTW. The font looks good only for 13px. So if you must please at least change default size.

Also BTW, notice this font rendering is bad in almost any other size then 13px.

(In reply to comment #26)

IE6? OK. Let's be serious. Majority of other users INCLUDING users of IE6 are
served with a bad font.

IE 6(or anything less) will not get this font as per latest changes. Screenshot of https://www.mediawiki.org/w/index.php?title=UniversalLanguageSelector/AutonymFont using IE6 on Windows XP

http://app.crossbrowsertesting.com/public/id1b2af101bb8ef7/livetests/1196571/snapshots/z7f7810ca035e37ef94c

(In reply to comment #0)

Created attachment 13624 [details]
Blurred Autonym font

Interwiki names blurred and legible with difficulty (using the hardcoded
"Autonym" font)

Firefox 25 / WinXPSP3 / standard antialiasing (= no ClearType)

Screenshot of
https://www.mediawiki.org/w/index.php?title=UniversalLanguageSelector/AutonymFont
using FF22 on Windows XP SP3 with the latest version of font http://app.crossbrowsertesting.com/public/id1b2af101bb8ef7/livetests/1196568/snapshots/zb68cdfaa35be26924f1 , font size 13px

Attached:

Blurred_Autonym_font.png (260×180 px, 4 KB)

As I quickly realized, and confirmed at the Autonym website, the majority of Latin glyphs were extracted form FreeSans, possibly the worst-rendering (at leats on Windows) open source font available. FreeSans (as well as the other UCS/GNU FreeFonts) lack *any* type of hinting, on which Windows relies for proper font rendering. We could blame Windows, but that will not solve anything.

Autonym should find a better source for its Latin glyphs, preferably Nimbus Sans L (on which FreeSans is based!), which include at least some basic hinting.

(In reply to comment #30)

What's the plan for this bug?

ULS master has hinting added for the Autnym font. This is active in the Wikimedia branch 1.23wmf3.

mr.heat wrote:

(In reply to comment #30)

What's the plan for this bug?

I wonder why you are asking this question. There is no "plan". For the average Bugzilla report it's often like "ignore it long enough to make the users give up and stop complaining about, then pretend there is no problem". It doesn't even matter if a report is almost 10 years old and received 50 votes (e.g. bug 674). There is no developer assigned and there are no developers to assign. WMF always moves them to the next project in the moment a feature is deployed. Sometimes the original developers even left WMF. If you are lucky a volunteer will make a pull request.

I really, really don't understand why such bad changes aren't reverted (which would be simple), fixed (which may be hard) and deployed again (which is again simple). It's like WMF is more afraid of reverting something than being criticized for making bad changes. Why is that?

I'm sorry, Andre, I'm just trying to explain what many users actually think. Reports like this are just tips of the iceberg.

(In reply to comment #31)

ULS master has hinting added for the Autnym font.

What about excluding the Wikidata link from using the font? What about excluding all Latin languages from using a font they don't need? What about delivering the font only to browsers that would show rectangles otherwise?

(In reply to comment #31)

ULS master has hinting added for the Autnym font. This is active in the
Wikimedia branch 1.23wmf3.

I see no difference. Do I need to do something? (Tested on pl.wikt, which is at wmf3 now.)

Has there been any progress with this? What is the status? Is it supposed to be fixed? (Because it's not.)

e.martinson wrote:

screenshot font-rendering, en.wp in Firefox 25, Win7

Similar to "Autonym glyphs are missing pixels" shot.

Attached:

text_rendering_problem.png (414×351 px, 11 KB)

e.martinson wrote:

Comment on attachment 13809
screenshot font-rendering, en.wp in Firefox 25, Win7

Seeing problem in Commons as well in language choice link (upper right).

I share the opinion of many others:

Autonym is actually
a) a fix for a problem that most people never experienced.
b) making thinks even worse as they would have been without ULS and Autonym in the first place

The whole sad story from my point of view:

Common sense dictates that users have fonts installed for their native languages. If they work multilingual they're most likely do also have all fonts necessary for those languages installed on their system. On top of that we can assume that on all modern systems there's support for even more languages that a normal user never ever needs anyway.

Against this common sense ULS was designed to load webfonts for a huge bunch of languages. Therefore we started to load additional font support for every Wikipedia user, even if it was most unlikely that a user actually needed it.

It turned out, that ULS was detrimental to performance. The additional traffic due to webfonts loading seems to have been so large, that even WMF saw the need for a change (I'm quite sure WMF would not have even cared to develop Autonym if it had only affected client performance).

Now we have Autonym, which brings all sorts of problems, including the extremely bad rendering we're discussing in this bug...
And now we're trying hardly to fix font rendering of Autonym (which will by definition never be as good as a native font like Arial on Windows), because we put much effort into Autonym, which was a fix for the bad performance for ULS, which was a huge project supported by WMF which was supposed to fix problems and add features... that users never faced and seldom requested!

The whole thing seems pretty messed up to me, like many things that come from WMF lately. Since I see Siebrand Mazeland is active on this bug (who is the "Product Manager Language Engineering" of the WMF after all and therefore probably the one in charge for everything we're discussing here): Think about the doubts many users have about ULS, Autonym and all the other new language features. And think about why there are no users praising ULS, while there are tons of users facing problems with ULS and everything that is connected to it... I don't want to blame you for doing a bad job - but I'm afraid there is definitely room for improvement right now!

(In reply to comment #38)

[Removing random added keywords, see
https://bugzilla.wikimedia.org/editkeywords.cgi ]

https://bugzilla.wikimedia.org/describekeywords.cgi

The original link doesn't work for normal users.

e.martinson wrote:

(In reply to comment #38)

[Removing random added keywords, see
https://bugzilla.wikimedia.org/editkeywords.cgi ]

Sorry my addition of those keywords seemed random, Andre. It wasn't, but I'm not very experienced with Bugzilla. It seemed several commenters felt this bug was not drawing enough attention, and my reading of the design and community-consensus-needed keywords led me to believe they would be appropriate here and might draw more attention.
--Eric

michael wrote:

(In reply to comment #37)

I don’t completely agree with Eduard Brown.

WikiMedia hosts an unusual multilingual range of websites on unusual multilingual software. Text in some languages appears broken on many computers. There is nothing wrong with aspiring to fix the breakage when the technology is available.

Common sense dictates that users have fonts installed for their native
languages.

An ideal. The reality is that millions of people cannot install fonts on the devices they depend on, for many different reasons. Smartphone users? Refugee immigrants relying on library computers configured in a language they barely know? They deserve access to information.

Autonym (which will by definition never be as good as a native font like Arial on Windows)

I don’t understand the logic of this. Arial is hardly a great typeface anyway.

I'm afraid there is definitely room for improvement right now!

True. I don’t know why this was pushed out without some basic testing on common platforms.

Most modern browsers on modern OSes render most language scripts these days. A simple fix for much of this would be to embrace installed fonts, and push Autonym to the bottom of a font stack that includes the default sans-serif fonts on the various platforms. I wonder if, indeed, “font-family: sans-serif, autonym;” would work to make this font the fallback’s fallback. Ideally, it should only be rendering text that would show up as tofu. Ideally.

It would help if this bug identified the specific code revisions (Gerrit links) that are responsible for this recent regression.

It would also help, for triaging, to know how many operating systems/browsers are affected (or rather, what percentage of typical users)? Only Windows users?

(In reply to comment #42)

It would also help, for triaging, to know how many operating systems/browsers
are affected (or rather, what percentage of typical users)? Only Windows
users?

So far the only information provided pointed exclusively to Windows problems; Erik removed Windows from the platform/OS fields shortly after comment 36 without clarifying why.

@Nemo: Thanks for comment 43, very helpful!

@erik: Could you clarify comment 36? Which other systems did you experience the problem with?

(In reply to comment #42)

It would help if this bug identified the specific code revisions (Gerrit
links) that are responsible for this recent regression.

@MZMcBride:
https://blog.wikimedia.org/2013/10/28/the-autonym-font-for-language-names/ links to https://bugzilla.wikimedia.org/show_bug.cgi?id=40874 which links to https://gerrit.wikimedia.org/r/#/c/84850/ and https://gerrit.wikimedia.org/r/#/c/84851/

e.martinson wrote:

language-choice text blurring on Commons

screenshot of language-choice link text blurring, Win7 64-bit, Firefox 25, Chrome 31.0.1650.57 m

Attached:

text_rendering_problem_Commons.png (52×196 px, 1 KB)

e.martinson wrote:

(In reply to comment #43)

(In reply to comment #42)

It would also help, for triaging, to know how many operating systems/browsers
are affected (or rather, what percentage of typical users)? Only Windows
users?

So far the only information provided pointed exclusively to Windows problems;
Erik removed Windows from the platform/OS fields shortly after comment 36
without clarifying why.

Sorry, I switched the platform/OS to "all" because I thought this problem was not restricted to Windows. However, just now I checked on a Mac in Firefox and Chrome and did not encounter the text-blurring problem. But I'll leave it to someone else to switch back as I have not tested it on any other systems.
--Eric

e.martinson wrote:

(In reply to comment #44)

@Nemo: Thanks for comment 43, very helpful!

@erik: Could you clarify comment 36? Which other systems did you experience
the
problem with?

(In reply to comment #42)

It would help if this bug identified the specific code revisions (Gerrit
links) that are responsible for this recent regression.

@MZMcBride:
https://blog.wikimedia.org/2013/10/28/the-autonym-font-for-language-names/
links to https://bugzilla.wikimedia.org/show_bug.cgi?id=40874 which links to
https://gerrit.wikimedia.org/r/#/c/84850/ and
https://gerrit.wikimedia.org/r/#/c/84851/

Um, now I must admit I had not tested it on any other systems. Don't know why I thought it was cross-platform. Mea culpa.
--Eric

e.martinson wrote:

(In reply to comment #47)

(In reply to comment #44)

@Nemo: Thanks for comment 43, very helpful!

@erik: Could you clarify comment 36? Which other systems did you experience
the
problem with?

(In reply to comment #42)

It would help if this bug identified the specific code revisions (Gerrit
links) that are responsible for this recent regression.

@MZMcBride:
https://blog.wikimedia.org/2013/10/28/the-autonym-font-for-language-names/
links to https://bugzilla.wikimedia.org/show_bug.cgi?id=40874 which links to
https://gerrit.wikimedia.org/r/#/c/84850/ and
https://gerrit.wikimedia.org/r/#/c/84851/

Um, now I must admit I had not tested it on any other systems until now. Don't > know why I thought it was cross-platform. Mea culpa.
--Eric

e.martinson wrote:

(In reply to comment #48)

(In reply to comment #47)

(In reply to comment #44)

@Nemo: Thanks for comment 43, very helpful!

@erik: Could you clarify comment 36? Which other systems did you experience
the
problem with?

(In reply to comment #42)

It would help if this bug identified the specific code revisions (Gerrit
links) that are responsible for this recent regression.

@MZMcBride:
https://blog.wikimedia.org/2013/10/28/the-autonym-font-for-language-names/
links to https://bugzilla.wikimedia.org/show_bug.cgi?id=40874 which links to
https://gerrit.wikimedia.org/r/#/c/84850/ and
https://gerrit.wikimedia.org/r/#/c/84851/

Um, now I must admit I had not tested it on any other systems until now. Don't know why I thought it was cross-platform. Mea culpa.
--Eric

Also setting "Windows 7" as the most recent Windows cited as having the problem ("PC" is a bit too generic).

Created attachment 13823
Screenshot on a default Windows 8, IE 10

Good news? It seems better on Windows 8.

Attached:

win8-autonym.png (768×1 px, 134 KB)

Created attachment 13824
ENWP Sidebar, Windows 8, IE 10

Attached:

win8-sidebar.png (768×1 px, 217 KB)

(In reply to comment #51)

Good news? It seems better on Windows 8.

Could you test it with either Firefox/Chrome/Opera to make sure it's not just IE?

(In reply to comment #51)

Screenshot on a default Windows 8, IE 10

Good news? It seems better on Windows 8.

From your screen-shots, it seems there's something fundamentally wrong with your font rendering(besides anything Autonym may cause):

  • Do you notice the uneven thickness of vertical lines? This should not occur if Hinting works as expected.
  • You seem to have Subpixel-anti-aliasing (alias ClearType) turned off, but you seem to be still utilizing some sort of grayscale-anti-aliasing. Is this on purpose?

(In reply to comment #54)

(In reply to comment #51)

Screenshot on a default Windows 8, IE 10

Good news? It seems better on Windows 8.

From your screen-shots, it seems there's something fundamentally wrong with
your font rendering(besides anything Autonym may cause):

  • Do you notice the uneven thickness of vertical lines? This should not occur

if Hinting works as expected.

  • You seem to have Subpixel-anti-aliasing (alias ClearType) turned off, but

you
seem to be still utilizing some sort of grayscale-anti-aliasing. Is this on
purpose?

I have no idea at all. I happened to be using a Windows 8 PC (not mine) and took those screenshots.

(In reply to comment #52)

Created attachment 13824 [details]
ENWP Sidebar, Windows 8, IE 10

Seems like ordinary Arial. In other words, it does not use Autonym in this screenshot.

Attached:

win8-sidebar.png (768×1 px, 217 KB)

(In reply to comment #41)

(In reply to comment #37)

Autonym (which will by definition never be as good as a native font like Arial on Windows)

I don’t understand the logic of this. Arial is hardly a great typeface
anyway.

Don't confuse design with legibility. Arial is the default sens-serif on Windows, and renders *very* well on Windows. Autonym renders *very* bad on Windows, because the font (and its source FreeSans) were never desigend with Windows in mind, and therefor lack any hinting.

A simple fix for much of this would be to embrace installed fonts, and push
Autonym to the bottom of a font stack that includes the default sans-serif
fonts on the various platforms. I wonder if, indeed, “font-family:
sans-serif, autonym;” would work to make this font the fallback’s fallback.
Ideally, it should only be rendering text that would show up as tofu. Ideally.

I doubt it. Windows usually uses the first available font in the font stack and sticks with it, and revert to its default unicode font if it can't find a glyph (usually Lucida Sans Unicode).

michael wrote:

(In reply to comment #57)

(In reply to comment #41)

(In reply to comment #37)

Autonym (which will by definition never be as good as a native font like Arial on Windows)

I don’t understand the logic of this. Arial is hardly a great typeface
anyway.

Don't confuse design with legibility. Arial is the default sens-serif on
Windows, and renders *very* well on Windows. Autonym renders *very* bad on
Windows, because the font (and its source FreeSans) were never desigend with
Windows in mind, and therefor lack any hinting.

Sure, the design of a typeface doesn’t matter as much if your system distorts the character shapes to fit a pixel grid. As higher-resolution displays are becoming common, screen hinting is going away and good font design is starting to be considered good font design.

Anyway, the immediate fix is to import hinted character outlines from a better free font. Yes, easier said than done, but there’s no reason it couldn’t match Arial’s quality in any particular OS or browser.

A simple fix for much of this would be to embrace installed fonts, and push
Autonym to the bottom of a font stack that includes the default sans-serif
fonts on the various platforms. I wonder if, indeed, “font-family:
sans-serif, autonym;” would work to make this font the fallback’s fallback.
Ideally, it should only be rendering text that would show up as tofu. Ideally.

I doubt it. Windows usually uses the first available font in the font stack
and
sticks with it, and revert to its default unicode font if it can't find a
glyph
(usually Lucida Sans Unicode).

I think it might be more complicated than that. MSIE 6 ignores fallbacks for individual characters missing from a font, but I understand that newer versions have improved on this. As far as I know, Firefox uses its own font choice and rendering engine.

For reference, you may look at this page which tests the Autonym font with all Active Wikimedias, under various styles:

https://meta.wikimedia.org/wiki/Wikimedia_Forum/Autonym_font_test_with_active_Wikipedias

Note that the rendering of the Autonym font may be tweaked with ULS, provided that the span using the autonym class also sets the lang attribute (as it should).

So you can use ULS to select another supported webfont for each language supported by ULS. The Autonym font will be used only if you select "use system font" in ULS for that language. If you select another proposed webfont, it will override the font specified in that class (and this other webfont will be loaded by your browser to render this language).

(In reply to comment #37)

Against this common sense ULS was designed to load webfonts for a huge bunch
of
languages. Therefore we started to load additional font support for every
Wikipedia user, even if it was most unlikely that a user actually needed it.

You're wrong! Autonym in ULS was designed to load ONLY this font instead of a bunch of fonts.

Performance tests show that

  • the download bandwidth uis highly reduced because ULS no longer needs to download many fonts (OK you don't download fonts for your installed system fonts)
  • the time to render the page is faster (the Autonym webfont is normally cached by your brower, it won't load again when you navigate, unless it has been updated on the server)
  • the browser needs to load many less fonts to render lists of language names: with your many system fonts configured for each script or language by your CSS preferences or by your browser settings, the browser will need to load all these system fonts in memory (even if the browser also uses a font rendering cache).
  • Nothing is changed for the rest of the page content (outside language autonyms): the content still uses your own choice of fonts
  • You can tweak webfonts used by ULS to use other more complete (and hinted) webfonts proposed by ULS: they will override the Autonym font if the span element containing the language name does not only use the CSS class "autonym", but also sets the lang="" attribute to specify its language.
  • You can tweak you own CSS rules to provide other local fonts you have for a specific language using CSS lang() selectors.

Note that it's true that the Autonym font lacks hinting; but my test page demonstrates that it should not be used with all CSS styles, but that it works quite well within some limits (font-size:13px;font-weight;normal;font-style:normal;letter-spacing:normal), provided that the browser does not zoom in, on displays with low pixel densities (96dpi).

Displays with high pixel densities look OK (example: smartphones that typically have densities around 190 dpi or more). With their default zoom (you'll experience poor readability only if you soom out).

The critical bad rendering occurs:

  • when the browser zoom is around 120-150 dpi, because glyphs are just scaled up from a low dpi rendering (around 96 dpi on desktops/notebooks displays) without hinting, or
  • when rendering font sizes below 13px (for example, small text: you loose important pixels: don't render lists of languags with a reduced font size; unfortunately the default font-size of MediaWiki is reduced from the browser standard, so you see these effects with lost pixels).

(In reply to comment #62)

The critical bad rendering occurs:

  • when rendering font sizes below 13px (for example, small text: you loose

important pixels: don't render lists of languags with a reduced font size;
unfortunately the default font-size of MediaWiki is reduced from the browser
standard, so you see these effects with lost pixels).

Actually, 13px (0.8 x 16px) is the default for Vector, and Autonym looks acceptable at 13px. However, the navbar has its font-size set to 0.75em, which causes Autonym to drop to 12px, and the problem is very apparent.

Seems like a little CSS tweaking on #p-lang might solve the primary concern.

To achieve comment 63 suggestion:

#p-lang-list li {
    font-size: 0.8125em !important;
}

Then the font in this one sidebar section would be larger than in all the other ones, which is going to look even more silly than the current state. I suggest not doing that.

Here some screenshot on Firefox 25, Windows LCD rendering (*).

12px (en.wikipedia): http://files.espace-win.org/2013/Capture.PNG
13px (fr.wikipedia): http://files.espace-win.org/2013/Capture2.PNG

(*) There are two fonts rendering mode on Windows, one with LCD support when ClearView option is enabled, one without.

Autonym font is a soulution for a problem, that never existed before ULS. Even now I didn't understand the reason to serve (load) webfonts by default to Latin languages. Nowadays even problematic Indic languages have much better support in OSs, a situation which makes Webfonts less than a necessity there. So I think most favorable solution may be disabling webfonts than 'temporarily intelligent' solutions like problematic Autonym font.

Hi. Not sure what you've used for generating the font, but if you've used fontsquirrel.com, then consider the following:

  1. Generate SVG font (if you're not doing it already).
  2. Use ttf above woff.
  3. Preserve original hinting if it was available.

You might want to experiment a bit, but that was the best options for me when generating web fonts from open type font.

Created attachment 13871
Fixed font - notice how the font is defined in stylesheet.css!

Fontsquirrel to the rescue :-). Attaching fixed font (made from automnym.ttf).

Have tested on IE 10, Opera 12, Chrome 31 and Firefox 25 (Windows 7). All glyphs in all sizes seem fine and readable beyond font size 11.

Also tested in IE compatibility modes. The font remains readable even in IE 8, though kerning seem worse in this version (better from IE 9 up). If you see this as a problem then you can set letter-spacing:.1em.

Attached:

(In reply to comment #69)

Created attachment 13871 [details]

Nice! Thank you Maciej!
It looks great on Windows XP too (Windows 7 was the worse).

Now, could we replace the current

https://bits.wikimedia.org/static-current/extensions/UniversalLanguageSelector/data/fontrepo/fonts/Autonym/Autonym.ttf?version=20131112
and
https://bits.wikimedia.org/static-current/extensions/UniversalLanguageSelector/data/fontrepo/fonts/Autonym/Autonym.woff?version=20131112

with these uploaded by Maciej?

This should then allow us to close this bug.

Attached:

Note that the SVG version includes 76 glyphs which are not selectable at all (they have no unicode attribute and no id attribute referenced by some assigned glyph with script features. I wonder how they can be present, or if the tool converting TTF fonts to SVG fonts simply lacks info on how to map the opentype features needed to select and render these contextual glyphs. Some of these glyphs are also non-spacing (no horiz-adv-x attribute) but also not correctly selected. Some of them are for mandatory ligatures, and should be selectable by an unicode attribute specifying 2 Unicode characters, or sometimes more.
Only the contextual forms of letters in the Arabic script (up to 3 in addition to the default isolated form) are selectable.

I don't why are those glyphs present but I think they won't be used anyway if they don't have unicode attribute. If someone knows for sure they are not needed, then I guess they can be removed. BTW selecting characters (and copy&paste) works fine for me for all characters used for Main Page.

Also the "missinglyph" item of the SVG is just mapped with an advance-width equal to the space character; it has no path for the glyph itself, where it should have a path drawing an empty box in that space.

The unselectable glyphs seem to come from an incomplete conversion tool, that does not know really how to map the glyphs selectable by OpenType features into the necessary mappings for making them selectable by the SVG font.

An SVG font cannnot use OpenType feature, because it lacks the logic included in the OpneType renderers to select these features. OpenType features have implicit selection rules from the script and character properties. Then each feature adds GSUB and GPOS rules for mapping glyph definitions. In an SVG font, all the implicit rules must be mapped (the only exception supported by current relase of the SVG font spec being the Arabic joining type). This means that Indic script cannot work with simple one-to-one mappings of a character code to a glyph, with their required features. You need to map Unicode strings (not characters) into a SVG glyph definition to support the equivalent of contextual substitutions, and use the "hkern" (or "vkern") definitions to support the equivalent of OpneType GSUB (for contextual positioning).

I think that the tool used to convert OpenType fonts to SVG font is insufficient for "complex scripts" (those that have REQUIRED features needed; see the OpenType specfiications). This means that extra manual tweaking are needed in the SVG font to make it work correctly (the existing tool is not doing things properly with its one-to-one character-to-glyph mappings, and its bogous unselectable zero-to-one mappings).

For this project, just uploading the generated fonts into the MediaWiki repository is clearly not enough, there should exist a full project containing or documenting the build tools used to perform these conversions.

For this project, just uploading the generated fonts into the MediaWiki

repository is clearly not enough, there should exist a full project containing
or documenting the build tools used to perform these conversions.

I don't agree. The generated font is better in practice. In practice all fonts are visible and selectable (or at least in all browsers I tried).

Also note that there is a configuration in the attachment describing the process ("generator_config.txt"). So the effect should be reproducible without any problems.

If you can make a better font go ahead. Seriously. I know that the tool (fontsquirrel) is not perfect. I've converted a different font and had to do a lot tweaking to get good results. BUT we should progress to a better solution then the current one and then fix other issues (if we can).

Nux, (In reply to comment #68)

Hi. Not sure what you've used for generating the font, but if you've used
fontsquirrel.com, then consider the following:

  1. Generate SVG font (if you're not doing it already).
  2. Use ttf above woff.
  3. Preserve original hinting if it was available.

ULS Preserve the hints in woff and eot. See
https://gerrit.wikimedia.org/r/#/c/95987/ (Nov 20)

Documentation about how the fonts are developed: https://github.com/santhoshtr/AutonymFont/

And about how webfonts are created: see http://www.mediawiki.org/wiki/Extension:UniversalLanguageSelector#Preparing_webfonts

Note the 200% size increase in your version of the font from fontsquirrel.

importing glyphs from some free fonts is not enough; some scripts need that you also borrow the OpenType features allowing their contextual selection (for now only the Arabic contextual forms are borrows, Indic scripts are ignored so you only get the isolated forms of their letters, e.g. in Devenagari or Myanmarese). The result is then incorrect.

The mandatory features should be obeyed, even if we can safely ignore optional features such as variant forms, decorated forms (unless they are usually used in autonyms, but anyway even in HTML these optional features are not rendered unless you activate them specifically with specific CSS styles).

  • see the OpenType specs about which opentype feature is mandatory for each documented script; note that some scripts are still not documented, OpenType specifications are still a work in progress, even for scripts which are already encoded in Unicode, notably for the most recently added ones. For example the Myanmarese and Tibetan scripts require some mandatory features still not documented, but that have been tried by various font vendors with more or less success. Only the Arabic script is very precisely documented, not just informatively in OpenType, but normatively in the Unicode specifications for its contextual joining types, which become madatory features in the OpenType specs, and similar features in other font technologies like AAT, Graphite, or SVG fonts. --

Thanks for pointing where the fonts are developed.

I note that your Github repository contains a "languages.txt" file that you maintain yourself manually, when it should be generated from the wiki, using a Lua or PHP script enumerating all supported language codes (from the Site matrix, or from languages.php) and generating the list using te equivalent of {{#language:code}} in wiki syntax. This list may evolve over time when the Language Committee will add new wikis, or when it will decide to change the preferred language names.

Your list is completely unordered, and as it is long, it will be hard to see if it does not contain duplicates, or alternate names no longer supported, or typos. I suggest you sort this list by importing it in an Excel column to sort it in CLDR order (make sure you've selected the English language in your Excel document so that its CLDR collation tailoring will be correct and stable).

The other option is to prefix each line in languages.txt by the language code and a TAB or SPACE before the autonym, allowing to sort the file simply by language code. These language codes (using lowercase Badic latin letters a-z and the hyphen-minus) and TAB or SPACE will not pollute the list of characters to include into the Autonym font which already need these characters for the autonyms themselves (the only addition would be the TAB or SPACE separator and the newline characters which are already mapped in the font, so you don't need specific character filters for these, when your script will generate the list of Unicode characters to map in the generated font).

For stability of the font (if there are several deployed versions of MediaWiki with different lists of languages supported by #language), you may want to add an additional "mul" language for adding additional characters to keep, or use a versioned line (such as "kk-x-1.2" containing the language autonym for Kazakh in version 1.2), so that the Autonym font for ULS will work on all wikis with different versions of their language support.

Note also that these generated language names may contain RLM/LRM marks (which you don't need to map to glyphs in the geenrated fonts, are they are to be supported only by the text renderers themselves; but if you map glyphs for these controls, they should be completely empty with a zero-width advance, or should be mapped to symbols for visible controls, but this is not a good thing to do for this Autonym font which is not intended to be used in text editors with a "visible controls" mode).

  • Bug 57999 has been marked as a duplicate of this bug. ***

ULS has been disabled on all wikis except Wikidata (bug 46306 + https://gerrit.wikimedia.org/r/108735), so this bug is not relevant for WMF wikis right now. Considering we also have two duplicate reports on GitHub I think it's clearer to close this one.

General discussion continues at bug 56292, bug 60304 tracks the revert of bug 46306, the future is unknown but should be bug 56433 comment 22.

This still applies if somebody chooses to enable ULS. I suggest reopening.

(In reply to comment #79)

This still applies if somebody chooses to enable ULS. I suggest reopening.

Indeed we don't know what was the rationale for bug 46306, but I suppose it was that one will just disable ULS if one doesn't see a benefit. So if the Autonym font gives you problems you won't enable it. Unregistered users can't enable preferences, so the "if somebody chooses to enable ULS" scenario doesn't apply to them.
So, yes, technically the issue still exists but not in default code.

Now that ULS is disabled by default the usage of webfonts is totally broken: https://de.wikipedia.org/wiki/Benutzer:Raymond/Webfonts :-(

I also agree that forcing ULS completely off is a bad situation. At least it should remain as an optional gadget in user preferences, possibly in the Beta preferences.

I really love the possibility of switching user langguage of the interface in one click, notably for testing translatable pages, usng the ULS icon at top of pages, even if I did not like much the ugly look of the Autonym font

Nothe that the Autonym font lacks is still lacking a BOLD version, for example in language navboxs displayinf the current language in Bold black instead of a normal link, or usable with the LangSelect template or the "multilingual" on Commonns, where all language autonyms are displayed. If the "autonym" class is present, and Autonym webfont downloaded, the browser will use synthetic bold which is almost unreadable in sme scripts like Hebrew.

This bug cannot be marked "fixed" but should become a tracker, expecting for more work to be done. IT should remain open or at least replaced by a new tracking bug collecting the efforts in ULS and all the various things to do to improve it.

So please reenable ULS at least in Beta preferences, and split ULS into several separate gadgets (Webfonts alone independantly of languages, IME, script variant selector for Chinese with automated transliterators, per-languuage font preferences with or without webfonts, and UI language selector) even if there remains a central dialog to setup all these components.

(In reply to comment #82)

I also agree that forcing ULS completely off is a bad situation. At least it
should remain as an optional gadget in user preferences, possibly in the Beta
preferences...

You can still enable it in the user preferences > User profile > Internationalisation.

I'd also reopen this bug. Only because ULS can be switched off now does not mean it will not be used by people who find it useful. Additionally I could personally imagine the default state of the newly created option to be switched to "enabled" in future (therefore enabling ULS by default unless the user opts out).

The bad font rendering was one of the worst annoyances around ULS (and probably the most visible one). Especially since it basically broke something that worked without problems before.

I'd keep this bug open to track the progress of the "upstream" bug in Git.

(In reply to comment #78)

ULS has been disabled on all wikis except Wikidata (bug 46306 +
https://gerrit.wikimedia.org/r/108735), so this bug is not relevant for WMF
wikis right now. Considering we also have two duplicate reports on GitHub I
think it's clearer to close this one.

This isn't in keeping with standard practice. We usually tag the local bug as upstream and leave the local bug open in order to make discovery easier.

Thank you for adding the GitHub links to the "see also" section.

With the tofu detection that recently added into ULS, users no longer see degradation of text quality for fonts they have. I think that addresses the main concern of this bug report.

For the fonts users don't have, the issue is still present. However, going from completely unreadable tofu to more readable autonym font is an improvement in my opinion.

(In reply to comment #82)

This bug cannot be marked "fixed" but should become a tracker, [...]

But the tracked bugs are on another bug tracker; that's the difficulty.

(In reply to comment #86)

With the tofu detection that recently added into ULS, users no longer see
degradation of text quality for fonts they have. I think that addresses the
main concern of this bug report.

That was https://gerrit.wikimedia.org/r/#/c/108024/

Given https://gerrit.wikimedia.org/r/#/c/108735/ I think the change can currently be tested (only) on test.wikipedia.org. The main page may suffice: https://test.wikipedia.org/wiki/Main_Page

(In reply to comment #87)

(In reply to comment #86)

With the tofu detection that recently added into ULS, users no longer see
degradation of text quality for fonts they have. I think that addresses the
main concern of this bug report.

That was https://gerrit.wikimedia.org/r/#/c/108024/

Given https://gerrit.wikimedia.org/r/#/c/108735/ I think the change can
currently be tested (only) on test.wikipedia.org. The main page may suffice:
https://test.wikipedia.org/wiki/Main_Page

This has not fixed the issue for me; the list of languages still looks just like in the screenshots provided here.

(In reply to comment #87)

(In reply to comment #86)

With the tofu detection that recently added into ULS, users no longer see
degradation of text quality for fonts they have. I think that addresses the
main concern of this bug report.

That was https://gerrit.wikimedia.org/r/#/c/108024/

Given https://gerrit.wikimedia.org/r/#/c/108735/ I think the change can
currently be tested (only) on test.wikipedia.org. The main page may suffice:
https://test.wikipedia.org/wiki/Main_Page

So, the fix is apparently not deployed on testwiki, but it *is* deployed on beta labs: http://en.wikipedia.beta.wmflabs.org/ (after you register and enable ULS). And it does seem to help there, most language names are displayed in normal fonts for me; the list still looks pretty silly with some language names being pretty and antialiased and some not, but it's definitely better.