Page MenuHomePhabricator

[Epic] SVG client side rendering
Open, LowPublic

Assigned To
None
Authored By
bzimport
Oct 4 2005, 12:31 AM
Referenced Files
F2323: mw-svg.patch
Nov 21 2014, 8:50 PM
Tokens
"Like" token, awarded by gymate."Like" token, awarded by 4nn1l2."Like" token, awarded by Habitator_terrae."Love" token, awarded by JoKalliauer."Like" token, awarded by ToBeFree."Like" token, awarded by Liuxinyu970226."Love" token, awarded by MichaelSchoenitzer."Heartbreak" token, awarded by sladen."Love" token, awarded by FoXFTW."Like" token, awarded by He7d3r."Like" token, awarded by fbstj."Love" token, awarded by Perhelion."Like" token, awarded by Ricordisamoa.

Description

:Note non-wmf users of MediaWiki can make use of wgSVGNativeRendering or older installation can use Extension:NativeSvgHandler

This epic is specifically about serving up SVGs that were uploaded to the File namespaces and included into wiki pages etc. These are currently served up as PNG thumbnails derived from the original SVG. However there is a desire to serve up the original SVG images instead.

Several challenges have been identified over the years that would be required to be solved for this to become a reality on WMF services.

  1. Fallback: Not all browsers support SVG, so we will need a fallback (This likely can be striken by now. By the time this gets deployed a fallback is likely no longer needed)
  2. Size: Some SVGs are way too large to serve them up raw. A limit (for instance 100Kb maximum on the original size ) has to be defined as to what is acceptable to serve up to user agents, SVGs above that either need to be simplified or served up as PNG.
  3. Feature: We support translations and these translations can be different than the user-agent language. This functionality should not be broken.
  4. Rendering: SVGs can contain text that is not converted to paths. This text depends on the fonts installed on the user-agent/browser. In order to preserve the intent of the uploader, it might be required to do on the fly conversion to paths or serving up SVGs with text as PNG to ensure that all users get consistent rendering
  5. Software adaptations for the above issues have to be implemented in the MediaWiki SVG handler
  6. Similar adaptations for the above issues might have to be implemented in the WMF Thumbor service which renders the thumbnails for Wikimedia
  7. The changes have to be behind a configuration flag, to assist with gradual rollout
  8. The changes have to be regression tested
  9. The changes have to be announced to the community
  10. The changes have to be rolled out and any feedback needs to be collected and follow up on.

See Also: T10901: [DO NOT USE] SVG rasterisation and management on Wikimedia sites (tracking)

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

If you're intersted in see SVGs here is a script for you.

JoKalliauer claimed this task.

If you're intersted in see SVGs here is a script for you.

Thanks to @Habitator_terrae : there is now an option to allow client side SVG rendering:

Installation

Add following code to https://meta.wikimedia.org/wiki/Special:MyPage/global.js:

//[[:de:Benutzer:Habitator terrae/svgEinbinden]]
mw.loader.load('https://de.wikipedia.org/w/index.php?title=Benutzer:Habitator terrae/svgEinbinden.js&action=raw&ctype=text/javascript');

Finish!

Wasn't this bug about native client-side SVG rendering, without relying on third-party scripts?

@AfroThundr3007730 : The description is "Please add an option to allow client side SVG rendering.", therfore in my opinion this request/task is resolved, but I would agree there should be another new phabricator-Task about "Native client-side SVG rendering, without relying on third-party scripts".

I believe that's what this bug is already doing, judging by the related tasks list, comments, and commits. Changing the title might be more apt than closing (something like "Support native client-side SVG rendering"). Also when considering that its related subtasks are still open, it doesn't make much sense to close at this point.

@AfroThundr3007730 : According to https://meta.wikimedia.org/wiki/Community_Wishlist_Survey_2019/Archive/Option_to_embed_SVGs_as_SVGs supporting native client-side SVG rendering seems to be impossible/stalled within the next years, not even for specific SVGs T208578 .
Anyway if someone whats to render it on client-side (f.e. animated SVGs) it is possible T5593#5234639 .

That's why this is an "Epic" bug (and should track such blocking issues)... nobody said it would be easy (if at all possible) to implement.

Custom user scripts can not be quoted as a "solution" here, as the issue is obviously unfixed in the MediaWiki software itself.

In that case, since this is a long-running epic -- one who's main task has not yet been resolved -- I have reopened it, and also added the other task for tracking.

Seeing https://www.mediawiki.org/wiki/Inline_SVG_use today i was wandering whether there is a phabricator issue for allowing svg direct rendering in Mediawiki core if site administrators choose to allow this. For many intranet sites this would be a very useful feature especially for generated svg that e.g. comes from graphviz or similar tools.

@AfroThundr3007730 : According to https://meta.wikimedia.org/wiki/Community_Wishlist_Survey_2019/Archive/Option_to_embed_SVGs_as_SVGs supporting native client-side SVG rendering seems to be impossible/stalled within the next years, not even for specific SVGs T208578 .
Anyway if someone whats to render it on client-side (f.e. animated SVGs) it is possible T5593#5234639 .

Publishing on WP is publishing for public - not for those installing something special for themselves only!

I suggest general client side SVG rendering for inline SVG in WP and all SVG media smaller than its 500px PNG representation. SVG media should allow links into WP (and interwiki) only. Inline SVG may be limited to templates and links there to WP (and interwiki) only as well.

Advantages:

  • data driven graphics
  • multiple icons from single file
  • graphics transforms (mirrored hieroglyphs for rtl writing e. g.)
  • many many more

I'd really appreciate using a single SVG source file instead of about 100 png icons to write a hiero sentence. Out in the wild most hieros face right. I'd really appreciate a possibility for Wikipedia hieroglyphs to do so as well. Client side svg rendering would do that.

And what about some [[File:ATQuellenNoLnk.svg]]? As you might guess I have an ATQuellen.svg file here having a couple of links pointing to Wikipedia pages. How could I have such links in an svg file uploaded to Wikimedia?

I'd really appreciate using a single SVG source file instead of about 100 png icons to write a hiero sentence. Out in the wild most hieros face right. I'd really appreciate a possibility for Wikipedia hieroglyphs to do so as well. Client side svg rendering would do that.

And what about some [[File:ATQuellenNoLnk.svg]]? As you might guess I have an ATQuellen.svg file here having a couple of links pointing to Wikipedia pages. How could I have such links in an svg file uploaded to Wikimedia?

About hieroglyphs, that's probably T214232: Hieroglyph images should be replaced with SVG versions or unicode font (Noto Sans Egyptian Hieroglyphs)

About links inside SVG images, if SVG are ever embedded as SVG, they won't have hyperlink capabilities, because they'll be served inside <img> tags and not as *inline* SVG.

About hieroglyphs, that's probably T214232: Hieroglyph images should be replaced with SVG versions or unicode font (Noto Sans Egyptian Hieroglyphs)

That's next step after SVG can be handled senibly.

they'll be served inside <img> tags and not as *inline* SVG.

Why? Why not offer both and especially inline SVG for a limited number of paths especially for replacement of annotated images?

SVG inside <img> tags have some restrictions in place that make them safe enough to be enabled on WMF wikis. Inline SVG would expose users to privacy problems or malicious scripts, and I foresee that they will never be allowed.

We don't enter html on Wikipedia. We don't enter img tags there either. Why not integrate a limited subset of SVG tags for inline image capabilities? Why not offer path tags, simple shape tags, use tags, interwiki links and transform tags only? What security flaws could you think of then?

I believe simple Unicode hieroglyphs already display on Windows browsers because Windows has a hieroglyph font.

To make hieroglyphs work in general, WMF just needs to supply the CSS that points to a webfont on a WMF server. (I understand that using Google's CSS is a bad idea because it allows Google tracking. However, I believe WMF may point to a copy of a Google font on a WMF server.)

See https://commons.wikimedia.org/wiki/Help:SVG_guidelines#Background stating

Your browser may or may not have a font for Egyptian hieroglyphs (𓀀𓀐𓀠) or Siddham script (𑖎𑖚𑖕𑖿𑖧𑖲𑖨𑖿𑖎𑖜𑖿𑖧𑖯).

Cartouches and stacking may be a problem, but simple scripts should work. Doing them in Unicode also means that copy/paste work.

Cartouches and stacking may be a problem, but simple scripts should work. Doing them in Unicode also means that copy/paste work.

These simple scripts are the big exception and may be OK for hieratic transcripts. In addition they ignore the fact of default text direction and provide left-to-right hieroglyphic forms only

This comment was removed by Locke_Cole.

Forgive me if I'm posting on the wrong bug, or if this seems naïve, but SVG images (as a src for <img> tags at least) are supported by basically all of the things now (even IE 11 is passable as long as it's not animated):
https://caniuse.com/?search=svg

Perhaps we could add a simple option "none" or "direct" for the $wgSVGConverter variable, as described at Manual:Image administration, which will render uploaded SVG images directly, without needing to install the NativeSvgHandler extension?

Full disclosure: I'm working on T49578 for the Score extension, and it would be much simpler if we didn't have to generate PNG fallbacks for the SVG output! :)

@Jonathanischoice: Thank you. I fully agree. The only problem left seams to be the fear of security flaws that I still don't understand.

Why not translate wikiSVG into SVG like wikitext is translated into html, so both is to be rendered client site?

The only problem left seams to be the fear of security flaws that I still don't understand.

That's not entirely true. There are three main problems:

  1. Security (mostly solved by using <img> instead of direct embeds)
  2. Translation
  3. Fonts (Webfonts are generally unsupported in <img> tags, default font support for many languages is still poor)

There are also differences between browsers on specific element support and spec compliance.

Perhaps we could allow web fonts that point to Commons.

So we might have a font we might use. We might translate some new wikiSVG with a reduced tag set into plain svg just as we translate wikiText into plain html already. So what translation problem might be left?

I guess I'm only talking about the rendering of an uploaded SVG image, that would be used by [[File:some_image.svg | thumb | etc.]] - which is what $wgSVGConverter is dealing with. It would be up to the uploader to ensure they embedded their fonts or converted their glyphs to paths properly. I'm not sure what impact translation would have. Perhaps we do need to split this task into "support native uploaded SVG image rendering without PNG thumbs" which is basically emulating the behaviour of the NativeSvgHandler extension (I haven't looked at the code to see how it works), and keep this bug as "support embedded SVG in the markup"? Seems to me they are two separate problems, with different concerns applying to each.

Is it possible (or desirable) to run SVG files on http://svgcheck.toolforge.org or (once working) http://commons.wikimedia.org/wiki/Commons:Commons_SVG_Checker upon upload to warn the uploader of missing fonts or other potential issues?

This comment was removed by Locke_Cole.

The only problem left seams to be the fear of security flaws that I still don't understand.

That's not entirely true. There are three main problems:

  1. Security (mostly solved by using <img> instead of direct embeds)
  2. Translation
  3. Fonts (Webfonts are generally unsupported in <img> tags, default font support for many languages is still poor)

There are also differences between browsers on specific element support and spec compliance.

Network bandwidth is a significant issue. A couple years ago there was an estimate of average SVG file size that was several hundred thousand bytes. A PNG thumbnail is probably less than 40 kB.

Security is an issue, but the scripts are blocked on upload and img elements disable scripts. Animations still run, so there is an issue of a small percentage of SVG files inducing seizures in a small percentage of the population. ADA may be implicated.

Translation is a small issue. Most users will not notice. If a French speaker just visits the fr.Wiki, then she will just see the French translations. It only gets weird when she visits the Japanese Wiki and starts seeing illustrations in French or English rather than Japanese. Technically, the issue can be avoided by localizing the SVG for each language wiki -- that's what is done now, but the localization is done just for the PNG and not for the SVG.

I see fonts as an issue for graphics artists rather than WMF. Webfonts are not SVG 1.1, and WMF only claims to support SVG 1.1. Most illustrations do not need webfonts.

Element support is a minor issue. Most modern browsers have better SVG support than librsvg. The textPath element comes to mind -- it is useful for maps, coats of arms, and badges. Modern browsers have some support holes. IIRC, Firefox still does not handle sub- and superscripts. That omission would cause some minor confusion, but users can live with it until Firefox fixes it. The implementation of filter has many inconsistencies, but the filter element is used in a small percentage of files.

Network bandwidth is a significant issue. A couple years ago there was an estimate of average SVG file size that was several hundred thousand bytes. A PNG thumbnail is probably less than 40 kB.

This is especially problematic with maps. For other SVG files, several users e.g. Sarang, Mrmw, TSamuel and cmglee have been actively compressing overly large files.

In T5593#8664865, @Glrx wrote:

Translation is a small issue. Most users will not notice. If a French speaker just visits the fr.Wiki, then she will just see the French translations. It only gets weird when she visits the Japanese Wiki and starts seeing illustrations in French or English rather than Japanese. Technically, the issue can be avoided by localizing the SVG for each language wiki -- that's what is done now, but the localization is done just for the PNG and not for the SVG.

I would not call translation a small issue. Wikimedia sites have better language support than most user agents. Relying solely on browser language selection would prevent translation into all languages.

Network bandwidth is a significant issue. A couple years ago there was an estimate of average SVG file size that was several hundred thousand bytes. A PNG thumbnail is probably less than 40 kB.

This is especially problematic with maps. For other SVG files, several users e.g. Sarang, Mrmw, TSamuel and cmglee have been actively compressing overly large files.

There was a previous discussion about only serving SVG files if the file sizes were the same or smaller than the PNG thumbnails. I believe it was part of a larger discussion about a way to flag SVG files so that they would not be directly embedded, which would be done based on a variety of factors (size, whether they use translation, etc.).

This can be an issue. It needs not. We already ensure that database access doesn't exceed a certain amount. There are technical possibilities to keep huge svg thumbnails from client side rendering. However Only 8 out of 40 svg files I uploded exceeded 10kB. Only one out of 7 raster files I uploaded was smaller than 100kB. In the end it's not about the thumbnails. It's about describing structures and relationships where thumbnails might be about 40 kB but without any value whereat the full scale raster graphics would exceed the size of any sensible vector variant of same quality. So yes we had this argument already and I estimate it rebutted for quite a while.

In T5593#8664865, @Glrx wrote:

Translation is a small issue. Most users will not notice. If a French speaker just visits the fr.Wiki, then she will just see the French translations. It only gets weird when she visits the Japanese Wiki and starts seeing illustrations in French or English rather than Japanese. Technically, the issue can be avoided by localizing the SVG for each language wiki -- that's what is done now, but the localization is done just for the PNG and not for the SVG.

I would not call translation a small issue. Wikimedia sites have better language support than most user agents. Relying solely on browser language selection would prevent translation into all languages.

Oh it's no issue at all! Those svg usually don't come on their own but with a wiki page using them. So the possibility of translation just raises the value of a given svg for all languages implemented. svg files can have a translation. png files can't. This is no issue speaking against svg. So you might use a given svg not for enWiki only but for frWiki and deWiki as well. Still it's the wp author accounting for the usage of a given image in a given wiki page. And it's not the images fault not to fit into swahili wiki.

Serving SVG or PNG should be determined by the SVG file size. What is the best way to handle that choice?

Currently, updating an SVG file or a PNG file does not require rebuilding the wiki pages that use the file. All of those pages use an invariant src attribute.

That seems to be desirable. If an SVG file is used in 500 articles, rebuilding those 500 articles could be a significant time sink. The rebuilds could be lazy, but avoiding the rebuilds seems to be a good way to go. It would also isolate the entire SVG-PNG-choice mechanism to Thumbor.

So Thumbor gets a request for

It parses the request and realizes it is about File:First Ionization Energy.svg (a 35 kB SVG file).

At this point, the Thumbor could build the PNG file and do everything as before. Subsequent requests for the file hit the browser's or WMF's cache.

Alternatively, Thumbor could decide to serve the SVG directly. Even though the path ends in ".png", Thumbor could supply the SVG with the SVG MIME type. I suspect the img element will do the right thing.

Subsequent requests will access the browser's or WMF's cache.

The browser and WMF caches will have multiple copies of the SVG for different src paths (one copy for each pixel width).

That is wasteful, but not necessarily a bad thing. If the SVG < PNG, then it would use less cache space than the current approach.

Should the mechanism try to be better? For example, should Thumbor issue a redirect to

That way, the caches could use the same bits for many different image sizes (rather than caching the same bits for each size).

If a redirect is issued, does that mean WMF must handle twice as many packets? One for the redirect and one for the bits?

If the SVG file is then updated to a 10 MB monster, will the redirect expire and suck down a small PNG, or will the browser follow the old redirect and suck down the monster? I do not know much about cache management.

What is the sensible approach?

In T5593#8664865, @Glrx wrote:

Translation is a small issue. Most users will not notice. If a French speaker just visits the fr.Wiki, then she will just see the French translations. It only gets weird when she visits the Japanese Wiki and starts seeing illustrations in French or English rather than Japanese. Technically, the issue can be avoided by localizing the SVG for each language wiki -- that's what is done now, but the localization is done just for the PNG and not for the SVG.

I would not call translation a small issue. Wikimedia sites have better language support than most user agents. Relying solely on browser language selection would prevent translation into all languages.

Does that mean there are minor language preferences that cannot be set in a browser? If I cannot set a langtag with the user interface, can it be set by editing some system properties?

Does this also come down to one can set some locales but not langtags? WMF likes zh-Hans and zh-Hant, but operating systems probably want zh-CN and zh-TW.

I still see the problem as minor because WMF should just run the SVG through a localizer that uses langtags. That is the only way WMF can keep its current SVG semantics. That can choke on some exotic CSS, but I do not see that as a big issue; current CSS is mundane.

I suppose it'll be all the same if any image is updated by an other one. There are rules the system will follow and these may be different for small and for big files. In either case all cache content needs to be dropped, no matter whether it's a redirect or a set of raster images and no matter what it's going to be replaced by.
And now back to translations:
I don't get the point! An image is included in a Wiki. If it's deWiki the author will expect it to show an image with German content. Now some Chinese try to translate the page. Let's say they see an image with German content. Either they try to adapt the image and ensure to have it be shown in Chinese or they decide to have their own image. Now let's say they see that image with English or French content. Why should they react different? (except gawking why those Germans obviously have non German image content on their page) But once more: This is no issue to deal with especially as there are few images that should be multi language. In most cases to have one file version per content language will be the better choice anyhow.

This ticket was opened in 2005, nearly twenty years ago. I would like to see T49578 Score SVG output on Wikipedia, hopefully some time before the sun runs out of hydrogen and engulfs our planet in a fiery apocalypse. What can I do to help?

This ticket was opened in 2005, nearly twenty years ago. I would like to see T49578 Score SVG output on Wikipedia, hopefully some time before the sun runs out of hydrogen and engulfs our planet in a fiery apocalypse. What can I do to help?

Write code or hire someone to write the code.

I've already done that. How do we get the new version of the Score extension deployed out to Wikipedia so we can enjoy our new SVG output?

Step 4: If fonts are linked (SIL open fonts e.g.) font rendering with not installed fonts can be done without server side processing.

Step 4: If fonts are linked (SIL open fonts e.g.) font rendering with not installed fonts can be done without server side processing.

Can you explain what linked means in this context ?

In T5593#8702404, @Vollbracht wrote:
Step 4: If fonts are linked (SIL open fonts e.g.) font rendering with not installed fonts can be done without server side processing.

Can you explain what linked means in this context ?

in css:
@font-face {
font-family: "Liberation Serif";
font-weight: normal;
src: url(OEBPS/LiberationSerif-Regular-webfont.woff);
}
(Replace OEBPS by some Wikimedia provided web address.)

This comment was removed by Locke_Cole.

Not sure where best to bring this up, but would it be possible for this to play nice with "dark mode"? In other words, for objects (like logos for example) that conceivably have a "reverse" variant, there's a way to either let SVG authors specify both and then have it automatically selected based on some setting (perhaps CSS styling in the SVG)? I guess my concern is that if there's any "optimization" done to an SVG it might strip/trim what appears to be redundant data, and I know some of the related tasks contemplate "optimizing" SVG's prior to sending them.

For what it's worth, the method that seems to be correct is to use @media (prefers-color-scheme: dark) within the SVG. To that end, I've started uploading SVG images that support light/dark mode, and if anyone needs them for testing I've created a tracking category on EN here: https://en.wikipedia.org/wiki/Category:SVG_images_that_support_dynamic_color_schemes.

One step at a time! It's great to have such features I couldn't even think of with pixel graphics only. But it's even much better to have a supported svg subset available for usage in wikis at all. Later on we might support svg files that make use of template styles. That would be great and by the way it would solve your problem as well.

@Vollbracht and @Locke_Cole please keep the ticket on topic, this has nothing to do with the topic of this ticket. All separate issues should be filed as separate tickets. Please find the relevant ticket, or file a new one.