Page MenuHomePhabricator

Support server-side JavaScript
Open, LowestPublicFeature

Assigned To
None
Authored By
Brettz9
Dec 29 2013, 3:46 PM
Referenced Files
None
Tokens
"Orange Medal" token, awarded by Xinbenlv."Like" token, awarded by Ricordisamoa."Love" token, awarded by Esanders.

Description

Given that JavaScript is or has, in essence, the eventual potential to be the world language of web development even on the server side, I think it would be quite nice if the proliferation of Scribunto scripts could take place using a syntax already familiar to web developers and likely to be so far into the future... I would think a safe subset could be employed...


Version: unspecified
Severity: enhancement

Details

Reference
bz59101

Event Timeline

bzimport raised the priority of this task from to Lowest.Nov 22 2014, 2:35 AM
bzimport added a project: Scribunto.
bzimport set Reference to bz59101.
bzimport added a subscriber: Unknown Object (MLST).

JavaScript was considered during the creation of Scribunto, both V8 and Rhino. But Lua turned out to be better suited than either.

The main "pro" for either JS engine was that JavaScript is likely more familiar to our users than Lua. Problems with V8 include:

  • Lack of documentation on embedding.
  • Unclear whether embedding will continue to be supported upstream.
  • Lack of a memory allocation hook, making it difficult to prevent poorly-written or malicious scripts from overrunning memory limits.
  • Huge standalone binary.

Problems with Rhino include:

  • Written in Java.
  • Impossible to embed in PHP, meaning that WMF wikis would have to use slow IPC.
  • Very slow startup. This could be mitigated by running as a daemon, but...
  • Enforcing time and memory limits would require killing the external process, necessitating another slow startup.

Either way, JavaScript didn't seem likely without a lot of effort.

On the other hand, about the only "con" with Lua is that it would be relatively unfamiliar to most of our users. Pros include:

  • Designed for embedding, with detailed documentation.
  • Includes easy hooks to limit memory and CPU time usage.
  • Standalone binary is small and fast. And a JIT is available if more speed were needed.
  • Easy to sandbox, with no odd globals.

If someone were to come along with a well-done patch to integrate a JavaScript engine we'd likely review it when time permitted, but it's unlikely that we're going to encourage anyone to spend time on it or prioritize it in any way. Lua is already here and is already being learned by the community.

I think having to learn Lua is a huge barrier to entry that is preventing wider participation. JavaScript is not just "likely more familiar to our users than Lua", it is, by most measures[1][2], orders of magnitude more popular. By using such an unpopular language we are limiting ourselves to a fraction of potential contributors. I think this runs counter to our mission to "empower a global volunteer community", and I hope this can be made a higher priority in the future.

  1. http://githut.info/
  2. http://langpop.com/

Making it a higher priority is highly unlikely, particularly without any compelling reasoning.

Lua is not a large language and has a style similar to many common languages including JavaScript, meaning it's likely to be easy for JS developers to pick up. Thus your argument, being based solely on the fact that JS is a more popular language for non-embedded uses, isn't very convincing to me.

You've also not addressed any of the drawbacks identified with the available JavaScript implementations.

Esanders: I've actually just done this the other way around - learned JavaScript after knowing Lua. I first learned how to program in MediaWiki templates (ick, I know), then taught myself Lua after it was introduced to Wikimedia wikis two years ago, and I've just learned JavaScript in order to write a user script. Going from templates to Lua was hard, but going from Lua to JavaScript was fairly easy. The hardest thing about JavaScript was figuring out asynchronous code, because there's no need to deal with it in Scribunto modules. I think someone coming from JavaScript to Lua would find it relatively easy. Of course, it always takes effort to learn a new language, but I think that Lua fares pretty well on the learnability scale.

On the other hand, I think the boat has already sailed on the question of which server-side scripting language to use on Wikimedia wikis now that hundreds of Lua modules have been written for them. Anyone wanting to work with templates using JavaScript would also likely have to learn Lua to work out how they all fit together. It's hard enough for people to learn how templates and Lua go together - throwing another language into the mix would only complicate things further. (And then there are the technical issues which Brad brings up.)

I do agree with you in principle, but I think the practicalities make this undesirable.

@cscott mentioned IRL he had some local patches implement JS support so CCing him in...

I think having to learn Lua is a huge barrier to entry that is preventing wider participation. JavaScript is not just "likely more familiar to our users than Lua", it is, by most measures[1][2], orders of magnitude more popular. By using such an unpopular language we are limiting ourselves to a fraction of potential contributors. I think this runs counter to our mission to "empower a global volunteer community", and I hope this can be made a higher priority in the future.

  1. http://githut.info/
  2. http://langpop.com/

It seems the site http://langpop.com/ is compromised?

Xinbenlv raised the priority of this task from Lowest to Needs Triage.EditedMar 25 2019, 2:54 AM

Hello from 2019!

Happy to join a discussion from 6 years ago, I guess many things weren't unclear at the time but might be able to resume discussion nowadays.

First of all, let me give kudos for the team and individuals who created Scribunto, this is certainly a great tool and I don't take it for granted. Kudos and thank you!

Secondly, I'd like to second @Brettz9’ argument:

Choosing Lua over major languages such as Javascript, have the following several disadvantages:

  1. It greatly reduces the amount of potential editor (developer) contributions.
  2. It restricted all the language support by the language community. That's not just about the language usage itself, but also tooling, dependencies, libraries, let alone new language features.
  3. Lua's scripting grammar is not similar to any major widely adopted languages, which add to the barrier for developers This is not a slight point, when designing new languages, language designers take the grammar easy-to-use-ness as an important considerations.

In terms of one of the major argument of V8 unclear to continue support embedding: Given the javascript is so widely used, many other languages and companies are doing their best to support embedding javascript, such as Dart, even if Lua is "born and design for embedding", javascript is and will continue to get way more supported in this side-usage. Speaking of "side-usage", wait, is isn't Javascript born to be "embedded" as a <scripty> bracket of HTML?. V8 is designed to run Javascript in a browser, too. Even if V8's continue commitment is questionable, there are a good amount of alternatives and more coming up: https://blog.sessionstack.com/how-javascript-works-inside-the-v8-engine-5-tips-on-how-to-write-optimized-code-ac089e62b12e

All in all, in my humble opinion, as 6 years have passed, it might be a good time to rethink the templating now.

@Xinbenlv riding a timemachine ~~~~

Anomie triaged this task as Lowest priority.Mar 25 2019, 1:59 PM

I'm very interested in this discussion and understanding the scope of the problem. I'm a front end developer working on a VueJS single page application, but I also very actively use wiki templates, parser function, etc. Recently I was trying to do some string operations and found out about Scribunto, modules, etc. Being a frontend dev, my first thought was why Lua over Javascript.

In 2019 with all the changes since 2013, do we have a new status on Javascript's ability to be correctly and securely implemented in a ScribuntoJS-like manner? I'm very interested in trying this out in my free time, especially with the state of online REPL's like repl.it, CodePen, JSFiddle, etc.

Update: 2019-08-16 C Scot gave a talk in Wikimania2019 and says we will have javascript support in Scribunto engine.
https://upload.wikimedia.org/wikipedia/commons/b/b2/Wikimania_2019_-_Let%27s_Completely_Change_How_Templates_Work%21.pdf

Choosing Lua over major languages such as Javascript, have the following several disadvantages:

  1. It greatly reduces the amount of potential editor (developer) contributions.

It reduces the number of people who already know the language coming in, but potential developers should be able to pick up Lua pretty easily. I know I did.

  1. It restricted all the language support by the language community. That's not just about the language usage itself, but also tooling, dependencies, libraries, let alone new language features.

This point seems to be making some major assumptions that have not born out in practice. For example, libraries would have to be licensed appropriately, and then be somehow copied into the wiki to be used, and would have to not make assumptions about the environment that are not valid inside Scribunto's sandbox. Similar issues apply to tooling and dependencies, and possibly even "language features" if by that you're including standard libraries that may not make sense inside Scribunto's sandbox.

  1. Lua's scripting grammar is not similar to any major widely adopted languages, which add to the barrier for developers This is not a slight point, when designing new languages, language designers take the grammar easy-to-use-ness as an important considerations.

Lua and JavaScript both heavily borrow their syntax and structure from C. There are some differences, but they're not particularly significant. The biggest are (IMO) that Lua uses "do ... end" instead of "{ ... }" and "~=" rather than "!=".

Speaking of "side-usage", wait, is isn't Javascript born to be "embedded" as a <scripty> bracket of HTML?

Not the same kind of embedding we're talking about here. Lua was designed to be a self-contained library embedded in some other program to provide scripting. V8 was designed as a part of a web browser, and at the time it wasn't clear whether use in non-browser contexts was going to continue to be supported.

All in all, in my humble opinion, as 6 years have passed, it might be a good time to rethink the templating now.

On the other hand, the time passed also means we have 6 years worth of existing Lua code now. Adding JS would mean template coders would now have to know three languages, or that all that existing code would have to be replaced.

Update: 2019-08-16 C Scot gave a talk in Wikimania2019 and says we will have javascript support in Scribunto engine.
https://upload.wikimedia.org/wikipedia/commons/b/b2/Wikimania_2019_-_Let%27s_Completely_Change_How_Templates_Work%21.pdf

If he said "we will have JavaScript support", he hasn't discussed it with the person who primarily maintains Scribunto these days (i.e. me). I don't know if he has discussed it with Tim. More likely he was referring to it as a potential idea at a brainstorming stage, as I know he personally has suggested it as such in the past.

A real proposal would have more "pros" than the one in that slide deck and would identify "cons".

Re: existing code being ported, FWIW, I see there is at least one transpiler for Lua that might assist: https://github.com/basicer/lua2js (though this obviously would not be a complete solution).

Aklapper changed the subtype of this task from "Task" to "Feature Request".Feb 4 2022, 11:14 AM
Aklapper removed a subscriber: Mholloway.

Similar issues apply to tooling

There’s tooling that simply doesn’t bother about anything outside of the code and thus could be used more or less out of the box. For example, there’s a de facto standard linter (ESLint) and a de facto standard documentation tool (JSDoc), while I don’t know about any linters or documentation tools for Lua, let alone de facto standard ones. (Lack of documentation is oftentimes a problem when working with metamodules, there is little documentation on what the parameters do, even less on what types they should be of.)

and possibly even "language features" if by that you're including standard libraries that may not make sense inside Scribunto's sandbox.

I don’t know what @Xinbenlv means by language features, but I mean things that are closely related to the syntax. There are several language features introduced in ES2015 or later that don’t exist in Lua (neither in 5.1 – which was also released in 2015 – nor AFAIK in the latest 5.4) that would be helpful in certain wiki scenarios:

  • Class syntax. Lua has metatables and ES5 has prototypes, but creating a class with class syntax (class keyword; fields, methods, constructor, getters and setters, all between a single pair of brackets) makes the code much easier to understand. Most modules are probably fine with functions and built-in data types, but some of the most-used ones (e.g. ScribuntoUnit) do use class-like things.
  • Template strings. Mixing business logic and presentation is not great, but it happens, and including bits of data in the sea of wikitext in that string is hard-to-understand and easy-to-regress in Lua, being able to directly put ${var} where it belongs would be much better.
  • Arrow functions. When functions are used as lambda functions rather than global functions or methods, the JavaScript () => {} syntax avoids much of the unnecessary ceremony, making us focus on the real code. Lambda functions are rarely used in modules, but this is probably in part because they’re hard to read in Lua. If I want to map values in a table to other values in Lua, I’d probably just create a new table, do a for-in loop, and assign values to the new table, while in JavaScript I’d just call .map() on the array to be mapped with an arrow function.

None of these contribute to Turing completeness, but they make programmers’ lives easier.