Page MenuHomePhabricator

Allow jvm non-cgi webapps
Closed, ResolvedPublic

Description

From http://webchat.freenode.net/?channels=#wikimedia-labs
[12:17] <Ilya_> I want to run Scala on PlayFramework (JVM/Netty based). I think toollabs does not allow any open port to serve http from application.
[12:27] <YuviPanda> so no, you can't really run Netty on labs
[12:27] <YuviPanda> well
[12:27] <YuviPanda> on tool labs
[12:27] <YuviPanda> yet
[12:28] <Ilya_> yet? are there any plans to allow it?
[12:29] <YuviPanda> Ilya_: it's not a policy issue, but just one of 'we haven't had anyone work on it yet'
[12:29] <YuviPanda> so if you can figure a way out to make netty play nice with apache httpd, it shouldn't be *that* hard to get it running
[12:40] <Ilya_> it is easy? (don't know about toollabs specifics) to configure httpd to be proxy to other port https://community.jboss.org/wiki/UsingApacheHTTPDAsReverseProxyAndLoadBalancerForCustomNettyHTTPEngine?_sscc=t
[13:05] <YuviPanda> Ilya_: file a bug?


Version: unspecified
Severity: enhancement

Details

Reference
bz50453

Event Timeline

bzimport raised the priority of this task from to Needs Triage.Nov 22 2014, 1:52 AM
bzimport added a project: Toolforge.
bzimport set Reference to bz50453.

This should be possible now if you don't mind getting your hands dirty and don't overload the instance :-). Cc'ing Coren to give his administrative okay; the rest are directions for green lights:

  1. Log into tools-webgrid-01.pmtpa.wmflabs and take a look at /usr/local/bin/tool-lighttpd (that is called by "webservice start"). It basically starts /usr/local/bin/portgrabber with "$tool" (the name of the tool a webserver should be started for), "/usr/local/bin/lighttpd-starter" (the program to start) and "$tool" (again; this time arguments for lighttpd-starter).
  2. portgrabber takes to the portgranter on the local host, sends it "$tool" and gets a free port number in return.
  3. portgranter sets up tools-webproxy to relay requests to "tools.wmflabs.org/$tool/ to a webserver on that port number.
  4. portgrabber then calls lighttpd-starter with the argument "$tool" and the added port number which starts a webserver on that port number.

To braid in Netty there (or any other thing that speaks http) you need to make a wrapper script that takes the port number as its command line argument and starts up a Netty instance on that given port. Then you submit a job to the queue "webgrid" with the command "portgrabber wrapper-script". This will start portgrabber on webgrid, that will start your wrapper script with a free port, and this can then start Netty.

Don't use the normal webgrid for that, but use the tomcat queue. The normal lighttpd queue are /severly/ overcommited under the presumption that nothing but lighttpd runs there.

A (possibly) better model is to look at tomcat-starter and tool-tomcat.

That said, this is now a "normal" scenario and in use by some tools. The exact process is available:

https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Help#Other_web_servers