Timo, I have unpooled integration-slave1009 since it is missing hhvm and left a note pointing to this bug. https://integration.wikimedia.org/ci/computer/integration-slave1009/

(In reply to Antoine "hashar" Musso (WMF) from comment #1)

Timo, I have unpooled integration-slave1009 since it is missing hhvm and
left a note pointing to this bug.
https://integration.wikimedia.org/ci/computer/integration-slave1009/

I'm not sure that's needed.

While integration-slave1006, 1007 and 1008 do have hhvm installed. None of them run hhvm under the 'php' command. And we don't use hhvm anywhere yet in Jenkins.

Jobs using PHP are assigned to Precise slaves, not Trusty. Because that comes with PHP 5.4/ PHP 5.5 which we don't want to use. Once we have hvvm installed and php aliased to hhvm, we can start creating hhvm jobs but afaik integration-slave1009 is equipped to be pooled. The others having hhvm is a coincidence at this point, it's not part of the manifest yet afaik.

Moved to SRE , need to figure out an entry point in puppet manifests to easily install hhvm on the contint slaves.

I have poked the ops list to get help on installing hhvm and properly configure it on Jenkins slaves.

On top of my head and from a thread I posted on the internal ops list. That is a bit messy but I think it covers the needs.

We want to use HHVM for the PHPUnit jobs and the qunit.

The Zend PHP jobs are tied to Precise instances which comes with PHP 5.3.x. HHVM PHP jobs need to be tied to Trusty instances.

The HHVM package can be installed via the puppet class ::hhvm. The settings there need to be investigated and tweaked for the CI context.

@hashar wrote:
We have a very [lame script] to run PHPUnit which invokes:

hhvm -vEval.Jit=1 \

-vEval.PerfPidMap=false \
-vDebug.CoreDumpReportDirectory="$LOG_DIR" \
-vRepo.Central.Path="$WORKSPACE/hhvm.hhbc.sqlite" \
--php phpunit.php \
--with-phpunitdir "$PHPUNIT_DIR" \
--conf "$MW_INSTALL_PATH/LocalSettings.php" \
--log-junit $JUNIT_DEST

[lame script]
http://git.wikimedia.org/blob/integration%2Fjenkins/master/bin%2Fmw-run-phpunit-hhvm.sh#L42

A few more questions I had:

• can we override HHVM parameters via env variable? Repo.Central.Path comes to mind since we don't want different builds/code to use the same repo.

Ori replied:

If you leave the repo path ini setting unset, HHVM will look for HHVM_REPO_CENTRAL_PATH in the environment. Doing it via the command-line (as you did previously) is fine too.

Giuseppe recommended to use a shell wrapper around HHVM to define the settings.

• I had to set Eval.Jit=1, otherwise the PHPUnit tests were damn too

slow. The cli defaults in puppet have it false though.

Ori replied:

The JIT is not beneficial for short-running invocations. Since the unit tests take a while to run, it's not surprising that they benefit from the JIT. It's fine to have it on.

• The qunit tests are running using apache mod_php5 (we include the

mediawiki class which includes apache::mod::php5) and points to a vhost
localhost:9413. Is there a straight forward way to add the fastcgi
frontend for hhvm?

Ori replied:

Yep. Just include ::apache::mod::proxy_fcgi. A minimal (but functional) Apache 2.4 vhost for MediaWiki on HHVM looks like this: https://gist.github.com/atdt/8dcd1ebcb442efd2585f

We will have to figure out how to vary settings on a per build basis, since the various HHVM caches should probably not be shared. The Repo.Central.Path parameter can be overridden with env variable HHVM_REPO_CENTRAL_PATH.

On Ubuntu /usr/bin/php is managed by the Debian alternatives system (Zend being /usr/bin/php5 and HHVM /usr/bin/hhvm). The Jenkins jobs just use 'php' but I noticed the alternative can be borked and point to either php5 or hhvm :/

We will probably want a php shell script wrapper inserted in the PATH environment, this way we can have Jenkins switch easily between the flavors and still reference 'php'. Something like:

#!/bin/bash -e
HHVM_OPTS="foo"

case $PHP_FLAVOR in
    ( 'hhvm' ) exec /usr/bin/hhvm "$HHVM_OPTS" "$@" ;;
    ( 'zend' ) exec /usr/bin/php5 "$@" ;;
    ( * ) 
        echo "\$PHP_FLAVOR unrecognized ( hhvm | zend )"
        exit 1 ;;
esac

Finally have to figure out the Jenkins Job Builder changes to generate jobs for both flavors and without copy pasting which is exactly what it is meant for. Each job templates would just set a PHP_FLAVOR parameter which is populated in the build environment and then used by the php wrapper.

In T75356#834842, @hashar wrote:

We will probably want a php shell script wrapper inserted in the PATH environment, this way we can have Jenkins switch easily between the flavors and still reference 'php'. Something like:

#!/bin/bash -e
HHVM_OPTS="foo"

case $PHP_FLAVOR in
    ( 'hhvm' ) exec /usr/bin/hhvm "$HHVM_OPTS" "$@" ;;
    ( 'zend' ) exec /usr/bin/php5 "$@" ;;
    ( * ) 
        echo "\$PHP_FLAVOR unrecognized ( hhvm | zend )"
        exit 1 ;;
esac

If you want to go this route, use https://github.com/CHH/phpenv just like Travis-CI does. It's a variant of rbenv that setup up a virtual tree for switching PHP interpreter versions from the CLI or via a magic .php-version file in a directory. I use it on my laptop and in several testing VMs to make testing things in multiple PHP interpreters easy. In a jenkins job you can then parametrize the PHP version and switch versions easily.

In T75356#834842, @hashar wrote:

@hashar wrote:
We have a very [lame script] to run PHPUnit which invokes:

hhvm -vEval.Jit=1 \

-vEval.PerfPidMap=false \
-vDebug.CoreDumpReportDirectory="$LOG_DIR" \
-vRepo.Central.Path="$WORKSPACE/hhvm.hhbc.sqlite" \
--php phpunit.php \
--with-phpunitdir "$PHPUNIT_DIR" \
--conf "$MW_INSTALL_PATH/LocalSettings.php" \
--log-junit $JUNIT_DEST

[lame script]
http://git.wikimedia.org/blob/integration%2Fjenkins/master/bin%2Fmw-run-phpunit-hhvm.sh#L42

This looks about right to me honestly. You may find a few more -vWHATEVER flags you want to apply but it should work fine. What's the major problem with this approach?

The easiest route to HHVM testing would seem to me to be to spin up Ubuntu 14.04 slaves, install HHVM and run tests there via Jenkins tags that pin the HHVM jobs to 14.04 slaves. I can see the utility in a nicer multiple PHP interpreter setup as well, but it seems like a big bite to take in order to accomplish the near term goal.

Most of the puppet config for HHVM in operations/puppet is runtime configuration that can be handled by the tests themselves if you use a wrapper script like the one you showed. If you don't use a wrapper script and expect php foo.php to run HHVM on some servers and PHP 5.x on others you will still need to pin jobs to particular OS versions on the slaves or delve into custom PHP builds to get older PHP versions tested on newer slaves.

100% of the WMF app and api servers are now running HHVM. I imagine that the production jobrunners and imagescalers will follow sooner rather than later. All of beta is running HHVM. We should be treating HHVM as a first class citizen for testing.

In T75356#834989, @bd808 wrote:

The easiest route to HHVM testing would seem to me to be to spin up Ubuntu 14.04 slaves, install HHVM and run tests there via Jenkins tags that pin the HHVM jobs to 14.04 slaves. I can see the utility in a nicer multiple PHP interpreter setup as well, but it seems like a big bite to take in order to accomplish the near term goal.

php points to the Zend PHP by default, but we can set the Debian alternative to hhvm on CI slaves. We have a bunch of scripts invoking php directly (ex: shell shebangs, php maintenance update.php), so we would need sane defaults settings in the HHVM configuration files and settings that needs to vary on a per build basis to be passed as env variables (ex: HHVM_REPO_CENTRAL_PATH="$WORKSPACE/hhvm.hhbc.sqlite")

Most of the puppet config for HHVM in operations/puppet is runtime configuration that can be handled by the tests themselves if you use a wrapper script like the one you showed. If you don't use a wrapper script and expect php foo.php to run HHVM on some servers and PHP 5.x on others you will still need to pin jobs to particular OS versions on the slaves or delve into custom PHP builds to get older PHP versions tested on newer slaves.

We pin jobs already:

• Zend PHP 5.3.x to Precise (since Trusty has 5.5 and we want to make sure we are 5.3 compatible)
• HHVM jobs are pined to Trusty since Precise does not have it. Then Trusty defaults php to Zend PHP.

100% of the WMF app and api servers are now running HHVM. I imagine that the production jobrunners and imagescalers will follow sooner rather than later. All of beta is running HHVM. We should be treating HHVM as a first class citizen for testing.

Yeah finally. That has been acknowledged by Greg and raised during the RelEng weekly checkin. Now a high priority.

Change 178806 had a related patch set uploaded (by Hashar):
contint: provision hhvm on CI slaves

https://gerrit.wikimedia.org/r/178806

Patch-For-Review

I have applied the puppet patch https://gerrit.wikimedia.org/r/178806 on the continuous integration puppetmaster, so we have hhvm installed and more or less setup for cgi scripts. Still have to figure out the proper configuration for fcgi runs (i.e. for the qunit jobs).

Change 178806 merged by Faidon Liambotis:
contint: provision hhvm on CI slaves

https://gerrit.wikimedia.org/r/178806