Page MenuHomePhabricator

Set up automatic builds for extensions ported to HHVM
Closed, ResolvedPublic

Description

http://hhvm-build.wmflabs.org/ is great. In addition to building HHVM, it should also build extensions ported to HHVM. There are instructions for building extension DSOs here: https://github.com/facebook/hhvm/wiki/Extension-API#dynamically-loadable-objects.


Version: unspecified
Severity: enhancement

Details

Reference
bz63120

Event Timeline

bzimport raised the priority of this task from to High.Nov 22 2014, 3:04 AM
bzimport set Reference to bz63120.

Moving under continuous integration

I don't mind helping with the Jenkins integration. May I get:

  • the git repo used for hhvm
  • the git repos of each extensions
  • how to build each extensions (hopefully they all use the exact same sequence)

From there we can get an integration job that will build hhvm and attempt to build each extensions whenever a change appear in hhvm or one of the ext.

The integration job will need to run on a Trusty-based host.

To build:

  • apt-get install apt-get install autoconf automake binutils-dev build-essential cmake g++ git \ libboost-dev libboost-filesystem-dev libboost-program-options-dev libboost-regex-dev \ libboost-system-dev libboost-thread-dev libbz2-dev libc-client-dev libldap2-dev \ libc-client2007e-dev libcap-dev libcurl4-openssl-dev libdwarf-dev libelf-dev \ libexpat-dev libgd2-xpm-dev libgoogle-glog-dev libgoogle-perftools-dev libicu-dev \ libjemalloc-dev libmcrypt-dev libmemcached-dev libmysqlclient-dev libncurses-dev \ libonig-dev libpcre3-dev libreadline-dev libtbb-dev libtool libxml2-dev zlib1g-dev \ libevent-dev libmagickwand-dev libinotifytools0-dev libiconv-hook-dev libedit-dev \ libiberty-dev libxslt1-dev ocaml-native-compilers \ php5-imagick libyaml-dev hhvm hhvm-dev liblua5.1-dev
  • cd path_to_extension_checkout
  • hphpize
  • cmake .
  • make

The three extensions that we should do this for are:

Blocked by Bug 68256 - Jenkins: Set up integration slaves with Ubuntu Trusty

Ori, can you craft a puppet class that provides all the packages?

You can add them to modules/contint/manifests/packages.pp , the class is included on all Jenkins slaves.

Regarding compilation, how do you get hhvm installed? Is that as simple as installing the hhvm debian package? Can be added to modules/contint/manifests/package.pp as well though it will need a harness for Trusty.

The sequence is quite easy to reproduce in Jenkins Job Builder. We would need a new job template like '{name}-hhvm-build'.

I am wondering whether there are any tests suite included to ensure the result is usable. I guess we would want to run MediaWiki core test suite under HHVM with the extensions.

Also apt.wikimedia.org is missing the hhvm-dev which I believe provide hphpize.

Should be added to modules/contint/manifests/packages.pp as well.

(In reply to Antoine "hashar" Musso from comment #6)

Also apt.wikimedia.org is missing the hhvm-dev which I believe provide
hphpize.

Should be added to modules/contint/manifests/packages.pp as well.

The Jenkins slaves in labs are currently running Ubuntu Precise. That is why the packages are not available.

Change 150813 had a related patch set uploaded by Hashar:
hhvm: create module + list all dev dependencies

https://gerrit.wikimedia.org/r/150813

Change 150820 had a related patch set uploaded by Hashar:
Build PHP extension with HHVM

https://gerrit.wikimedia.org/r/150820

Change 150822 had a related patch set uploaded by Hashar:
Build PHP extension with HHVM (non voting)

https://gerrit.wikimedia.org/r/150822

Change 150822 merged by jenkins-bot:
Build PHP extension with HHVM (non voting)

https://gerrit.wikimedia.org/r/150822

A first build: https://integration.wikimedia.org/ci/job/php-FastStringSearch-hhvm-build/1/console

00:00:05.874 cc1plus: error: /root/hhvm/joe/hhvm: Permission denied

:-D

Ori quoted a discussion from Giuseppe:

hphpize adds to the list of paths it configures the $PWD at the time of building
so when people try to use hphpize from the package I built, cmake complains it
cannot find /root/joe/hhvm

Sounds like an upstream issue.

Forked the issue to Bug 68944 - hphpize inject $PWD used at time of building to HHVM_INCLUDE_DIRS

Change 150820 merged by jenkins-bot:
Build PHP extension with HHVM

https://gerrit.wikimedia.org/r/150820

Change 151034 had a related patch set uploaded by Hashar:
Make hhvm-build jobs voting

https://gerrit.wikimedia.org/r/151034

Change 151034 merged by jenkins-bot:
Make hhvm-build jobs voting

https://gerrit.wikimedia.org/r/151034

The patches above create the Jenkins jobs + Zuul configuration. They are running and voting.

Of the two blockers:

Bug 68944 - hphpize inject $PWD used at time of building to HHVM_INCLUDE_DIRS

worked around using sed

Bug 68256 - Jenkins: Set up integration slaves with Ubuntu Trusty

it is not complete yet, but in a state that has hhvm and hhvm-dev deployed

how come this is resolved but there is still this unmerged?

https://gerrit.wikimedia.org/r/#/c/150813/4

is it unneeded or maybe the bug not really resolved yet?

Change https://gerrit.wikimedia.org/r/#/c/150813/4 installs HHVM development packages which are needed to compile extensions.

One of the first patchset has been installed on the contint puppetmaster in labs which thus provided the package and solve this bug.

The change is pending review, I have amended on Giuseppe suggestion to automatically determine the list of package to install instead of having to manually maintain a list of them in a puppet manifest (which is what is currently deployed).

I have marked the bug fixed to avoid cluttering my bug lists since the issue is effectively fixed.

Change 150813 merged by Giuseppe Lavagetto:
hhvm: create module + list all dev dependencies

https://gerrit.wikimedia.org/r/150813

The hhvm build deps have some conflict/cycle with libjpeg62 and libjpeg8 which do not play well together. Filled as https://bugzilla.wikimedia.org/show_bug.cgi?id=71413