Can be smashed during the Berlin Hackaton 2012.

Not going to happen till I figure out how build a deb package out of the operations/debs/* repos. If anyone has a step by step manual or a shell script, you are welcome to cast your voice on this report.

Also marking this has an enhancement request.

I can probably help with this in the next couple of weeks as I have some free time.

AzaToth mentioned 'jenkins-debian-glue' which is a wrapper to easily integration debian building in Jenkins.

The step by step guide is at http://jenkins-debian-glue.org/getting_started/manual/

A useful FAQ entry: http://jenkins-debian-glue.org/faq/#foreign_arch_builds

I have created the integration-debian-builder.pmtpa.wmflabs instance to be used as a build box with jenkins-deploy as a user and an appropriate sudo policy. The workspace is set to /mnt/build which is vdb and has lot of free space.

The demo job is working properly and managed to generate a debian package for pybal.

Now I need to:

• probably use a different username in labs than jenkins-deploy
• puppetize the installation of a Jenkins slave in labs
• puppetize the basic file layout that is going to be used in labs (/mnt/build and /mnt/repository)
• write YAML definitions in Jenkins Job Builder, making sure to reproduce the various hack I made in the jobs such as overriding the workspace to point to /mnt/build instead of the tiny /home/jenkins-deploy
• generate jobs and try them out
• write Zuul triggers for the repositories. Probably needs to be non voting

Finally, publish the build packages in a reprepro repository http://jenkins-debian-glue.org/faq/#single_repository . Either on gallium or somewhere else.

The testing jobs are:
https://integration.wikimedia.org/ci/job/jenkins-debian-glue-source/
https://integration.wikimedia.org/ci/job/jenkins-debian-glue-binaries/

For lintian, look under sudo part: http://jenkins-debian-glue.org/getting_started/manual/#sudo

I have created RT #5394 to get jenkins-debian-glue package uploaded on apt.wm.o

Change 71245 had a related patch set uploaded by AzaToth:
Add debian-glue macros

https://gerrit.wikimedia.org/r/71245

Change 71245 abandoned by AzaToth:
Add debian-glue macros

https://gerrit.wikimedia.org/r/71245

Change 71245 restored by AzaToth:
Add debian-glue macros

https://gerrit.wikimedia.org/r/71245

Change 84931 had a related patch set uploaded by Hashar:
trigger jenkins-debian-glue jobs for pybal

https://gerrit.wikimedia.org/r/84931

Change 84931 merged by jenkins-bot:
trigger jenkins-debian-glue jobs for pybal

https://gerrit.wikimedia.org/r/84931

Zuul change deployed!

I have uploaded a test change: https://gerrit.wikimedia.org/r/84932

The -source job fails: https://integration.wikimedia.org/ci/job/operations-debs-pybal-source/1/console

My test change is https://gerrit.wikimedia.org/r/#/c/84932/ which is against operations/debs/pybal.git
The related jobs are:
https://integration.wikimedia.org/ci/job/operations-debs-pybal-source/27/console https://integration.wikimedia.org/ci/job/operations-debs-pybal-binaries/7/console https://integration.wikimedia.org/ci/job/operations-debs-pybal-piuparts/7/console
Only the -source job is triggered by Zuul for now, thus only the source result is submitted back in Gerrit. We will want Zuul to triggers source, binaries and piuparts one after the other and tweak the job so they do not trigger each other. That is going to cause some dependencies trouble though, we want to make sure each build uses the proper artifacts and not the one from a job that have been launched in between.
The Jenkins slave is integration-debian-builder.pmtpa.wmflabs and it has a bunch of .deb in its repository:
$ ls -1 /mnt/repository/pool/main/p/pybal/
pybal_1.05+0~20130626213150.10~1.gbpd4b7ea_all.deb
pybal_1.05+0~20130626213150.10~1.gbpd4b7ea.dsc
pybal_1.05+0~20130626213150.10~1.gbpd4b7ea.tar.gz
pybal_1.05+0~20130919220936.27~1.gbp7c3ee2_all.deb
pybal_1.05+0~20130919220936.27~1.gbp7c3ee2.dsc
pybal_1.05+0~20130919220936.27~1.gbp7c3ee2.tar.gz
Congratulations Carl, we are almost there :-] Thank you very much!

Summary from IRC:

If I send 3 changes A,B,C , by the time the 'binaries' job for A is completed, the 'source' job last artifacts would be the one of change C. Hence, when 'binaries' job for B runs, it will end up fetching the artifacts of C.

all jobs are sharing the same build parameter ZUUL_COMMIT that is a merge commit of the patchset against the tip of the branch. Potentially the copy artifact plugin could look up in the other job for the last successful build that has the same ZUUL_COMMIT value.

Change 71245 merged by jenkins-bot:
debian-glue macros and jobs

https://gerrit.wikimedia.org/r/71245

The infrastructure has been setup. The Jenkins job builder template '{name}-debian-glue' can be used to run git buildpackage / cowbuilder.