In T47499#515090, @hashar wrote:

(In reply to comment #13)

I experimented in the past with minijail0, from the Chromium/Chrome project.
It might be worthwhile to experiment some more, it will certainly be much much
faster than anything VM-related.

The documentation is sparse :( I am not sure how it manage to isolate a process, apparently it works using a chroot which I am not sure how we can set it up :(

There's a new kid on the block that seems to do similar things to minijail0 (seccomp etc.) but which seems far easier to use: https://l3net.wordpress.com/projects/firejail/ -- maybe worth a look. I haven't audited from a security perspective, so I'm definitely not recommending it just yet, just throwing it out there in case you want to consider it.

(Just came across this, pasting for record keeping: https://wiki.jenkins-ci.org/display/JENKINS/JClouds+Plugin )

In T47499#937396, @greg wrote:

(Just came across this, pasting for record keeping: https://wiki.jenkins-ci.org/display/JENKINS/JClouds+Plugin )

I am not sure how fast an instance will boot or whether the plugin is able to maintain a pool of VM to consume. Worth looking at though. Filled T85933 to evaluate it.

Originally we were hinting towards using our OpenStack cloud for this. That seems to me still like the most feasible path forwards, because:

• We don't have to ensure that it is secure and isolated sufficiently. Wikimedia Labs is already responsible for that. If other backends are better, they can evaluate that accordingly.
• We know it scales.
• It is already in-place for the most part. (Less set up overhead.)
• We don't require operations resources we don't have to maintain it. If we were to use some other system, it will likely end up being created by someone who's allocating his/her resources on-off and isn't able to maintain it. Wikimedia Labs has maintainers (not many, but it has infrastructure and will naturally grow as needed).

As for the infrastructure for our pool of slaves specifically, that isn't part of plain OpenStack Cloud by default, however OpenStack maintains a package called nodepool that does this. And they're dedicated to maintaining it as they use it themselves for their Jenkins jobs.

More information: http://ci.openstack.org/nodepool/

Conversation with maintainers about how it works:
https://gist.githubusercontent.com/Krinkle/5aad511288e1515d01c7/raw

See also the draft RFC: https://docs.google.com/a/wikimedia.org/document/d/17g8cU2qkv256X5cb_TLfqubwMMgCIAxvJdoOwA5335g/edit (I *think* that's the latest version)

Yup definitely heading toward reproducing the OpenStack CI infrastructure which is supported/developped by more than a handful of person.

I found out today that the blocking tasks are rather messy, I talked to Greg about it and I would rewrite / split some of them.