We definitely want to use tox :-] That requires some macro definition in Jenkins Job Builder nothing to worry about.

One possible trouble is that setup.py will install third party libraries from pip and we do not want to blindly install third party stuff and run their code. Will have to be creative there.

(In reply to comment #1)

One possible trouble is that setup.py will install third party libraries from
pip and we do not want to blindly install third party stuff and run their
code.
Will have to be creative there.

Some ideas:

Access package artifacts between multiple tox-runs

http://tox.readthedocs.org/en/latest/example/general.html#artifacts

Access package artifacts between Jenkins jobs

http://tox.readthedocs.org/en/latest/example/jenkins.html#jenkins-artifact-example
We could run a PyPI clone:

Running our own PyPI

http://tox.readthedocs.org/en/latest/config.html#confval-indexserver
http://www.python.org/dev/peps/pep-0381/
http://wiki.python.org/moin/PyPiImplementations
http://brandonkonkle.com/blog/2010/mar/25/creating-personal-pypi-chishop/

Disabling dependency retrieval and enabling sitepackages instead

http://tox.readthedocs.org/en/latest/config.html#confval-sitepackages=True|False
This would allow the tox environments to use Python modules that are globally installed on the Jenkins host. To add additional modules for testing, you'd have to make sure they're in apt and add them via puppet to the integration server.

A few weeks ago I added tox on the integration slaves running in labs. So one can add a tox.ini file and run whatever tests they want :-]

For tox, one has a bunch of informations at https://www.mediawiki.org/wiki/Continuous_integration/Tutorials/Test_your_python