Jenkins should set $wgDevelopmentWarnings = true, and keep a count of E_USER_DEPRECATED errors in the PHP error log that are generated by the test suite. If a patch causes that number to go up, Jenkins should -1 it.
Version: wmf-deployment
Severity: enhancement