Page MenuHomePhabricator

Have a check for reported security issues in dependencies
Closed, ResolvedPublic

Description

As we start including 3rd party libraries into WMF code, we should have a way to regularly check that our deployed version includes any security fixes.

My initial thought is to have Jenkins use https://security.sensiolabs.org/api to check any composer.lock files that are included in our patches. Their database is open for anyone to contribute to and in the public domain: https://github.com/FriendsOfPHP/security-advisories

A simple job that runs the following will work:

curl -i -H "Accept: text/plain" https://security.sensiolabs.org/check_lock -F lock=@composer.lock -o sensiolabs.check
cat sensiolabs.check && grep -F "X-Alerts: 0" sensiolabs.check

The job should run on patchset proposal, and once a day. Upon status change, an email notification should be sent to security@.

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 22 2014, 3:46 AM
bzimport added a project: Deployments.
bzimport set Reference to bz72193.

I like the principle :-D

The database of CVE is at: https://github.com/sensiolabs/security-advisories which provides a small utility to validate the YAML based format. It is lacking a contributing license though.

I am not a huge fan of depending on a third party API and the above repository is missing the code to validate a composer.lock against the database. But that should be trivial to reimplement.

We would probably want to have a daily run against all repositories maintained branches and produce a report.

So a summary a possible .plan would be:

  • fill an issue to have the repository content under a free license
  • implement an utility that reads a composer.lock and match it against the security repo
  • figure out a database of repos / branch we want to run the utility against
  • Jenkins can daily run it on all repositories as well as on patch proposal (via the existing composer-validate job).

As a simple PoC, I setup

https://integration.wikimedia.org/ci/job/test-csteipp-sensiolabs-securityadvisorieschecker/

Which checks against https://security.sensiolabs.org/api. Now that the database is freely licensed, we can setup our own system to check against. But I'd like to see how stable their api is before taking on building that.

Eventually, I'd like a check like this to run each time the composer.lock file in mediawiki/vendor (or any repo with a composer.lock, but let's start with our vendor repo) is updated. I'm guessing that would be a zuul managed check?

bd808 raised the priority of this task from Medium to High.Dec 24 2014, 6:08 PM
bd808 added a project: Librarization.
bd808 set Security to None.

After https://github.com/FriendsOfPHP/security-advisories/pull/44 was merged, I reran the jenkins build and was notified about the update. Yay!

It looks like SensioLabs updated there DB within 5 minutes of the merge, so timing should be pretty quick on their end. I'd call the test a success.

Should I have Jenkins email security@ instead of just me?

Should I have Jenkins email security@ instead of just me?

+1 to redundancy

Can we get the job defined with Jenkins Job Builder in integration/config.git. Multiple people should be able to help doing the conversion.

As I understand it, the security advisories are in the https://github.com/FriendsOfPHP/security-advisories repository. The Jenkins Git Plugin can be made to poll the repo and trigger a build whenever it find a change. That will nicely automatize the notifications (as I understand it the job is run manually once in a while).

JJB definition for the git plugin is http://ci.openstack.org/jenkins-job-builder/scm.html#scm.git

Can we get the job defined with Jenkins Job Builder in integration/config.git. Multiple people should be able to help doing the conversion.

Is there documentation on how I can do that? Otherwise I'll ambush you while you're in town for a crash course :)

Instead of adding the logic to a Jenkins job (whether via Jenkins web interface or via integration/config.git), I'd recommend adding this test to an individual project.

That way developers can also easily run it locally, and it moves maintenance much lighter by not requiring a change to Jenkins jobs when we change this test.

E.g. Add it to mediawiki/vendor.git as a bash script file and run it from a composer.json:/scripts.security property (or even embed it in that property directly) and run that from a Jenkins job.

Change 192264 had a related patch set uploaded (by Legoktm):
Use sensiolabs/security-checker to check for any reported security issues

https://gerrit.wikimedia.org/r/192264

Patch-For-Review

Change 192265 had a related patch set uploaded (by Legoktm):
Add experimental mediawiki-vendor-composer job

https://gerrit.wikimedia.org/r/192265

Patch-For-Review

Change 192264 abandoned by Legoktm:
Use sensiolabs/security-checker to check for any reported security issues

Reason:
Figured out an easier way to do this.

https://gerrit.wikimedia.org/r/192264

Change 192265 abandoned by Legoktm:
Add experimental mediawiki-vendor-composer job

https://gerrit.wikimedia.org/r/192265

Change 192564 had a related patch set uploaded (by Legoktm):
Add '{name}-composer-security' template and use it for mediawiki/vendor

https://gerrit.wikimedia.org/r/192564

Patch-For-Review

Change 192564 merged by jenkins-bot:
Add '{name}-composer-security' template and use it for mediawiki/vendor

https://gerrit.wikimedia.org/r/192564

https://gerrit.wikimedia.org/r/193057

If there are any other repos committing a composer.lock file, a similar check can easily be set up for them as well.