Openstack is already doing this. There's probably a plugin for Zuul or Jenkins that can do this.
When we activate it we will need to make sure we update the links jenkins-bot outputs from Zuul to Gerrit to point to these static files instead of the urls in Jenkins.
And when update the Jenkins configuration to store no or a very limited build history.
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T216085 Release Engineering Data Collection and Retention (aka Data³) | |||
| Declined | None | T53447 Store CI builds output outside Jenkins (e.g. static storage) | |||
| Declined | None | T114998 Provide Swift object store(s) for the labs projects | |||
| Declined | None | T78705 Send Jenkins build log and results to ElasticSearch | |||
| Declined | None | T211904 Set up data storage to collect loosely structured data from CI |
Change 75632 had a related patch set uploaded by Hashar:
contint: python dependency for publish-console.py
Change 74424 had a related patch set uploaded by Hashar:
experimental HTML console publishing
I have sprinted a python script to let us fetch the HTML console and publish it under /srv/org/wikimedia/integration/logs
The scheme is looking like:
/12/74312,5/mediawiki-core-regression-phpcs-HEAD-1447/console.html
12 is the last two bytes of the Gerrit change number
URL vary by patchset number with a comma, might want a sub directory though to make it easier to browse all patchsets of a change.
The directory can later host debug log files and tests results. We can do that with Jenkins artifact publishing.
I compressed the old console log via gzip (Jenkins supports reading compressed logs). That give out some space.
I am busy with release/QA things right now, so unassigning bug from myself.
Change 74424 abandoned by Hashar:
experimental HTML console publishing
Reason:
no use for now.
Zuul upstream now supports passing Swift related informations to the jobs: http://ci.openstack.org/zuul/zuul.html#swift
Needs a wrapper / post build task in the jobs though.
OpenStack does do this, but they're working on phasing out Jenkins entirely in favour of letting Zuul communicate with Gearman directly.
It may be interesting to wait until they've finished that before we invest in building a solution to store Jenkins output, integrated with JJB etc.
Potentially, we can already write a wrapper that would send the log to a central storage. OpenStack uses swift with the settings being held directly in Zuul configuration ( http://ci.openstack.org/zuul/zuul.html#swift ).
They have a script to fetch the Jenkins job console output. There is then two macros in their JJB configuration: zuul-swift-upload-console-log and zuul-swift-upload-logs-with-console which are run in a postbuildscripts regardless of the build status. Ultimately that invokes zuul_swift_upload.py which crafts a basic web layout and push the changes to swift based on SWIFT_* env variable that have been passed by Zuul to the Jenkins job.
Zuul has support to reach a Swift cluster and establish everything that is needed to then be able to push files. We would need a container in Swift and write some code to push artifacts there.
To be determined is whether we should reuse the current Swift cluster used for SRE-swift-storage or whether we need a dedicated cluster. After all we don't need much redundancy.
Another way is to just use rsync but we would have to deal with the garbage collection and the trouble of scaling the storage.