Page MenuHomePhabricator

Package Parsoid and clean up deploy system
Closed, ResolvedPublic

Description

We'd like to make it easy for both ourselves and third parties to have a properly configured Parsoid install. This should include a proper init script, log rotation etc.

In production, we currently use puppet for config files and git-deploy for pushing out the code and restarting Parsoid (see https://wikitech.wikimedia.org/wiki/Parsoid). Files managed by puppet live in the puppet repository, where operations can review code that will be running with root rights. Our puppetization is not very complete yet, with log rotation for example still missing (which causes parsoid servers to hang after running out of disk space).

There are more moving parts though: We currently manage our dependencies and config in a private config repository which we deploy separately from the main code. For Jenkins testing, we are using another contrib repository which also needs to be updated manually.

It would be good to merge all this in a way that lets us

  • build Debian packages
  • still allows operations to review root-level code
  • supports current deployment methods (git-deploy)
  • minimizes manual dependency tracking and contrib updates

An idea that Ryan brought up is to use submodules for this. The details of how this might look are discussed in https://www.mediawiki.org/wiki/Parsoid/Packaging.


Version: unspecified
Severity: normal
See Also:
https://bugzilla.wikimedia.org/show_bug.cgi?id=61006

Details

Reference
bz53723

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 22 2014, 1:47 AM
bzimport added a project: Parsoid.
bzimport set Reference to bz53723.

physik wrote:

A good proof of concept is to repackage mediawiki-math (http://packages.debian.org/sid/mediawiki-math) for mathml+svg rendering.
In contrast to the old package the new version does no longer depend on latex itself, but on several node modules and phtomjs.
In addition to that a puppet module has to be developed that configures the package according to the wikimedia server configuration.
(https://gerrit.wikimedia.org/r/#/c/90733/6)
If there is a way for an automatic debianization (maybe jenkins could do that) I don't see a particular need for git deploy. Or did I miss something?

Change 110666 had a related patch set uploaded by GWicke:
Parsoid debianization

https://gerrit.wikimedia.org/r/110666

Change 110666 merged by jenkins-bot:
Parsoid debianization

https://gerrit.wikimedia.org/r/110666

Patch was merged a while ago - is there more work left to do here (if yes: please reset the bug report status to NEW or ASSIGNED), or can you close this ticket as RESOLVED FIXED?

The patch is merged, but there are still some things missing:

  • we don't have a longer-term Debian repo yet where we can upload this to; waiting for ops to implement this
  • use debs for deployments in testing and perhaps at some point production
Arlolra set Security to None.
ssastry claimed this task.

Between vagrant and debian repos and some work with docker, I am going to mark this resolved. The broader packaging and 3rd party support discussions are happening elsewhere.