Page MenuHomePhabricator

git-deploy should allow separate restarts
Closed, ResolvedPublic

Description

Currently, git-deploy automatically restarts the service after every deployment. For Parsoid, this is kind of evil, for two reasons.

First, we have two repositories, one for the code and one for the config. If we make dependent changes to these, deploying one of them will break the service, and deploying the second one will fix it again. When doing a deployment like this, we'd want to deploy both, then restart the service.

Second, even when only deploying one repo, it's generally desirable to test the change on one backend first. So it would be nice if we could deploy the change, restart one backend, sanity-test it, and then restart the rest of the cluster if it's fine. With the way things work currently, deploying broken code will instantly break the entire cluster.

So I'd like git-deploy to have a restart operation that restarts a service, either on the entire cluster or on one specific minion (this is needed to restart one backend for testing, because deployers don't necessarily have root), and for the restart-on-deploy behavior to be disableable.


Version: wmf-deployment
Severity: normal

Details

Reference
bz47393

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 22 2014, 1:18 AM
bzimport added a project: Deployments.
bzimport set Reference to bz47393.

There's now a service-restart command that can be run for this. The command batch runs a restart across the minions, so that not every service restarts at the same time. It also gives reporting back on whether or not the restart was successful.