Page MenuHomePhabricator

Integrate git/Gerrit and Bugzilla
Closed, ResolvedPublic

Description

Author: mediawiki

Description:
Would be nice if the two are integrated together, so that:

  • bug numbers are automagically linked in commit messages;
  • bug status can be automatically determined from commit message (e.g. status 17322: Fixed - wishful thinking :) ).
  • and tons of other bug-tracker/version control integration goodness (http://www.mkgnu.net/scmbug)

I have used integrated SVN/Bugzillas in the past, and it really makes for a far more efficient/traceable development workflow, and implementation and adoption is almost invisible and transparent.

In previous projects, developers just saw properly formed commit messages (and seeing how useful it was) they just started adopting it on their own.

I created this request primarily coz I was used to this integrated SVN/Bugzilla workflow, where even whilst I'm developing code for a new module (like SRF-Ploticus), I would create Bugzilla tickets for TODOs, ENHANCEMENTS and BUGS and assign them to myself. (see bugs 17318, 17319, 17320, 17321).

The Bugzilla tickets then became a traceable log of my development backlog and progress against it, and I used the Summary field to automagically generate the changelog too.

Even with ViewVC, the commit messages are kinda opaque, especially for large commits. But if developers get in the habit of creating tickets even for things they initiate on their own, it really makes for a great way to track the progress of a project.

And the beauty of it too, even for bugs committed by the user community, when you say you fixed a bug, the commit message will have a link to the changeset where the fix was implemented!

Just a suggestion,
Joel


Version: unspecified
Severity: enhancement
See Also:
https://bugzilla.wikimedia.org/show_bug.cgi?id=39399

Details

Reference
bz17322

Event Timeline

bzimport raised the priority of this task from to High.Nov 21 2014, 10:24 PM
bzimport added a project: Gerrit.
bzimport set Reference to bz17322.

Commit messages in CodeReview are linked this way, though not currently showing resolution.

http://www.mediawiki.org/wiki/Special:Code/MediaWiki

mediawiki wrote:

Neat stuff!
I just wish it will change the disposition of the bugs too based on the verb in front of the bug number (e.g. fixes bug 123, closes bug 123, fixes and closes bugs 123, 456, 789, etc.)
We did this in a project with some simple SVN post-commit hooks and it made a HUGE difference in backlog management, traceability, getting devs to really use bugzilla, etc.

http://www.subversionary.org/martintomes/using-a-post-commit-to-update-trac-tickets

But then, that's Trac.

If it's OK, I'll play around with CodeReview and will send a patch and the post-commit hook script if that's OK.

Mmmm, that'd be spiffy. :D

A few things we probably want to happen:

  • Addition of commit message ref'ing a bug to the bug in bugzilla (and mark FIX if appropriate)
  • Probably some automatic tagging within CodeReview based on keywords
  • 'revert' note in commit message to mark the revered revision w/ revered status in CodeReview and reopen bugs that that revision had marked "fixed" in Bugzilla (?)
  • Manual 'revert' on CodeReview to do the bug reopen also?

mediawiki wrote:

K!

I'll take your list and start designing it. I'm hoping we can reuse some of the code we developed last year - it turns out we already did a hook for SVN and Bugzilla and I just forgot about it.

Though I already have a bit of a backlog (which are most MW and SMW related anyways ;) ), I'll try to bump this up since it will really improve the MW dev workflow.

matthew.britton wrote:

stevertigo, is there a particular reason you changed this bug summary from "Integrate SVN and Bugzilla" to "Integrate SVN and MediaWiki"? The latter is not what the comments seem to be about.

stvrtg wrote:

I don't particularly remember doing that, Gurch. I did in fact sort of start writing a bug with that title, but in the course of actually trying to better understand what I was actually talking about, I did some reading into it, got better ideas, and wound up not sending it. Perhaps somehow the title itself was cached and sent upon a refresh or something. That's the only explanation I can think of. Sorry for the confusion.

The completely unrelated idea of an SVN backend comes from my filing of request/bug 18810 (now bug 639), which suggested a colorized edit-diff mode for a current article/page. Anyway that idea itself doesn't seem to make much sense for projects that need less diff-magic and more content storage/access, though more diff-magic is always good.

-Steven

(In reply to comment #6)

I don't particularly remember doing that, Gurch.

stevertigo <stvrtg@gmail.com> changed:

What    |Removed                     |Added

     CC|                            |stvrtg@gmail.com
Summary|Integrate SVN and Bugzilla  |Integrate SVN and Mediawiki

Changing summary back.

I don't know whether the configuration management of WMF wikis falls under SVN as well, but I'd also find it extremely helpful if a bug not only received a "Fixed in r4711." comment, but also that the fix went live on labs (or whereever) and finally on the main farm.

Fedora's Bugzilla has a similar feature where a bug gets comments that an update is available for testing and then for general release, e. g. cf. <URI:https://bugzilla.redhat.com/show_bug.cgi?id=496754>.

If we make use of the feature described in issue 124 (see comment 9), we could easily use patchset-created to ping BZ with a "A change for this bug was pushed for review in gerrit 123 [project: foo; branch; bar]" We could follow up with "Gerrit change 123 was merged on DATE [project: foo; branch: bar]"

Generally, I'm kind of leery about auto-closing an auto-opening bugs. I may commit half a fix for a bug and not want BZ to be closed yet.

(In reply to comment #8)

I don't know whether the configuration management of WMF wikis falls under SVN
as well, but I'd also find it extremely helpful if a bug not only received a
"Fixed in r4711." comment, but also that the fix went live on labs (or
whereever) and finally on the main farm.

The plan right now is to have deployment-labs running master (so it'll get all the latest changes) and deploy master to the cluster every 2 weeks. So I think in the long run this will be a little bit less of an issue. Once we get into the cycle you'll be able to assume that anything over ~2 weeks old has been deployed.

(In reply to comment #10)

[...]
Generally, I'm kind of leery about auto-closing an auto-opening bugs. I may
commit half a fix for a bug and not want BZ to be closed yet.
[...]

I have neither looked at the code I referenced nor worked in a project with auto-close, but I assume it would usually work in the way that the commit message would have to include some (semi-natural) phrase that triggers the close ("^This fixes bug #(\d+)\.", "^Fixes: (\d+)$", etc.) So if a commit doesn't fix a bug, you just don't use the exact phrase but maybe "This partially fixes bug #123"." and everything's as you expected.

Regarding deployment, while I see your point, I think not using this facility would still leave the ambiguity: Has the deployment *really* been done?  In the past, there have been delays when difficulties were encountered or manpower was short.  Another automated comment on a closed bug will cause no harm, but *actively* inform the reporter who cares about the bug and not require *them* to do something.

This of course *could* depend on how the deployment branch is organized, i. e. how a commit that is "reverted" is recognizable to some script so I would consider this part low priority.

Another argument pro auto-closing: Unambiguity. Bug 35964 just got closed as "RESOLVED FIXED" while the commit is still in review. https://bugzilla.wikimedia.org/page.cgi?id=fields.html says for "FIXED": "A fix for this bug is checked into the tree and tested." That's technically obviously true in this case but it doesn't mean that it is resolved in master so that a user can profit from it without too much hassle.

If there's no auto-closing, it's up to some developer to follow the review of his commit and then close the bug which will probably lead many developers to close their bugs prematurely. Auto-closing would enable them to "fix and forget".

Hmm you're reusing an old bug so the first several comments seem unrelated or outdated...

AFAIK Christian is working on this, based on comment 14, hence assigning this to him.

Note: I'm VERY afraid of duplicated work in this area.

Hence I also CC Anja who's been working on WMDE's wikidata bot (see bug 44100 comment 2 for an example),
and Dereckson who's been working on communicating between Gerrit, Bugzilla and IRC according to bug 43655.
It would be great if you could share some implementation details so far to see if some parts are "reusable".

Yeah, we got it partially working :) But it's a bit spammy, so I turned it off until we can fine-tune it a bit further.

As the patch to adjust spammyness is sitting upstream already for a few days without comments from the maintainer, I'll link it here, so we can keep track of it:

https://gerrit-review.googlesource.com/#/c/43280/

(In reply to comment #16)

Example: https://bugzilla.wikimedia.org/show_bug.cgi?id=41889#c6

Cool! The layout needs some work, though :-).

Plugin is live again, and is now only notifying on new patch and merge.

Slightly bikesheddy:

  • I wonder whether it should also set the "patch-in-gerrit" keyword (current situation/workflow that I'm not a fan of), or whether I should pick up the debate again on creating something like a "PATCH_IN_GERRIT" bug status - I consider "getting closer to resolving this report by having a patch in Gerrit" a status but no constant property (=keyword). Related: bug 39399, bug 39402, http://lists.wikimedia.org/pipermail/wikitech-l/2012-December/065046.html - might be best to discuss in bug 39402.
  • I wonder whether it should also notify on Abandoned ("Somebody could pick up the work" notification in the bug report)?

We could turn on notifs for abandoned/restored (I was thinking those might be useful), in addition to merges.

Per the feedback I got yesterday, we still need to tune down the noisiness on new patches (we only want notifs on new changesets, not each patchset attached to a change)

(In reply to comment #22)

We could turn on notifs for abandoned/restored (I was thinking those might be
useful), in addition to merges.

See https://gerrit.wikimedia.org/r/55048

Shouldn't Jenkins' messages be filtered out?

We did disable notifications on comments.

I was asking because of bug 34557 comment 5

(In reply to comment #22)

Per the feedback I got yesterday, we still need to tune down the noisiness on
new patches (we only want notifs on new changesets, not each patchset
attached
to a change)

See https://gerrit-review.googlesource.com/#/c/43843/

This is now enabled again, with the following config (based on feedback):

Do notify on:

  • New change
  • Restored change
  • Abandoned change
  • Merged change

Do NOT notify on:

  • New patchset
  • New comment

Basically, leave a comment when there's something you should check out in Gerrit (someone's started work, someone's giving up, or work is done), but not during the normal review cycle (comments & patchets).

I hope this meets everyone's approval.

(In reply to comment #28)

This is now enabled again, with the following config (based on feedback):

Do notify on:

  • New change
  • Restored change
  • Abandoned change
  • Merged change

Do NOT notify on:

  • New patchset
  • New comment

Basically, leave a comment when there's something you should check out in
Gerrit (someone's started work, someone's giving up, or work is done), but
not during the normal review cycle (comments & patchets).

I hope this meets everyone's approval.

Of course this doesn't trigger when a new patchset adds a new reference to a bug, which is mildly irritating. :-)

(In reply to comment #29)

(In reply to comment #28)

This is now enabled again, with the following config (based on feedback):

Do notify on:

  • New change
  • Restored change
  • Abandoned change
  • Merged change

Do NOT notify on:

  • New patchset
  • New comment

Basically, leave a comment when there's something you should check out in
Gerrit (someone's started work, someone's giving up, or work is done), but
not during the normal review cycle (comments & patchets).

I hope this meets everyone's approval.

Of course this doesn't trigger when a new patchset adds a new reference to a
bug, which is mildly irritating. :-)

This was by request, as many changes go through many patch iterations before merging.

(In reply to comment #30)

(In reply to comment #29)

Of course this doesn't trigger when a new patchset adds a new reference to a
bug, which is mildly irritating. :-)

This was by request, as many changes go through many patch iterations before
merging.

Sure; I assume there's no way to trigger only on new-patchset-which-has-newly-added-bug-reference?

J.

(In reply to comment #31)

(In reply to comment #30)

(In reply to comment #29)

Of course this doesn't trigger when a new patchset adds a new reference to a
bug, which is mildly irritating. :-)

This was by request, as many changes go through many patch iterations before
merging.

Sure; I assume there's no way to trigger only on
new-patchset-which-has-newly-added-bug-reference?

J.

I can't think of a non-hacky way that would allow me to sleep at night.

(In reply to comment #32)

(In reply to comment #31)

(In reply to comment #30)

(In reply to comment #29)

Of course this doesn't trigger when a new patchset adds a new reference to a
bug, which is mildly irritating. :-)

I can't think of a non-hacky way that would allow me to sleep at night.

Would
https://gerrit-review.googlesource.com/#/c/45201/
allow you to sleep at night?

Should be fixed since we upgraded gerrit and hooks-bugzilla on 2013-06-27.