I do not think there is a way to easily find out who has +2 on
projects.

• totally unfeasible approach below ---

If you absolutely have to find out, you can do by hand by going
through the projects settings (and parent projects recursively),
resolve the ACLs by hand, and resolve the groups with +2
permission to contributors. But that's a huge amount of work.
And note that while resolving the groups, gerrit is hiding groups that
you have no permission to see. So you might miss groups, if you
haven't got sufficient permissions.
And note that the ls-members (even with --recursive) does not expand
ldap groups.

Isn't https://gerrit.wikimedia.org/r/#/admin/groups/11,members the (glibal) list of folks with +2 in Gerrit, or am I wrong?

Maybe a first approach could be to list the people that have used the +2 rights. With the actual data gathered from gerrit I think it should be too difficult.

Let me try:

mysql -u root acs_gerrit_mediawiki_2428

mysql> select count(distinct(changed_by)) from changes where new_value='2';
+-----------------------------+

+-----------------------------+

+-----------------------------+

149 people have used the +2 rights. We have join with upeople_companies table to search for its affiliations.

mysql> select count(distinct(changes.id)) as total, changed_by, people.name, company_id, companies.name from changes, people_upeople, people, acs_cvsanaly_mediawiki_2428.upeople_companies, acs_cvsanaly_mediawiki_2428.companies where people.id = people_upeople.people_id and changes.changed_by=people_upeople.people_id and upeople_companies.upeople_id = people_upeople.upeople_id and upeople_companies.company_id = companies.id and new_value='2' group by changed_by order by total desc limit 20;
+-------+------------+---------------------+------------+-----------------------+

+-------+------------+---------------------+------------+-----------------------+

+-------+------------+---------------------+------------+-----------------------+

With this query you have the top20 +2 reviewers with the org.

If we want to exclude Wikimedia:

mysql> select count(distinct(changes.id)) as total, changed_by, people.name, company_id, companies.name from changes, people_upeople, people, acs_cvsanaly_mediawiki_2428.upeople_companies, acs_cvsanaly_mediawiki_2428.companies where people.id = people_upeople.people_id and changes.changed_by=people_upeople.people_id and upeople_companies.upeople_id = people_upeople.upeople_id and upeople_companies.company_id = companies.id and new_value='2' and companies.name not like '%Wikimedia%' group by changed_by order by total desc limit 20;
+-------+------------+------------------+------------+-------------------+

+-------+------------+------------------+------------+-------------------+

+-------+------------+------------------+------------+-------------------+
20 rows in set (0.41 sec)

We can create querterly reports also from this core SQL query.

Something like that Quim?

(In reply to comment #2)

Isn't https://gerrit.wikimedia.org/r/#/admin/groups/11,members the (glibal)
list of folks with +2 in Gerrit [...]

For projects underneath mediawiki/* group #11 (mediawiki) should have
+2. But they are not the only one, and they do not have +2 on all
branches (see refs/heads/wmf/* section of [1]). There mediawiki only
has +1 and for example wmf-deployment and ldap/ops have +2.

And looking also outside of mediawiki/* ... All-Projects (the root of
the hierarchy [2]) comes with +2 for “Project Owner”. “Project Owner” is
(typically) different for each repository. And some projects again
override this.

That's only the tip of the iceberg :-/

[1] https://gerrit.wikimedia.org/r/#/admin/projects/mediawiki,access
[2] https://gerrit.wikimedia.org/r/#/admin/projects/All-Projects,access

Alvaro, I think your approach is very good. *Exercising* +2 rights is what matters about *having* +2 rights.

Useful data that we will use *tomorrow* at the Enginnering Community team quarterly review, if available:

(The scope is July - September 2013. All the better if we can have results of the previous three quarters in order to have a one-year view)

• Total amount of users that exercised +2 rights.

• Full list of users with affiliation.

This will be also useful to identify more "unknown". For instance:

jenkins-bot: shouldn't be counted
IAlex: independent
Santosh: WMF
Petr Onderka: independent (GSoC student, yay!)
Ladsgroup: independent

We might still fall in situation where two email addresses belong to a same person, or Contributor X was independent a year ago and now works at the WMF, but exposing this data will help us fine tuning user data.

Getting active +2 contributors right is a good goal per se. Setting priority accordingly. It is ok to let other tasks aside in order to get this data asap. Thank you very much for the flexibility!

Sorry for the bugmailspam.

Quim, i need to review the companies mapping because there are some probs in the data. Once I debug it, I will provide the data you need. I am sorry not to have the data available for today!

Ok Quim, finally I have found time for gathering the results.

From Jul to Sep 2013:

• 34 people with +2 actions not from WMF
• The complete list below

mysql> select count(distinct(changes.id)) as total, changed_by, people_upeople.upeople_id, people.name, company_id, companies.name from changes, people_upeople, people, acs_cvsanaly_mediawiki_2428.upeople_companies, acs_cvsanaly_mediawiki_2428.companies where people.id = people_upeople.people_id and changes.changed_by=people_upeople.people_id and upeople_companies.upeople_id = people_upeople.upeople_id and upeople_companies.company_id = companies.id and new_value='2' and changed_on > '2013-06' and changed_on < '2013-10' and companies.name <>'Wikimedia Deutschland' and companies.name<>'Wikimedia Foundation'group by changed_by order by total desc;
+-------+------------+------------+------------------+------------+-------------------+

+-------+------------+------------+------------------+------------+-------------------+

+-------+------------+------------+------------------+------------+-------------------+
34 rows in set, 2 warnings (0.63 sec)

I hope it is useful for you!

(In reply to comment #8)

59	94	529	Santhosh	172	Unknown

might work for WMF if it's https://wikimediafoundation.org/wiki/User:Sthottingal

20	159	537	KartikMistry	172	Unknown

works for WMF: https://wikimediafoundation.org/wiki/User:KMistry_%28WMF%29

1	235	98	QChris	3054	Independent

works for WMF as contractor so depends on how you interpret it; http://lists.wikimedia.org/pipermail/wikitech-l/2012-February/058294.html

29	21	417	Brian Wolff	3054
3	81	551	Nischayn22	3054

FYI, have worked for WMF in the past

Ok, thank you very much! With this I have enough for today. Enjoy your weekend! This task is also useful to resolve the Gerrit review queue key performance indicator: https://www.mediawiki.org/wiki/Community_metrics#Gerrit_review_queue

I think the best approach for the unclear cases is to ask them to fill The Google Form where thy can provide details such as the affiliation.

The "cox", "btinternet", and "googlemail" entries can be reassigned as "Independent".

Also, Wikimedia Germany seems to be missing here, since they have +2 developers but they are not WMF.

With this SQL query you can get the information for all quarters:

SELECT COUNT(DISTINCT(changes.id)) AS total, QUARTER(changed_on) as quarter, YEAR(changed_on) year, changed_by,

people_upeople.upeople_id, people.name, company_id, companies.name

FROM changes, people_upeople, people, acs_cvsanaly_mediawiki_2428.upeople_companies, acs_cvsanaly_mediawiki_2428.companies
WHERE people.id = people_upeople.people_id AND changes.changed_by=people_upeople.people_id AND upeople_companies.upeople_id = people_upeople.upeople_id
AND upeople_companies.company_id = companies.id
AND new_value='2'
AND companies.name <>'Wikimedia Deutschland' and companies.name<>'Wikimedia Foundation'
GROUP BY QUARTER(changed_on), YEAR(changed_on), changed_by ORDER BY year, quarter, total DESC;

+-------+---------+------+------------+------------+---------------------+------------+-------------------+

+-------+---------+------+------------+------------+---------------------+------------+-------------------+

....

...
185 rows in set (0.48 sec)

As you said it is a pretty good query to debug companies mapping!

(In reply to comment #8)

Ok Quim, finally I have found time for gathering the results.

From Jul to Sep 2013:

• 34 people with +2 actions not from WMF
• The complete list below

mysql> select count(distinct(changes.id)) as total, changed_by,
people_upeople.upeople_id, people.name, company_id, companies.name from
changes, people_upeople, people,
acs_cvsanaly_mediawiki_2428.upeople_companies,
acs_cvsanaly_mediawiki_2428.companies where people.id =
people_upeople.people_id and changes.changed_by=people_upeople.people_id and
upeople_companies.upeople_id = people_upeople.upeople_id and
upeople_companies.company_id = companies.id and new_value='2' and changed_on

'2013-06' and changed_on < '2013-10' and companies.name <>'Wikimedia
Deutschland' and companies.name<>'Wikimedia Foundation'group by changed_by
order by total desc;

Guys, this query was not correct because "changed_on > '2013-06' and changed_on < '2013-10'". This should be "changed_on > '2013-06-31' and changed_on < '2013-10-01'".

It is better for quarter analysis to use:

"QUARTER(changed_on) = 3 and YEAR(changed_on) = '2013'"

(In reply to comment #12)

(In reply to comment #8)

Ok Quim, finally I have found time for gathering the results.

From Jul to Sep 2013:

• 34 people with +2 actions not from WMF

With the correct query this number is:

• 49 people with +2 actions not from WMF in 2013 Q3.

I am sorry about the noise!

• The complete list below

mysql> select count(distinct(changes.id)) as total, changed_by,
people_upeople.upeople_id, people.name, company_id, companies.name from
changes, people_upeople, people,
acs_cvsanaly_mediawiki_2428.upeople_companies,
acs_cvsanaly_mediawiki_2428.companies where people.id =
people_upeople.people_id and changes.changed_by=people_upeople.people_id and
upeople_companies.upeople_id = people_upeople.upeople_id and
upeople_companies.company_id = companies.id and new_value='2' and changed_on

'2013-06' and changed_on < '2013-10' and companies.name <>'Wikimedia
Deutschland' and companies.name<>'Wikimedia Foundation'group by changed_by
order by total desc;

Guys, this query was not correct because "changed_on > '2013-06' and
changed_on
< '2013-10'". This should be "changed_on > '2013-06-31' and changed_on <
'2013-10-01'".

It is better for quarter analysis to use:

"QUARTER(changed_on) = 3 and YEAR(changed_on) = '2013'"

Note that self-merges must not be counted as +2 actions, see bug 53485 comment 35 and 39 for references.

Ok, we need to find a way to get this data in http://korma.wmflabs.org/browser/gerrit_review_queue.html

What about a table like "Reviews merged"at http://korma.wmflabs.org/browser/who_contributes_code.html but counting the amount of people exercising +2 every month, by organization?

Let's change this to low until we really know what we need. It is unclear now. Ideas welcome.

Maybe a way to approach this task is to think what do we want to report in the ECT quarterly reviews.

This entire idea is flawed because it does not account for multiple affiliations (e.g. Independent and WMF/WMDE).

The list of people is short, and we can look at the details when we have it. Potentially flawed data is still better than no data, imho.

As for "Percentage of users that have exercised +2 OR -2 by affiliation in a given month", I think that no data is better. What's the point?

To see whether there is any trend over time, mainly WMF vs the rest. I think a community with 50% of non-WMF users actively using their +2 is more desirable and healthier than a community where basically all commits go through WMF employees.

But hey, if I'm the only one finding this data point interesting, no problem. What really matter is to see the trend of users exercising their +2 powers.

In T59038#968641, @Qgil wrote:

To see whether there is any trend over time, mainly WMF vs the rest. I think a community with 50% of non-WMF users actively using their +2 is more desirable and healthier than a community where basically all commits go through WMF employees.

As I thought. :) I agree this is interesting, but perhaps more easily seen by measuring how many +2 come from users who qualified themselves as "Independent" first (this delegates "independency" considerations to each user). This way we don't need to worry about all affiliations, nor about precise numbers given to each affiliation.

It's not that we have dozens of types of affiliation. In order to understand "Independent" we need to see the magnitude of "Unknown", because otherwise we would assume that !independent == professionals, which is probably not true. Once you have Independent and Unknown, it is not difficult to identify WMDE and WMF. Is there any other affiliation likely to have +2 members?

Once we have these numbers and a list of names, then we can discuss details like people with more than one affiliation if needed.

That's not quite a detail. The WMF people doing code review for non-WMF people are typically those who were active in MediaWiki before being in WMF, or who consider themselves independent/MediaWiki first, or who have other clients/projects.

If you don't collect this information, considering it a "detail", you have good chances to miss the only important driving factor and possible way for improvement we have.

In T59038#972901, @Nemo_bis wrote:

That's not quite a detail. The WMF people doing code review for non-WMF people are typically those who were active in MediaWiki before being in WMF, or who consider themselves independent/MediaWiki first, or who have other clients/projects.

Less dramatic: "The people who do the most code review are those that have been around the longest and have the most experience."

Nemo, I don't understand your arguments in this task. First you say that it is not worth analyzing data by affiliation. Then you make a statement about WMF employees reviewing changes contributed by independents versus those who don't, which (true or not) it shows that you care about affiliations indeed.

In order to have data to reply to complex questions like "Are WMF developers prioritizing WMF patches / discriminating independent patches?" (which seems to be your underlying assumption), we need some building blocks first. I will keep focusing on these basic building blocks before speculating about the complex questions.

I have edited the description proposing the data we want to gather in this task. Suggestions for improvement are welcome.

Let's start with this:

• A table with the full list of contributors who have exercised +2 OR -2 in Gerrit (name, primary affiliation, number of changes resolved) in the past month.
• A graph showing total number of contributors who have exercised +2 OR -2 each month.
  • X: months
  • Y = number of contributors
  • Tooltip shows the data for each month

Hi,

I've been working a bit on this. Let's have a look at first results just to be sure that we're working on the right direction and check that the dataset makes sense :).

First, this is the analysis of active core reviewers. An active code reviewer is defined as those people that can exercise a +2 or -2 in the review process.

The attached chart shows the evolution of the different people that exercised that review in 1,206 gerrit projects available in the database in 2013, 2014 and 2015 in a monthly basis.

In addition, the following list shows the list of people that exercised that action and that number of actions. In this table I'm missing affiliations just to check. I have also removed the Translation bot. I have also added only those with at least 30 different +2/-2 actions.

• People exercising a +2/-2 action in Gerrit between 2015-01-18 and 2015-02-18.

[...]

These results were retrieved using a python script, not sure if I should upload that here as an example of use.

Thank you, this is going in the right direction.

About the graphic, I wonder why there is such spike on April 2013. The rest looks good.

About the table, for this +2/-2 metric I think it is worth listing absolutely everybody, with affiliation.

You can use Paste to share simple scripts, and then you can include those pastes here with this format: {P123}

I have also removed the Translation bot.

Why did you need to remove l10n-bot? It's not needed, if you exclude self-merges from counts (which you really must).

Oh well, it's not a big deal. I just removed what seemed to me bots.

In the following iteration I'll keep that info in the list and double check the self-merges.

Thanks for the pointers!

Hi, new numbers and list of developers.

First, I've been working on removing self-merges. These are defined as reviews that are merged by the same developer that submitted the changeset. There may be cases where in the review process third parties upload new patchsets. Those third party developers were not traced in this analysis.

As mentioned by @Nemo_bis, removing the self-merges removed the bots. Numbers are a bit lower than previous chart.

In addition the list of developers that at some point during the last month added a +2 or -2 together with their affiliation.

Code available at https://phabricator.wikimedia.org/P329

The work described in T59038#1017504 is covered by this graphic and this table. Thank you! Now we need to decide where to publish it.

There may be cases where in the review process third parties upload new patchsets. Those third party developers were not traced in this analysis.

Good point. This needs to be explained somewhere in korma when we publish this metric.

As the following step, the defined chart and table will be part of the dashboard in the code review overview at http://korma.wmflabs.org/browser/scr.html.

A first prototype will be provided with information from this task (T59038: Basic metrics about contributors exercising +2/-2 permissions in Gerrit) and T86152: Active Gerrit users on a monthly basis.

Please, have a look at the first version.

In order to keep in line with upstream, the first approach is to add the list of top ten in the scr overview page. This first step can be seen in the attached image.

If we want to keep this version, we're missing the following information:

• the rest of the list up to 900 participants and some dozens of active core reviewers.
• we're missing affiliations.

Thus, in order to add the missing information. We're considering the option of adding them to the contributors page. If you check the left hidden menu, there's no current version of scr-contributors.html, but this exists for instance for scm (scm-contributors.html). So, the idea would be to add the full list of contributors (participants or core reviewers) there.

With this second step, it would be difficult to add the affiliation (let's say that this is not in general supported in the dashboard, except in some special cases for Wikimedia). But, I'd say that this is doable if this is necessary.

Well, there is a disconnect between what we had agreed in February and what we are being offered now. We don't need to discuss the information provided again, right? I understand the constraints you have keeping an upstream version, but in this case we are not really interested in who is in the top 10, we are interested in knowing whether the total amount of users exercising their +2/-2 rights increases or decreases, and what is the trend of activity of paid developers vs volunteers.

If we need to have a Wikimedia-only page for this, so be it. We will assume the cost of production and maintenance.

There is indeed a disconnect, but as I commented, we were trying to keep inline with upstream.

Metrics are clear and what you expect is what I provided in previous comments as a table and as an evolutionary chart.

Regarding to the panel, we'll produce it with exactly that information. Probably adding both metrics (participants and active core reviewers) in the same page would be enough.

Hi again!

To follow up with the discussion, please find attached the aforementioned contributors page (please, ignore the dataset you can see):

As commented in the previous comment with the first image, that would be the information added to the scr overview panel. I'd say that it's interesting to have such data there in any case.

And then, the attached image would contain the specific information you asked for (without affiliations, some comments about this at the end of this comment).

So, summarizing: the addition of participants and active core reviewers would be added into two main panels:

• Code review -> Overview, that would be similar to the first attached image.
• Code review -> Contributors, that woulud be similar to the image attached in this comment.

Next steps:

• You agree or disagree with the proposal.
• If you agree, then we have to wait to close T92953: Migrate Korma identitites database to SortingHat and T92952: Migrate Korma functionality into upstream to produce the final results.
• And we add affiliations to the tables you see in the prototype.

Alright, thank you!

About

• OK to the graph.
• OK to the two tables.

BUT I think there is a problem with the labels used, because they are difficult to understand if you just land in that page, even if you are a code review expert.

• "Participants" could be renamed to "Code review users"
• "Active core reviewers" could be renamed to "Code review committers"
• "Events" is not bad, but maybe "Actions" is clearer?
• "Merged" is inaccurate if we are counting -2 there as well. "Resolutions"?

About

• I think it is better to reuse the same graph with both data combined, because it allows to compare them better.
• OK to the tables, waiting for affiliations.

All this considering the same changes proposed for labels.

I agree with your labels-renaming suggestions. Let's use them

And with respect to the chart, not an issue to merge the two charts into one. This would be consistent with the overview panel.

Thanks for the comments!.

This task is blocked till we get updated dataset from the migration of Korma to SortingHat and upstream.

This panel is ready to go (http://korma.wmflabs.org/browser/scr-contributors.html).

Please have a look at it.

Affiliations were added and bots removed as final steps

@Qgil, we've (@Lcanasdiaz and me among others) discussed about the renames a bit more.

• "Participants" makes sense to be renamed as "Code review users"
• We disagree with the use of "Code review committers". The use of "committers" may be misunderstood with the usual "committer" found in Git. In addition to this, the term "Active core reviewers" is used by other projects such as OpenStack. We'd use "Active core reviewers".
  • Active because we only count those that are active (we do not know anything about those that have not voted a +2/-2).
  • Core because only core people can vote a +2/-2.
  • And reviewer because they are reviewers and they are not committing anything to the source code afaik.
• It's ok to use "Actions" instead of "Events".
• Regarding to Merged, you were right that this is wrong. We should use another term. We're not sure about "Resolutions" and we prefer to rename this as "votes". So probably adding "+2/-2 Votes" is enough with this respect and helps to avoid misunderstanding with other review systems.

Core is generally meant to mean MediaWiki core and the MediaWiki core group in gerrit, who can do CR almost everywhere: see https://www.mediawiki.org/w/index.php?title=Developers

Technically, people who do +2 *are* the committers in git sense: they commit the merge (at some point jenkins-bot was the committer).

These graphs are interesting.

As Nemo say, in our context a "core reviewer" is a reviewer of MediaWiki core. Not the end of the world, but if we can find a better term. They are git committers indeed, a role different than the "authors" shown in http://korma.wmflabs.org/browser/scm.html

"+2/-2 votes" works in our case with Gerrit now, but other systems don't have the same concept. Anyway, good for now.

We are not going to keep this task blocked for the labels, though. Please implement the labels agreed and close the task. We can continue the discussion about labels here if needed.

The labels were changed according to the agreement.

• Code review users instead of participants
• Code review committers instead of active core reviewers
• Actions instead of events
• +2/-2 votes instead of merged

Thanks @Lcanasdiaz!