Page MenuHomePhabricator

Story: WikimetricsUser reports pages edited by cohort {kudu} [13 pts]
Closed, ResolvedPublic

Description

Story
As a Program leader or grant recipient Wikimetrics user, I want to be able to report on the number of pages on a wiki that the members of my cohort edited during a specified timeframe so that I can report on the global metric "number of articles created or improved in Wikimedia projects".

Pages edited option is made available when the user selects metrics to report for their cohort. The report can contain individual and aggregated results, filtered by namespace, just like the pages created metric.

Notes
Needed for January 2015 Grants reporting standards for Global metrics. Specifically to answer the number of articles improved and for analysis of topical content generation efforts re diversity and gaps.
if necessary, a template query in Quarry could be used instead.

SOLUTION:
Qarry is able to provide the numbers needed for the Grantmaking team.

Details

Reference
bz73072

Event Timeline

bzimport raised the priority of this task from to Needs Triage.Nov 22 2014, 3:53 AM
bzimport set Reference to bz73072.

Ignore description above. I cut and pasted the wrong thing.

DESCRIPTION:
Story
As a Program leader or grant recipient Wikimetrics user, I want to be able to report on the number of pages on a wiki that the members of my cohort edited during a specified timeframe so that I can report on the global metric "number of articles created or improved in Wikimedia projects".

Pages edited option is made available when the user selects metrics to report for their cohort. The report can contain individual and aggregated results, filtered by namespace, just like the pages created metric.

Notes
Needed for January 2015 Grants reporting standards for Global metrics. Specifically to answer the number of articles improved and for analysis of topical content generation efforts re diversity and gaps.
if necessary, a template query in Quarry could be used instead.

Change 174773 had a related patch set uploaded by Mforns:
Add pages edited metric

https://gerrit.wikimedia.org/r/174773

kevinator renamed this task from Story: WikimetricsUser reports pages edited by cohort to Story: WikimetricsUser reports pages edited by cohort [13pts].Nov 26 2014, 1:02 AM
kevinator lowered the priority of this task from High to Medium.Dec 1 2014, 3:53 PM
kevinator raised the priority of this task from Medium to High.Dec 9 2014, 12:22 AM
kevinator moved this task from Backlog to triaging-high on the Analytics-Wikimetrics board.
kevinator lowered the priority of this task from High to Low.Dec 16 2014, 12:46 AM

After much discussion with the dev team, it became apparent that this is not a simple change to Wikimetrics. It is built to sum up numbers at the reporting stage, but in this case it cannot deduplicate results. So when a report is configured to show individual results, they will be correct, but the total for the cohort will contain duplicate counts of pages that were edited.

Quarry ( http://quarry.wmflabs.org/ ) can be used in the meantime for Grantmaking's need to pull the number of pages edited.

Priority is changed to low so we can focus on more important things before re-evaluating the complexity of this task.

kevinator renamed this task from Story: WikimetricsUser reports pages edited by cohort [13pts] to Story: WikimetricsUser reports pages edited by cohort.Dec 16 2014, 12:47 AM

Hi Kevin,

This metric isn't just for Grantmaking, or even all of CE, but for every person that leads a program, especially grantees who are required to report this global metric. Unfortunately we can't expect these program leaders to learn SQL for this.

Can we please make the simple change to wikimetrics that allows you to pull the individual results? This half-solution would be better than no solution for now.

Thanks, and please do let me know if I can help with this, as it is important for L&E priorities.

@Abit:

Can we please make the simple change to wikimetrics that allows you to pull the individual results?

Would the number of pages edited "deduplicated" be good enough for you?. That means that if users #1 and user#2 edited the same page the agreggated number will be 1, not two.

Seems that if we are going for this metric: "number of articles created or improved in Wikimedia projects". We should be deduplicating, otherwise results might be missleading.

Thanks for responding, @Nuria. I may be misunderstanding "individual results." I thought it would be a list of individuals with the pages they have edited, but maybe not? This would require post-processing in a spreadsheet, but that would still be better than no metric for the program leaders.

I agree we should be deduplicating results. If we can get deduplicated results in Wikimetrics and without post-processing, even better. Would it be helpful to talk about this face to face or in a hangout? It might be the fastest way to hash out definitions.

@Abit: "individual results" is what you think it is, but we also use that term to refer to the option of outputting "individual results" from the report. The trouble is that reports have the option to output "aggregate results" (sum, avg, and std). Because of this, we have to change wikimetrics to have "deduplication" logic when it aggregates, which it hasn't done in the past. We realized this in the fall, while implementing this metric, and told Johnathan about it. We estimated it would take some additional work, and he chose to lower the priority of this issue in favor of other work. It's a feasible change that we roughly know how to do, it's just a question of priority.

Thanks for the explanation, @Milimetric, it's super helpful. I think our program leaders and evaluators would find individual results very useful, even without the option to get aggregate results. If individual results without aggregate results is an easy addition, I would be excited to support it. But I don't want to bungle your guys' priorities. I'll email Jaime and Jonathan about this and it can go through the appropriate channels.

Apologies for jumping in the middle of this thread. Thanks so much for helping me understand, and I hope we can see this global metric in Wikimetrics soon.

Change 174773 restored by Milimetric:
Add pages edited metric

Reason:
Thought of a nasty hack to make this work! :)

https://gerrit.wikimedia.org/r/174773

another thought: maybe a way to do this without wikimetrics? the wikiwomen page already has code to list the articles improved and sum them. it's not on wiki, but maybe something could write it to wiki?

http://2015.wikiwomen.in/

@Abit: I was able to figure out a hack to make this work for the program metrics. When you test it as part of that tool, you can let us know if it works the way you hope here, and we can easily make it available in the UI.

@Milimetric Woah hey, that's great news! I tried to test it using the link Madhu shared yesterday but didn't see a pages edited or pages improved metric. Am I missing something, is there another place I should go?

@Abit, no we have to deploy the code to the link that Madhu shared. In general nothing really happens automatically but we can update you more often if you'd like to stay in the loop.

Change 174773 merged by Mforns:
Add pages edited metric

https://gerrit.wikimedia.org/r/174773

Milimetric renamed this task from Story: WikimetricsUser reports pages edited by cohort to Story: WikimetricsUser reports pages edited by cohort {kudu} [13 pts].Dec 10 2015, 6:23 PM
Milimetric claimed this task.

Why this isn't displayed in the UI? Because the missing deduplication?

I believe this metric is missing from the general UI because it was only ever requested as a part of the "global program metrics" project. So if memory serves, this is one of the numbers reported there. In theory it's easy to enable this metric, but yes, the deduplication remains a confusing complication in using/explaining this metric in the general UI.