Page MenuHomePhabricator

Implement Flow support in Pywikibot
Closed, ResolvedPublic

Description

Flow is the new communication extension for MediaWiki, see https://www.mediawiki.org/wiki/Flow. It is not supported by Pywikibot.

See T86571: [1] O6. Schedule and prepare "Update bots to handle Flow boards" workshop for Dev Summit for notes from the devsummit.
See https://www.mediawiki.org/wiki/Flow/Pages#Engineers_.28technical_notes.29 for links to all documentation (API etc.).

The concepts in Flow need to be implemented as python classes, inheriting from the pywikibot base class BasePage for objects which exhibit 'mediawiki page' like functionality. It may be sufficient to copy the classes from PHP, however it is likely that some remodelling will be needed to mesh Flow concepts into pywikibot.

To complete this task, unit tests must be added to the test suite to interact with test Flow pages on a Flow enabled Wikimedia site (e.g. MediaWiki.org). The tests should include interacting with Flow objects using the Flow specific functionality, and interacting with the same pages using the standard Page functionality where appropriate.

The unit test should be configured to run on travis-ci, and Flow related tests run when the test site has Flow enabled, and Flow related tests be skipped when Flow isn't installed.

If time allows, the Flow support should include server version tracking, so Flow features can easily be enabled/disabled as the Flow extension is enhanced. (T110046)


GSOC Plan (based on T93991); will be updated

Goals

Obviously, I want to have working code in Pywikibot that can interface with Flow objects through the MediaWiki API by the end of the coding period. But doing that would have to involve learning the codebases of both Pywikibot and Flow, learning and using the testing system, and figuring out the API.

During this project, I hope to:

  1. Establish working relationships with the Pywikibot and Flow teams.
  2. Learn prerequisites, including the MW API, Travis-CI and the testing framework, and the respective codebases and conventions of Pywikibot and Flow.
  3. Create an initial implementation of Flow support in Pywikibot.
  4. Test frequently, including in live testing environments and using Flow-specific and generic constructs where appropriate.
  5. Finish a stable implementation of Flow support in Pywikibot.
  6. If time allows, add a version-tracking construct to enable/disable specific Flow features as appropriate.

Time

I start my summer around May 18, so I can certainly start work by then. I will start my prerequisite research and community outreach sooner than that, having it completed by the start of the coding period. I will have an initial implementation completed by the midterm evaluations, with intensive testing and bug squashing planned for the second half. I plan on documenting the code throughout the process, with the ideal goal of writing and commenting code simultaneously and writing any necessary formal documentation at major stopping points. The end of my summer vacation should be the last week-or-so of August. I will fill in any other needed details later.

Detailed timeline

Before May 25Communication with Flow and Pywikibot teams; research into architectures; learning any requisite technologies; finalizing plan
May 25 - June 7Creating new classes for Flow objects and achieving a partial implementation
June 8 - June 21Completing initial implementation of Flow mappings
June 22 - June 28Documenting new code
June 26 - July 3Midterms
July 4 - July 15Writing new unit tests
July 16 - July 26Fixing bugs found with unit tests; finalizing mapping code
July 27 - August 2Adding version capability
August 3 - August 9Additional thorough testing and documentation writing
August 10 - August 17Buffer for any possible overruns
August 18 - August 23Final evaluations
August 24Start of semester

Related Objects

StatusSubtypeAssignedTask
OpenNone
InvalidNone
Resolvedhappy5214
Duplicatehappy5214
Duplicateakashagarwal
Resolvedjayvdb
ResolvedNone
Resolvedhappy5214
Resolvedhappy5214
Resolvedhappy5214
Resolvedhappy5214
Resolvedhappy5214
Resolvedhappy5214
Resolvedhappy5214
Resolvedhappy5214
Resolvedhappy5214
Resolvedhappy5214
Resolvedhappy5214
Resolvedjayvdb
Resolvedhappy5214
Resolvedjayvdb
Resolvedhappy5214

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

I am willing to write this up as a project, if a flow developer can be a co-mentor to

CCing more Flow devs. The deadline for candidate proposals is in a week, so realistically speaking we need a yes/no... today?

Change 187066 had a related patch set uploaded (by John Vandenberg):
[WIP] Flow support I don't even know what this is

https://gerrit.wikimedia.org/r/187066

I am willing to write this up as a project, if a flow developer can be a co-mentor to

CCing more Flow devs. The deadline for candidate proposals is in a week, so realistically speaking we need a yes/no... today?

I'm willing to co-mentor this if @Legoktm would prefer to focus on other projects. This would be my primary project. I would only mentor T91655: Editor-focused dashboard if not mentoring this.

In T67119#1150448, @Mattflaschen wrote:

I'm willing to co-mentor this if @Legoktm would prefer to focus on other projects. This would be my primary project. I would only mentor T91655: Editor-focused dashboard if not mentoring this.

\o/ thanks for volunteering :) It'll be great to have a significantly more active Flow dev mentoring. And I'll always be around on IRC to help as well :)

If I can get a proposal done in time, I'd love to work on this for GSoC. I have plenty of Python and PHP experience, I have an affinity for Flow, and the fact that nobody else has jumped on makes this the perfect project for me. I'm already working on T69000, and though I have an assignment due tonight and at least four hours of classes tomorrow, I'll do as much as I can possibly do to have my proposal accepted!

@happy5214 , could you do a very basic proposal to get it 'in', with at least links to your existing Python and PHP work (e.g. gerrit, github account, etc). This should take you 30 mins max to register in melange.

You can then continue to expand your proposal (in your Phabricator task) as time allows, up until we make a final decision.

If I can get a proposal done in time, I'd love to work on this for GSoC.

Great, I look forward to reviewing your application

I have plenty of Python and PHP experience, I have an affinity for Flow, and the fact that nobody else has jumped on makes this the perfect project for me. I'm already working on T69000, and though I have an assignment due tonight and at least four hours of classes tomorrow, I'll do as much as I can possibly do to have my proposal accepted!

I've replied to your questions at T69000: Flow: failed API call doesn't provide exception message.

Hi @jayvdb, @Mattflaschen, @Legoktm , thanks for volunteering to mentor for this project and putting up a clear description for it.

I am a bit late at this but will try to iterate on my proposal based on your feedback. Currently, I have tried to quickly put in, my understanding of Flow and what needs to be done to implement its support in Pywikibot.

Also, I plan to work on more micro tasks and go through the code base / testing framework soon.

This is probably later than it should have been, but my goals for the next two weeks are to:

  1. Continue to introduce myself to the wider Flow and Pywikibot communities, including having a presence in both teams' IRC channels.
  2. Learn general coding practices for Pywikibot, become familiar with the designs of Flow and Pywikibot, and fill in any gaps in needed technological knowledge.
  3. Use the exposed Flow API and underlying architecture, Pywikibot design principles, and common sense to create a design for new Pywikibot classes that accurately reflect core Flow entities and fit into the existing Pywikibot framework.
  4. Refine a schedule for the summer, including meeting times, methods of communication, work times, and weekly goals.
  5. Complete the GSoC administrative tasks due before the start of coding, including submitting tax forms and setting up a Payoneer account.

I hope this works.

A sixth point for the above plan will be added as a subtask shortly. I will add on to point 3 by creating a diagram for basic Flow entities to be mapped in Pywikibot that reflects the planned type hierarchy and related Flow API calls.

I've imported parts of the proposal from T93991. Appropriate edits can of course be made (that's part of #4).

This is exciting!

and interacting with the same [Flow] pages using the standard Page functionality where appropriate.

create a design for new Pywikibot classes that accurately reflect core Flow entities and fit into the existing Pywikibot framework.

You eventually need to decide what should happen when Pywikibot high-level Page operations interact with a Flow board or topic. Flow has to make similar decisions for which, if any, MediaWiki page APIs should "just work" on a Flow board or topic; see Flow API architecture page and the discussion in T59989 and T85197.

Hello. The weekly reports task for this project has still not been created. Please check T100998: [tracking] GSoC 2015 and Outreachy 10 Weekly reports and create it asap.

@happy5214 Let me know if you have questions about the weekly reports. They now need to be submitted for "25 May to 31 May", "1 June to 7 June", soon " 8 June to 14 June", then weekly after that. You should first file a 'Weekly reports for 'Implement Flow support in Pywikibot'" subtask of T100998: [tracking] GSoC 2015 and Outreachy 10 Weekly reports, then individual reports should be subtasks of that.

Hi @happy5214, I notice you haven't yet filed your evaluation in Melange. Please do so ASAP. Failing to do so will result in automatic disqualification from the program.

Hello!

End of GSoC is fast approaching. 17 August is "Suggested pencils down" deadline and 21 August is "Firm pencils down" deadline. It is expected that you don't dive into new features which might take longer than two weeks to complete and instead work on polishing up your project, testing thoroughly and getting your code merged into the main branch. I hope this project is almost complete so you can merge it and make it available to everyone as quickly as possible. :)

A few questions (for both mentors and student):

  • Are you confident in completing the project on time?
  • By when do you think you can merge the code, if at all?
  • Are there any major blockers or important missing features?

We are looking for projects which are (nearly) complete to feature on our post on Wikimedia and Google OSPO's blogs (for example: http://google-opensource.blogspot.in/2015/02/google-summer-of-code-wrap-up-processing.html). If you're interested in getting yours up there, hurry up and get this finished!

The hard deadline on getting code merged is September. T101393: Goal: All completed GSoC and Outreachy projects have code merged and deployed by September for details.

We'll be asking the students to demo their projects towards the end of the program as well.

Good luck!

In T67119#1514643, @NiharikaKohli wrote:

We'll be asking the students to demo their projects towards the end of the program as well.

@happy5214 Please think about what a good demo would be. It might just be a POC, but should clearly indicate the functionality available to bots.

Hi, I have associated two blocked-by tasks with this project.

For the student:

  1. Please go through the checklist in the end-term evaluation and fill out the fields which require any links. The checkboxes are for the mentor(s) only. Adding information on the past projects page is your task.
  2. Ensure that you have completed all the items listed in the end-term evaluation task. If there's a strong reason about why a particular item was not completed, please comment on the task and we shall look into it.
  3. Wrap-up report is mandatory and so is a demo-able link to the project (either in production or in a demo server).
  4. If you want your project to be featured in the blogpost on the Google OSPO blog, kindly comment back with a short, catchy description of the project along with a screenshot.

Change 187066 abandoned by John Vandenberg:
[WIP] Flow support I don't even know what this is

Reason:
Pywikibot now has Flow support

https://gerrit.wikimedia.org/r/187066

While there is still a lot of Flow functionality not yet supported, the foundations are in place and solid, which is what this task aimed to achieve. The project can remain open.