Page MenuHomePhabricator

Migrate Pywikibot bugs from Sourceforge to Bugzilla (set up product etc)
Closed, ResolvedPublic

Details

Reference
bz52692

Event Timeline

bzimport raised the priority of this task from to High.Nov 22 2014, 2:03 AM
bzimport set Reference to bz52692.

Required information: https://www.mediawiki.org/wiki/Bug_management/Project_Maintainers#To_add_a_project_or_component

Please agree on the components you would like to have (plus descriptions for them, and default CCs for them).

For mailing list configuration: Bugmail will be sent

From: bugzilla-daemon@wikimedia.org

"Pywikipedia-bugs@lists.wikimedia.org" it's the mailing list

Account created in Bugzilla. Set to "disabled" so nobody can log in via it (or trigger a password reset request that would end up on the public mailing list), but it will still receive bugmail once in use.

Next step is to agree on a list of components to create in Bugzilla, under the new product, and defining default assignees (one email address per component, could be the default "Nobody") and default CC (can be several email addresses per component).

Good, I am going to talk to people and start to discuss on components

Legoktm started work on a migration script here: https://github.com/pywikibot/sf-export

André, if I remember correctly, your preferred way of importing bugs was to just use the API and create new bugs, instead of (for example) using an XML import. Did I remember that correctly? :-)

If the JSON/XML-RPC API allows what you plan to do, this would definitely be the prefered way to interact with Bugzilla.

For WebService related stuff see http://www.bugzilla.org/docs/4.2/en/html/api/index.html however I'm afraid I won't be a big help with that (only got basic knowledge).

I'm basically waiting for anybody to tell me "Go ahead and create the components in Bugzilla, here is the list that we agreed on". Just for your info. :)

Dear Andre, Legoktm made a repository for migration to bugzilla, you can see the codes and make some comments: https://git.wikimedia.org/summary/pywikibot%2Fsf-export.git;jsessionid=15o181f6ixjjm1tiqekojib2l1
we are discussing on some aspects of the migration here:
https://gerrit.wikimedia.org/r/81176

we talk about the migration almost every day in IRC

Be sure we'll do it :)

best

Basically, the initial components would be:

components = [	
    'category.py',	
    'copyright.py',	
    'Cosmetic changes',	
    'General',	
    'i18n',	
    'interwiki.py',	
    'login.py',	
    'network',	
    'redirect.py',	
    'solve_disambiguation.py',	
    'weblinkchecker.py',	
    'Wikidata',	
]

I think it would be good to create the project and these components already, as that would allow testing the migration etc.

(In reply to comment #10)

Basically, the initial components would be:

components = [    
    'category.py',    
    'copyright.py',    
    'Cosmetic changes',    
    'General',    
    'i18n',    
    'interwiki.py',    
    'login.py',    
    'network',    
    'redirect.py',    
    'solve_disambiguation.py',    
    'weblinkchecker.py',    
    'Wikidata',    
]

I think it would be good to create the project and these components already,
as
that would allow testing the migration etc.

Agreed (I wish we could like comments here :D)

Product and initial components created:

https://bugzilla.wikimedia.org/describecomponents.cgi?product=Pywikipedia

with Pywikipedia-bugs@ as default assignee, and nobody in the default CC.

I still need descriptions for the components. Did not find out how to get a list of existing labels for a project in SourceForge.net's tracker. :(

{1}.py: 'bugs related to the {1}.py script'
cosmetic changes: 'bugs related to cosmetic changes applied during edits'
i18n: 'bugs related to localization/internationalization: for example templates used on different sites and translatewiki integration'
network: 'bugs related to the communication between the bot and the server - for example HTTP or API errors'
wikidata: 'bugs related to the wikidata implementation or wikidata-related scripts'

It's pretty hard to come up with good descriptions, though, so I can imagine someone else being able to think of something better.

Per some discussions in the mailing list the official name of the project is Pywikibot, can you change it?

Best

(In reply to comment #15)

the official name of the project is Pywikibot

Fixed. Sorry, got confused. :-/

(In reply to comment #16)

(In reply to comment #15)

the official name of the project is Pywikibot

Fixed. Sorry, got confused. :-/

Thank you :)

Any news on the import?
Andre, can you please set the emailformat to text only?

(In reply to comment #18)

Any news on the import?

I myself cannot offer any news as I don't know the Sourceforge API and as I only know a little bit the Bugzilla API. I've pointed to http://www.bugzilla.org/docs/4.2/en/html/api/Bugzilla/WebService for anybody who would like to try writing a script, or https://github.com/awjrichards/bingle for an example of interacting with Bugzilla's API.
If this is overkill (I honestly don't know about how many tickets we talk, e.g. also closed ones?) I'm also happy to help transfering tickets manually.

Andre, can you please set the emailformat to text only?

That's a user setting, see section "Preferred email format" on https://bugzilla.wikimedia.org/userprefs.cgi?tab=settings

Hi Andre,

(In reply to comment #19)

Andre, can you please set the emailformat to text only?

That's a user setting, see section "Preferred email format" on
https://bugzilla.wikimedia.org/userprefs.cgi?tab=settings

I know, but I can't change it for emails send to Pywikipedia-bugs@lists.wikimedia.org . I believe you're the owner of that account.

legoktm is working on the export - see https://github.com/wikimedia/pywikibot-sf-export for the repository.

The number of bugs is relatively small (100, maybe a few 100) as we are not transferring closed bugs. We could therefore choose to fix the metadata afterwards manually, instead of trying to get it right in the bot.

(In reply to comment #20)

I know, but I can't change it for emails send to
Pywikipedia-bugs@lists.wikimedia.org

Oops, I misunderstood. It should be fixed now.

Ok, script is all ready, tried importing a bug (bug 54415!) which basically worked until I ran into bug 54416. Once that is taken care of, it should be reasonably simple to import the rest. I already have the OAuth keys for sf.net set up, so I'll take care of running the script.

dzahn has merged Legoktm's patch so a SourceForge URL can be set if they do not end in a slash /.

Update:

I imported all of the patches from sourceforge. It mainly worked, except that it didn't add a comment on the sf tickets pointing to bugzilla. Not sure why...

Some patches failed fetching, but it should be possible for us to pull which ones from the log...worst case we have to check the sourceforge link and get the patch from there manually.

dr.trigon wrote:

Has somebody experience of doing the same with JIRA tickets?

Done!

I sent an email to the list as well: http://lists.wikimedia.org/pipermail/pywikipedia-l/2013-October/008393.html

I have a 1.6MB debug log file if anyone wants ;-)

Thanks everybody, great work!

One last thing that would be nice: could we get a version selector, to select between 'core (2.0)', 'compat (1.0)' and 'unspecified'?

(In reply to comment #27)

I have a 1.6MB debug log file if anyone wants ;-)

Any private information in there? If not, please attach it here for reference.

By the way Legoktm you're a crazy martyr, why didn't you use an alternate account for filing? :O Luckily with 5.0 you'll be able to ignore all those bugs when you get tired of them https://bugzilla.mozilla.org/show_bug.cgi?id=148564

The escaping on comments is rather awful, but not a tragedy.

Is the project on sourceforge going to stay forever? Many bugs originally had patches and other stuff necessary to understand or act on the report, but the attachments have not been imported.

Also, where should I propose new components? Maybe a wiki page for the first bunch?

(In reply to comment #29)

could we get a version selector, to select between 'core (2.0)',
'compat (1.0)' and 'unspecified'?

Added.

Migration has been finished; closing as FIXED. Followup issues can be handled in separate reports and/or talk pages, mailing lists.