Page MenuHomePhabricator

Jenkins: Set up generated php documentation for MediaWiki extensions
Open, LowPublic

Description

It'd be useful to have a doxygen install for extensions.

Or have the current one list extensions too

Is this possible?

Event Timeline

bzimport raised the priority of this task from to Low.Nov 21 2014, 11:24 PM
bzimport set Reference to bz25978.
bzimport added a subscriber: Unknown Object (MLST).

I think that extensions should be separate, lots of them aren't well-maitained and would just create needless noise.

+shell, cc'ing Ryan (As I imagine he'd be likely the one to sort it)

Maybe we could (ab)use @package and @subpackage for grouping them? Shouldn't be
hard to add it to the cron that updates svn.w.o/doc/

Maybe svn.w.o/extension-doc/?

Well, I'll take care of it, when we decide if we want to do it or not :)

Translate extension already has its own documentation located elsewhere with preferred configuration.

It isn't much use to create documentation for extensions which are not documented with documentation in mind. Case-by-case basis seems reasonable though, but how much work is it to maintain it

Indeed. I was after it whilst working on CodeReview, would've been good to be able to have documentation open in a different window on a different monitor

(In reply to comment #4)

Well, I'll take care of it, when we decide if we want to do it or not :)

After some discussion, we've decided this is something we want, so can you take care of this now?

Can someone describe what I'm supposed to be doing, and how I'm supposed to do it?

Moving to testing infrastructure since we want Jenkins to do this sort of stuff.

Right now our gen script has an option to include/exclude locally installed extensions, but not to exclude core itself.

Assuming the generated documentation works standalone (e.g. without including core for cross-reference in the same output) we should be able to do this.

Then we can enable the build step from the postmerge pipeline of an extension that would like this autogenerated and published to doc.wikimedia.org (like we already do for core).

For Translate extension [2] we have been using TAGFILES [1] to do cross references.

[1] TAGFILES = /home/betawiki/doxygen-tagfile.xml=http://svn.wikimedia.org/doc
[2] http://translatewiki.net/docs/Translate/html/ (but apparently the tagfile handling is currently broken)

TAGFILES is described in Doxygen documentation 'Linking to external documentation' at http://www.stack.nl/~dimitri/doxygen/manual/external.html

(In reply to Niklas Laxström from comment #5)

Translate extension already has its own documentation located elsewhere

But no longer, "Generated on Tue Oct 29 00:00:23 2013" (cf. https://gerrit.wikimedia.org/r/#/c/157791/ ).

Looks trivial to do in Tools as we already have an extensions checkout; will try later.

(In reply to Nemo from comment #13)

Looks trivial to do in Tools as we already have an extensions checkout

It is, once you throw enough memory at it. :-)
https://tools.wmflabs.org/mediawiki-mirror/html/dirs.html (I'll set up a daily update)
Configuration adapted from translatewiki.net, with some heavy features removed for faster browsing: https://tools.wmflabs.org/mediawiki-mirror/doxygen.conf
Warnings (https://tools.wmflabs.org/mediawiki-mirror/warn.doxygen.log ):

$ grep -Eo "warning:.+" warn.doxygen.log | sort | uniq -c | sort -nr | head -40

911 warning: Found unknown command `\group'
873 warning: Found unknown command `\Elastica'
743 warning: Found unknown command `\dataProvider'
574 warning: Found unknown command `\covers'
349 warning: Found unknown command `\Query'
323 warning: member with no name found.
315 warning: Found unknown command `\licence'
213 warning: Found unknown command `\access'
198 warning: Found unknown command `\expect'
174 warning: Found unknown command `\Builder'
142 warning: Found unknown command `\expectedException'
131 warning: missing argument after \category.
126 warning: no uniquely matching class member found for 
112 warning: Unexpected end of comment while inside link command
 87 warning: no matching class member found for 
 75 warning: Found unknown command `\Filter'
 74 warning: Found unknown command `\Exception'
 70 warning: Found unknown command `\Response'
 64 warning: Found unknown command `\depends'
 48 warning: Found unknown command `\InvalidException'
 47 warning: Found unknown command `\TODO'
 47 warning: Found unknown command `\Document'
 47 warning: Found unknown command `\Bulk'
 44 warning: Found unknown command `\Facet'

There is a mwext-doxygen-publish job that has been added in September 2016 ( 3ef7a9c52edc9ab58a38aff9853db1a22492763a ). I guess the repository needs a Doxyfile? Then it is all about adding the job to the postmerge pipeline.

The job is used by four repositories and their resulting documentations are:

https://doc.wikimedia.org/MinervaNeue/master/php/
https://doc.wikimedia.org/CollaborationKit/master/php/
https://doc.wikimedia.org/Kartographer/master/php/
https://doc.wikimedia.org/MobileFrontend/master/php/

Left to figure out:

  • the process to add doc publishing (just add Doxyfile?)
  • need a job that runs Doxygen when patches are proposed
  • a list of extensions for which we want the PHP documentation to be generated

Change 594464 had a related patch set uploaded (by Hashar; owner: Hashar):
[mediawiki/extensions/Thanks@master] doxygen: normalize output directory to 'docs'

https://gerrit.wikimedia.org/r/594464

Change 594465 had a related patch set uploaded (by Hashar; owner: Hashar):
[mediawiki/extensions/Translate@master] doxygen: normalize output directory to 'docs'

https://gerrit.wikimedia.org/r/594465

Repositories having a Doxyfile:

extensions/ChessBrowser/Doxyfile
extensions/CirrusSearch/Doxyfile
extensions/CollaborationKit/Doxyfile
extensions/Kartographer/Doxyfile
extensions/MobileFrontend/Doxyfile
extensions/Thanks/Doxyfile
extensions/Translate/Doxyfile
extensions/Wikibase/Doxyfile
extensions/Wikispeech/Doxyfile
skins/MinervaNeue/Doxyfile
skins/Vector/Doxyfile

Looking at them, it seems they all use more or less the same boiler plate. To avoid copy pasting, maybe we can have a shared script which generates the Doxyfile. It could be a php package published via composer or centrally maintained in mediawiki/core (maintenance/mwdocgen.php?).

Change 594465 merged by jenkins-bot:
[mediawiki/extensions/Translate@master] doxygen: normalize output directory to 'docs'

https://gerrit.wikimedia.org/r/594465

Change 594464 merged by jenkins-bot:
[mediawiki/extensions/Thanks@master] doxygen: normalize output directory to 'docs'

https://gerrit.wikimedia.org/r/594464